当CSV太大而无法加载到内存时，PHP将utf-16le转换为UTF-8

Question

Im trying to fix a CSV using PHP.

The file is also very large 500mb. I need to convert it from utf-16le to UTF-8. Issue is I used fgets() initially and found out it doesnt work with utf-16le.

How can I convert this file to UTF-8 without using fgets? The file is too big to load to memory.

I have already searched the forum and there are a lot of ways to convert the encoding but not for a file this big that uses utf-16le.

Answer 1

The solution here is to leave this task to the operating system by the means of exec, shell_exec or bactick operator.

I used:

shell_exec ( 'iconv -f utf-16le -t utf-8 1.csv > 2.csv' );

DO NOT USE FGETS FOR utf-16le!!!!!!! It will destroy the formatting of the file.