doudou0612 2011-04-07 10:37
浏览 106
已采纳

PHP fgetcsv()没有读取所有行

I have a php script that is reading a remote CSV file, and adding products to a database based on the contents of the CSV file. At present there are about 2800 lines (products) but the script keeps stopping at line 1388.

The code I used is as follows:

while(($data = fgetcsv($fopen, 0, ",")) !== false):
  //stuff is done here...
endwhile;

I have set the php memory limit to 64M and even tried 128M. I also set the max_execution_time to 60mins. I have also tried altering the code as follows:

while(($data = fgetcsv($fopen, 1000, ",", '')) !== false):
  //stuff is done here...
endwhile;

That DID result in more lines being parsed, BUT the data was then incorrect, i.e. image columns were becoming description columns etc. I assume that has to do with adding as my line ending. I tried , no luck. Lastly, I also added the auto_detect_line_endings as true in the ini.

Can anyone suggest reasons as to why my data is being cut short?

Regards, Simon

EDIT

I have noticed something interesting. I have a MySQL insert on each line that is looped over in the above code. Now, the last record in my database is the FIRST row in the CSV file, does this mean the file is being parsed from the last line up??

These seem to be the rows at or near the break:

W-3066,  I Love Love Cheap And Chic,     Moschino, 3.4 oz,EDT Spray,Women,,"Introduced by the design house of Moschino, I love love has a blend of grapefruit, orange, lemon, red currant, tea rose, cinnamon leaves, musk, cedar and tonka wood. It is recommended for daytime wear.",http://www.perfume-worldwide.com/products/Women/Final/W-3066large.jpg,0,0,0,8011003991457
W-3070,  Adidas Floral Dream,            Adidas,   1.7 oz,EDT Spray,Women,,"Introduced in 2008, the notes are bergamot, lily, rose, tonka bean and vanilla.",http://www.perfume-worldwide.com/products/Women/Final/W-3070large.jpg,0,0,0,3412244310024
W-3071,  Adidas Fruity Rhythm,           Adidas,   1.7 oz,EDT Spray,Women,,"Introduced in 2008, the notes are black currant, raspberry, cyclamen, freesia and musk.",http://www.perfume-worldwide.com/products/Women/Final/W-3071large.jpg,0,0,0,3412244510004

SOLUTION

As it turns out, it worked out a lot better for me to copy the file to my server, and work off the copy. The steps I followed are as follows:

  • I read the contents of the remote file using file_get_contents()
  • I then used iconv() function to re-encode data to UTF-8
  • I made a temp file using fopen(), fwrite() and fclose() functions, contents of the file was the encoded data above
  • I set the permissions of the file to 0750 using the chmod() function
  • I then applied the fgetcsv() function to my temp file
  • Did all that needed to be done
  • Deleted the temp file once done, using unlink() function

That did the trick. So, I suspect half the issue was actually the remote server timing out, and the other half encoding issues.

Thank you to everyone for all the nudges in the right direction

  • 写回答

2条回答 默认 最新

  • dongyou26216708 2011-04-07 10:46
    关注

    Firstly i have some questions for you:

    • What is on line 1388, 1388 and 1389
    • Is there any errors being outputted
    • When you reach the final line, do you get an ($data[0] === null)

    You information regarding the memory limit would probably not be the issue that's causing it, as fgetcsv reads a single line per iteration, there is only ever 1 line's worth of data in the memory at one time.

    Within your lop if your keep placing data into an array, or concatenating them together. this may cause a memory leak but you would have to show more in depth code

    A CSV File has to be pretty structured for the fgetcsv to be able to parse it correctly, some rules to remember when using CSV Files:

    • The first line must always be the column names
    • All other lines are the data lines:
      • Each element should be separated by a ,
      • If a element contains a space or a comma,' ','',' ', it should be wrapped in double quotes

    An example of a valid CSV File should be like so:

    id, firstname, lastname, age, profile_description
    0,  Robert,    Pitt,     22,  "this string has spaces, and has a comma"
    

    You should validate the the structure is correct, if it is not correct then you should fix this until the parse is able to read the data correctly, you can then cleanly place the data into a new CSV File taking care of all the little incorrect structures.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)

报告相同问题?

悬赏问题

  • ¥15 求差集那个函数有问题,有无佬可以解决
  • ¥15 【提问】基于Invest的水源涵养
  • ¥20 微信网友居然可以通过vx号找到我绑的手机号
  • ¥15 寻一个支付宝扫码远程授权登录的软件助手app
  • ¥15 解riccati方程组
  • ¥15 display:none;样式在嵌套结构中的已设置了display样式的元素上不起作用?
  • ¥15 使用rabbitMQ 消息队列作为url源进行多线程爬取时,总有几个url没有处理的问题。
  • ¥15 Ubuntu在安装序列比对软件STAR时出现报错如何解决
  • ¥50 树莓派安卓APK系统签名
  • ¥65 汇编语言除法溢出问题