duanbozhong9689
2015-12-02 14:53
浏览 23
已采纳

大csv文件上的fgets错误

I have got a couple of custom scripts running on my server to extract CSV files from a template. These files are sometimes 250,000 + lines. When i reach around this point I get this error.

Warning: Error while sending QUERY packet. PID=24022

This is linked to the packet size in the mysql query not been big enough.

I increased the mysql query size with the below, it seemed to improve it but not all the way.

    SHOW VARIABLES LIKE 'max_allowed_packet';
    set global max_allowed_packet=1073741824;
    set global net_buffer_length=1048576;

I am using a dedicated server and have lots of resource so i could increase this but am unsure how much is safe + not really sure if this is the best way to handle the situation

Below is the code that i am using (i have removed a lot of the code that is just confusing)

 $br = ']'; //this changes
 $handle = fopen($path."".$inputFileName, "r");
 while (($data = fgetcsv($handle, 0, "".$br."")) !== FALSE) 
     {
    ....... my code .... based on $data["...

图片转代码服务由CSDN问答提供 功能建议

我的服务器上运行了几个自定义脚本,用于从模板中提取CSV文件。 这些文件有时是250,000 +行。 当我到达这一点时,我收到此错误。

警告:发送QUERY数据包时出错。 PID = 24022

这与mysql查询中的数据包大小相关联不够大。

我增加了 使用下面的mysql查询大小,它似乎改善了它,但并非一直如此。

  SHOW VARIABLES LIKE'max_allowed_pa​​cket'; 
设置全局max_allowed_pa​​cket = 1073741824; 
设置 global net_buffer_length = 1048576; 
   
 
 

我使用的是专用服务器并拥有大量资源,所以我可以增加这个但不确定安全性多少+不确定是否 这是处理这种情况的最佳方法

下面是我正在使用的代码(我删除了很多令人困惑的代码)

  $ br =']';  //这会改变
 $ handle = fopen($ path。“”。$ inputFileName,“r”); 
 while(($ data = fgetcsv($ handle,0,“”。$ br。“”))  !== FALSE)
 {
 .......我的代码....基于$ data [“... 
   
 
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

1条回答 默认 最新

  • douba4275 2015-12-19 11:37
    已采纳

    This issue was my scripts (many) where taking a long time to run. Basically i just reconnected to Mysql. All comments were here helpful

    打赏 评论

相关推荐 更多相似问题