duanhunlou7051 2019-04-29 10:20
浏览 54

使用php将多个列和行的大型CSV文件导入mysql表

  1. I am trying to import particular columns of large CSV file which contain 1026 columns and 86400 rows to mysql table using PHP.
  2. The File size is 180MB.
  3. I am using LOAD DATA INFILE method.It is taking 39 seconds.I want to reduce the time to less than 2 seconds.

MySQL Table contains only 3 columns of ID,TiME,status.The code is as follows:

$servername = "localhost";
$username = "root";
$password = "signion";
$db = "In_band";
$conn = mysqli_connect($servername, $username, $password,$db);

if (!$conn) {
   die("Connection failed: " . mysqli_connect_error());
}
$x = "(ID,TIME";
for ($num=1;$num<=1024;$num++){

    $x = $x.",@status".$num;
}
$x = $x.")";

$sql = "CREATE TABLE IF NOT EXISTS table_7(ID INT,TIME TEXT,status TEXT)";
mysqli_query($conn, $sql);
$sql = "LOAD DATA INFILE '/var/lib/files/data86400_1024.csv' REPLACE INTO TABLE table_7 FILEDS TERMINATED BY ',' ENCLOSED BY '' LINES TERMINATED BY '
' IGNORE 1 LINES ".$x." set status=@status1024";

$query = mysqli_query($conn, $sql);

mysqli_close($conn);

The complete query is like this:

LOAD DATA INFILE '/var/lib/files/data86400_1024.csv' REPLACE INTO TABLE table_7 FILEDS TERMINATED BY ',' ENCLOSED BY '' LINES TERMINATED BY '
' IGNORE 1 LINES (ID,TIME,@status1,@status2,@status3,.......@status1024) set status=@status1024";

In the above code i am importing last column(status1024) of csv file to mysql table status column.

I have used fgetcsv method also.But it took 1 minute nearly.

How can i optimize importing of particular columns to mysql table in less than 2 seconds?

  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥15 微信会员卡接入微信支付商户号收款
    • ¥15 如何获取烟草零售终端数据
    • ¥15 数学建模招标中位数问题
    • ¥15 phython路径名过长报错 不知道什么问题
    • ¥15 深度学习中模型转换该怎么实现
    • ¥15 HLs设计手写数字识别程序编译通不过
    • ¥15 Stata外部命令安装问题求帮助!
    • ¥15 从键盘随机输入A-H中的一串字符串,用七段数码管方法进行绘制。提交代码及运行截图。
    • ¥15 TYPCE母转母,插入认方向
    • ¥15 如何用python向钉钉机器人发送可以放大的图片?