网站用户将数据从文件导入数据库

Question

It's necessary to allow users to import their data on the site (in mysql database). Every user who logs in, can import data from a file to the database and then work with them on the site. It takes a long time to import the data and creates a high load on the database as well. Could you please tell how to queue importing better? So the data would be imported at one online, but to queue data to be written periodically to the database. Thank you.

Answer 1

My suggestion would be to upload the .CSV file and post a message into a queue (or post the message on some other event like click on "Import" button). This way you will be able to return immediately and display a message like "Please wait for import". Then once the import is done and the database is updated the HTML page can refresh itself and display proper status. Depending on the servers and/or database load you can have only one or multiple simultaneous imports going on at the same time.