如何快速搜索1500万条记录

Question

I have excel file of 600 records, which I am going to compare with 15 million records from the database. For each excel record I need to find matches from 15 million records. currently it takes about 4 hours to complete the process. I want to minimize it at least up to 2 hours.

Answer 1

1. Export excel to CSV
2. Import CSV into mysql (LOAD)
3. Create index on the key column (the one you want to compare with the 15M records)
4. Make sure that the 15M records table has the proper index set up on the comparison key!
5. Write a simple query that joins the two table with the criteria you need.

All these points are fairly trivial and left as an exercise to the reader.