cURL的大数据[关闭]

Question

I own VPS server, I want to do something with my server which I have Big data about 15K usernames.

What I need, to send all usernames as cURL request with: website.com/getaccount.php?username= every minute.

so what is the solution for that case .. I don't want to make it with Cron because it will take 15K jobs and that is very bad.

in which language could I optimize 15K users ? bash ? perl ? php ? and if I can make it separate files and each file content 5K users.

Please give me a solution, thanks alot ..

Answer 1

Does all usernames change from minute to minute? If no, then it's useless to send them all. Only send differencies (deltas) like 'deleting user1' or 'adding user2' or 'renaming user1 to user2'

Also, If I understand your example correctly, you want to make a cURL request for each username, which means 15K request every minute. For me that look like a waste of resources similar to a dDos attack. I hope that is not what you want to acheive on your own server.

Maybe try to reduce your data amount to only relevant data and batch all that data in a single cUrl request!

15K isn't that big, but you need to handle it correctly!