too long

Question

Should I be using php cli to fetch emails from imap and store them in the mysql database, when it comes to dealing with large amounts of emails. It seems fine for small sets of emails, but when you get into the thousands, using php might not be the best language to use for large sets of data processing.

My question is, should I consider moving to a different language, and if so, any recommendations on which language, or can PHP handle the massive amounts of data processing.

Maybe there is a language that handles imap communication and mysql insertions better?

Answer 1

It really depends how much data you need to process and what your budget is.

If you have low budget you could probably stick to PHP and consider forking your scripts with pcntl_fork() or/and Apache's Hadoop.

With mid-budget I'd consider more efficient C++ or different low-level which supports multi-threading. Perhaps Python, it's really up to you.

With Amazon AWS you could work out an ultimate solution using their Elasic MapReduce Data Pipeline and MySQL on RDS server or one of noSQL solutions, which soould work perfectly in IMAP environment. it'll handle whatever language you like in the same way via their API calls