There is a requirement to asynchronously download logs, I have two versions of the project (Server and (Hosted)Cloud).
Logs are stored in the database when the user clicks on export logs, it is downloaded in CSV format.
Currently, there is no way to export huge logs without setting a PHP memory limit. In cloud launch container is 4gb shared across 30 active threads. So, the logs have to be processed in the background and be available for download later with a notification.
There can be expiry on these logs so it's cleaned after every interval.
What could be the best approach to solve this?
Tech Stack:
- PHP
- Codeigniter
- MySQL
I have referred to this for options.
