Scenario: I have a project which reads data from multiple different databases on different hosts with read-only access. All of them store the same type of information (for demo: Users).
Every database stores the information in their own way (different table names, column names, types etc.)
Now I want to get every User from every database and insert them into my own table 'User'.
Currently I am doing this by having 2 cronjobs - 1 imports Users & 1 updates Users (Select from Read-Only DB and insert into own DB the way I want).
Now I don't really like the idea of having multiple cronjobs running, just to import/update Users - plus I want to have the newest data in my database all the time - which means I need to run the cronjobs atleast every minute.
This with multiple tables are way too many minute-cronjobs IMO.
Question: Is there any way to achieve the same thing easier? Or is this already the correct way to do so?
Next issue is, that there is in most case no information which data is updated or not - meaning I need to import the same data over and over just to have the newest.
Any help will be gladly appreciated.
Using PHP 7+, Mysql with Symfony & Doctrine - if that helps.
