I am looking to scrape some elements from various websites that will all return an image and a description. Users of the extension will then select which websites they are interested in and a selection of images and descriptions from those sites will be displayed on their new tab page in masonry format.
The problem is that this data updates frequently, maybe a few times a day with elements appearing and disappearing.
The way I see it I have two options:
Scrape the data each time a user opens a new tab via an AJAX request, though I expect this will be incredibly slow.
Periodically scrape the data, store it in a mySQL database and fetch it from there.
Are either of these the best option, or is there something else I should consider? Perhaps there's a way to do this in the background so there's no delay when the user opens the new tab?
Do let me know if I've been unclear.