在KNIME刮板中设置“等待”

Question

I'm building a news scraper for a project, and I found my way through most of the sites, but one is giving me the headache, because whenever I try to bulk-extract the articles contents, most of html of the links won't load. I even tried in python, same obsolete results. My question is: how can I set a "wait until content is loaded"? I am reading that some Ajax thing may be needed to load first.

Answer 1

I think what you are looking for are the Selenium Nodes. They are particularly targeted for extracting data from Ajax-based websites, where content is loaded via JavaScript code.

You can find some example workflows e.g. here:

• https://nodepit.com/server/seleniumnodes.com