（网络爬虫）如何从新闻网站获取新闻文章的文本

Question

i m going to get the text from a news website that i have to get around 1k website content

the link is on below : http://www.dcfever.com/news/readnews.php?id=16727

this website post every latest news and the new url is formed in adding 1 in the id

readnews.php?id=16727

so ,next url will be

readnews.php?id=16728

the question is i would like to scrape the text from 16000 to 17000

how to implement in Java

Jsoup? or other web crawler?

thanks

Answer 1

You've tagged it python too. Look at beautifulsoup here: https://www.crummy.com/software/BeautifulSoup/bs4/doc/