如何在Go中抓取动态网站？ [关闭]

Question

With NodeJS I can use PhantomJS to help me to scrape dynamic website, with Python I can use Selenium to do this job.

How about in GoLang?

Answer 1

Give it a try on Go lang selenium web drivers

https://github.com/knq/chromedp : Package chromedp is a faster, simpler way to drive browsers (Chrome, Edge, Safari, Android, etc) without external dependencies (ie, Selenium, PhantomJS, etc) using the Chrome Debugging Protocol.

https://github.com/sourcegraph/go-selenium : Selenium WebDriver client for Go

https://github.com/tebeka/selenium : Selenium/Webdriver client for Go

---

Static Crawler Reference

You can use github.com/PuerkitoBio/gocrawl library or built your custom crawler, basic idea is here https://tour.golang.org/concurrency/10.