I'm trying to get colly to scrape the following page: https://www56.muenchen.de/termin/index.php?loc=BB.
Here is my code:
package main
import (
"fmt"
"log"
"github.com/gocolly/colly"
)
func main() {
c := colly.NewCollector(
colly.IgnoreRobotsTxt(),
colly.Async(false),
)
c.OnHTML("html", func(e *colly.HTMLElement) {
fmt.Println(e.Text)
})
c.OnError(func(_ *colly.Response, err error) {
log.Println("Something went wrong:", err)
})
c.Visit("https://www56.muenchen.de/termin/index.php?loc=BB")
c.OnScraped(func(r *colly.Response) {
fmt.Println("Finished")
})
}
The problem is that after the website is visited it loads some content. I'm unsure how to tell colly to "wait" until that has happened and then look at the result.
Looking forward to some ideas.