如何通过colly中的id或class查找一个html元素或一组html元素？

Question

I am using colly for scraping website. in OnHTML callback :

package main

import (
    "fmt"
    "github.com/gocolly/colly"
)

func main() {

    // Instantiate default collector
    c := colly.NewCollector()

    // On every a element which has href attribute call callback
    c.OnHTML("h3", func(e *colly.HTMLElement) {
        link := e.Text
        // Print link
        fmt.Printf("Link found: %q -> %s
", e.Text, link)
        // Visit link found on page
        // Only those links are visited which are in AllowedDomains
        c.Visit(e.Request.AbsoluteURL(link))
    })

    // Before making a request print "Visiting ..."
    c.OnRequest(func(r *colly.Request) {
        fmt.Println("Visiting", r.URL.String())
    })

    // Start scraping on https://hackerspaces.org
    c.Visit("https://bbs.archusers.ir/")
}

for example I want to get all with 'id Name' id or get all with 'class Name'. How can I do this ?!

Answer 1

I found my answer here. really great tutorial for the colly framework.

OnHTML is a powerful tool. It can search for CSS selectors (i.e. div.my_fancy_class or #someElementId), and you can attach multiple OnHTML callbacks to your collector to handle different page types.