I'm trying to parse some RSS feeds using Golang. A package I found was https://github.com/jteeuwen/go-pkg-rss
My process is
- Get feeds from my Postgres DB
- For each feed, fetch the articles
- Store each article into the Postgres DB
My code is as below
func main() {
db := dbConnect() // returns the DB connection variable
feeds := getRssFeeds(db) // returns feeds from my DB
for i := 0; i < len(feeds); i++ {
getFeedArticles(feeds[i].url, 5, db)
}
}
func getFeedArticles(uri string, timeout int, db *sql.DB) {
// using the package to get RSS feed contents
feed := rss.New(timeout, true, chanHandler, itemHandler)
if err := feed.Fetch(uri, nil); err != nil {
fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err)
return
}
}
func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) {
// no need to do anything...
}
func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) {
for i := 0; i < len(newitems); i++ {
fmt.Println(newitems[i].Title)
// insert the article into DB here
}
}
How do I pass the db
variable to itemHandler
so I can insert the article into my DB? I suppose I could call dbConnect()
to create a new db
variable to insert, but that would seem wasteful.
Is there a better way to do this process?