Use case
I want to persist a lot data in a MySQL database which I receive via a channel. For performance reasons I process them in batches of 10 items. I receive the input items only every 3 hours.
The problem
Assuming I get 10004 items, there will be 4 items left because my go routine waits for 10 items before it "flushes them away" in a batch. I want to make sure that it creates a batch with less than 10 items in case there are no more items in that channel (channel is also closed by the producer then).
Code:
// ProcessAudits sends the given audits in batches to SQL
func ProcessAudits(done <-chan bq.Audit) {
var audits []bq.Audit
for auditRow := range done {
user := auditRow.UserID.StringVal
log.Infof("Received audit %s", user)
audits = append(audits, auditRow)
if len(audits) == 10 {
upsertBigQueryAudits(audits)
audits = []bigquery.Audit{}
}
}
}
I am new to Go and I am not sure how I would properly implement that?