Is there an optimal way to improve the speed of requests to an insertAll request via the golang client?
My current structure is such that:
- Rows get submit to a queue
- Upon hitting job size, a new job is queued up (approx. 250 rows)
- A worker picks up a job from the queue
- Worker formats + sends insertAll request to BQ
The above works fairly well, however I seem to be having an average of 8-12 seconds per 250 row request.
I should mention that the number of workers being used is generally 200, however I have tried higher / lower values and it does not seem to make much a difference (higher usually seems to take longer per job).
