duanrui3480 2018-08-06 17:07
浏览 133

如何使用Go修复泄漏的MySQL conns?

I have data on my local computer in 2 MySQL databases (dbConnOuter and dbConnInner) that I want to process and collate into a 3rd database (dbConnTarget).

The code runs for about 17000 cycles, then stops with these error messages:

[mysql] 2018/08/06 18:20:57 packets.go:72: unexpected EOF
[mysql] 2018/08/06 18:20:57 packets.go:405: busy buffer

As far as I can tell I'm properly closing the connections where I'm reading from and I'm using Exec for writing, that I believe handles its own resources. I've also tried prepared statements, but it didn't help, the result was the same.

Below is the relevant part of my code, and it does similar database operations prior to this without any issues.

As this is one of my first experiments with Go, I can't yet see where I might be wasting my resources.

import (
    "database/sql"

    _ "github.com/go-sql-driver/mysql"
)

var dbConnOuter *sql.DB
var dbConnInner *sql.DB
var dbConnTarget *sql.DB

func main() {
    dbConnOuter = connectToDb(dbUserDataOne)
    dbConnInner = connectToDb(dbUserDataTwo)
    dbConnTarget = connectToDb(dbUserDataThree)
    // execute various db processing functions
    doStuff()
}

func connectToDb(dbUser dbUser) *sql.DB {
    dbConn, err := sql.Open("mysql", fmt.Sprintf("%v:%v@tcp(127.0.0.1:3306)/%v", dbUser.username, dbUser.password, dbUser.dbname))
    if err != nil {
        panic(err)
    }
    dbConn.SetMaxOpenConns(500)
    return dbConn
}

// omitted similar db processing functions that work just fine

func doStuff() {
    outerRes, err := dbConnOuter.Query("SELECT some outer data")
    if err != nil {
        panic(err)
    }
    defer outerRes.Close()

    for outerRes.Next() {
        outerRes.Scan(&data1)
        innerRes, err := dbConnInner.Query("SELECT some inner data using", data1)
        if err != nil {
            panic(err)
        }
        innerRes.Scan(&data2, &data3)
        innerRes.Close()

        dbConnTarget.Exec("REPLACE INTO whatever", data1, data2, data3)
    }
}
  • 写回答

0条回答 默认 最新

    报告相同问题?

    悬赏问题

    • ¥15 c语言怎么用printf(“\b \b”)与getch()实现黑框里写入与删除?
    • ¥20 怎么用dlib库的算法识别小麦病虫害
    • ¥15 华为ensp模拟器中S5700交换机在配置过程中老是反复重启
    • ¥15 java写代码遇到问题,求帮助
    • ¥15 uniapp uview http 如何实现统一的请求异常信息提示?
    • ¥15 有了解d3和topogram.js库的吗?有偿请教
    • ¥100 任意维数的K均值聚类
    • ¥15 stamps做sbas-insar,时序沉降图怎么画
    • ¥15 买了个传感器,根据商家发的代码和步骤使用但是代码报错了不会改,有没有人可以看看
    • ¥15 关于#Java#的问题,如何解决?