dongxiaoguang9108
dongxiaoguang9108
2019-04-29 21:49
浏览 63
已采纳

Golang:从切片中删除早于1h的结构

So I'm building a small utility that listens on a socket and stores incoming messages as a structs in a slice:

var points []Point

type Point struct {
    time   time.Time
    x    float64
    y    float64
}

func main() {
    received = make([]Point, 0)
    l, err := net.Listen("tcp4", ":8900")
    // (...)
}


func processIncomingData(data string) {

    // Parse icoming data that comes as: xValue,yValue
    inData = strings.Split(data, ",")
    x, err := strconv.ParseFloat(inData[0], 64);
    if err != nil {
        fmt.Println(err)
    }
    y, err := strconv.ParseFloat(inData[1], 64);
    if err != nil {
        fmt.Println(err)
    }

    // Store the new Point
    points = append(points, Point{
        time:   time.Now(),
        x:    x,
        y:    y,
    })

    // Remove points older than 1h ?

}

Now, as you might imagine this will quickly fill my RAM. What's the best way (faster execution) to remove points older than 1h after appening each new one? I'll be getting new points 10-15 times peer second.

Thank you.

图片转代码服务由CSDN问答提供 功能建议

因此,我正在构建一个小型实用程序,该实用程序侦听套接字并将传入消息作为结构存储在切片中:

 变量点[] Point 
 
type Point struct {
 time time.Time 
x float64 
y float64 
} 
 
func main(){\  n收到= make([] Point,0)
l,err:= net.Listen(“ tcp4”,“:8900”)
 //(...)
} 
 
 
func processIncomingData( 数据字符串){
 
 //解析传入的数据,例如:xValue,yValue 
 inData = strings.Split(data,“,”)
x,err:= strconv.ParseFloat(inData [0],64  ); 
 if err!= nil {
 fmt.Println(err)
} 
y,err:= strconv.ParseFloat(inData [1],64); 
如果err!= nil {
 fmt  .Println(err)
} 
 
 //存储新的Point 
 points = append(points,Point {
 time:time.Now(),
x:x,
y:y,
  })
 
 //删除早于1h的点吗?
 
} 
   
 
 

现在,您可能会想像这将很快填补我的R 上午。 在应用每个新点后,最好的方法(更快的执行)是删除早于1h的点? 我将获得10-15倍同行的新分数。

谢谢。

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

1条回答 默认 最新

  • duanbo7517
    duanbo7517 2019-04-29 22:01
    已采纳

    An approach I've used several times is to start a goroutine early in the project that looks something like this:

    go cleanup()
    
    ...
    
    func cleanup() {
      for {
        time.Sleep(...)
        // do cleanup
      }
    }
    

    Then what you could do is iterate over points using time.Since(point.time) to figure out how old each piece of data is. If it's too old, there's a slice trick to remove an item from a slice given it's position:

    points = append(points[:i], points[i+1:]...)
    

    (where i is the index to remove)

    Because the points are in the slice in order of the time they were added, you could speed things up by simply finding the first index that isn't an hour old and doing points = points[i:] to chop off the old points from the beginning of the slice.

    You may run into problems if you get a request that accesses the array while you're cleaning it up. Adding a sync.Mutex can help with that. Just lock the mutex before the cleanup and also attempt to lock the mutex anywhere else you write to the array. This may be premature optimization though. I'd experiment without the mutex before adding it in as this would effectively make interacting with points a serial operation and slow the service down.

    The time.Sleep(...) in the loop is to prevent from cleaning too often. You might be tempted to set it to an hour since you want to delete points older than that but you might end up with a situation where a point is added immediately after a cleanup. On the next cleanup it'll be 59 mins old and you don't delete it, on the NEXT cleanup it's nearly 2 hours old. My rule of thumb is that I attempt to clean up every 1/10 the amount of time I want to allow an object to stay in memory but that's rather arbitrary. This approach means an object could be at most 1h 5m 59s old when it's deleted.

    点赞 评论

相关推荐