drt96856 2019-01-16 07:11
浏览 56

获取切片中值的指针

Is there any way to get pointers to the actual value in slice? I have a tree of offers in the maps:

map[uint64]map[uint16][]offer

For fast access by id (which from 1 to n) I need a slice of pointers:

[]*offer

Actual offers lies in the first tree, in the slice should be pointers to the that actual.

I can't traverse tree and collect pointers of actual values because with range of this slice gives me a copies, but I have more than billions of this structs in total and copies will lead to waste of my memory: 40 bytes for one struct lead to the 100+ gb and much more in the future. I also need to store slice of values in the tree to contiguous in order to fact lookup of offers.

May be there are some way to use unsafe or reflect package to get these pointers?

Index of pointers build only one time and both structs are immutable and only used to lookup offers.

Update: I was totally wrong. There is no problem to get pointer to element in the slice. My code was:

        var offers []offer // actually there was about 50gb of offers
        for i := range offers {
            currentOffer := offers[i]
            s.Relations[currentOffer.Id] = &currentOffer
        }

After this snippet overall amount of ram consumed by the app became 100+gb. I immediately thought that take element from slice of values give me a copy of that value, but it is wrong. I just copied original value to the currentOffer variable by myself. Little mistake took 50gb of ram away.

Actually this snippet is work fine as expected:

    for i := range offers {
        s.Relations[offers[i].Id] = &offers[i]
    }
  • 写回答

1条回答 默认 最新

  • doudizhi947129 2019-01-16 09:31
    关注

    After some investigation I came to the following code:

    package main
    
    import (
        "fmt"
        "reflect"
        "unsafe"
    )
    
    type offer struct {
        id uint64
    }
    
    func main() {
    
        sl := []offer{{1}, {id: 2}}
    
        size := unsafe.Sizeof(offer{})
        header := (*reflect.SliceHeader)(unsafe.Pointer(&sl))
        for i := 0; i < len(sl); i++ {
            offset := uintptr(i) * size
            ptr := header.Data + offset
    
            o := (*offer)(unsafe.Pointer(ptr))
            o.id = 5
            fmt.Println(o)
        }
    
        for _, o := range sl {
            fmt.Println(o.id)
        }
    }
    

    Playground: https://play.golang.org/p/OM3i84cKAB_7

    Output:

    &{5}
    &{5}
    5
    5
    

    This snippet of code illustrate access to the actual data in the slice, not copied one.

    Update: Take pointers via unsafe pointer give the same effect as take regular pointer from the slice by index.

    评论

报告相同问题?

悬赏问题

  • ¥15 关于#hadoop#的问题
  • ¥15 (标签-Python|关键词-socket)
  • ¥15 keil里为什么main.c定义的函数在it.c调用不了
  • ¥50 切换TabTip键盘的输入法
  • ¥15 可否在不同线程中调用封装数据库操作的类
  • ¥15 微带串馈天线阵列每个阵元宽度计算
  • ¥15 keil的map文件中Image component sizes各项意思
  • ¥20 求个正点原子stm32f407开发版的贪吃蛇游戏
  • ¥15 划分vlan后,链路不通了?
  • ¥20 求各位懂行的人,注册表能不能看到usb使用得具体信息,干了什么,传输了什么数据