有没有一种方法可以使用编码/目标序列化循环数据结构？

Question

I'm working on porting a neural network library to Go. I want to be able to save and restore a trained network, so I'm attempting to serialize it directly. The problem is, the network struct contains cycles in its field (Neuron A has a connection to Neuron B, which has a connection to Neuron A). Whenever I try to serialize the entire network with encoding/gob, it fails with a stackoverflow.

Here's a very simple example of code that breaks in the same way:

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
    "log"
)

type P struct {
    Name    string
    Q *Q
}

type Q struct {
    Name string
    P *P
}

func main() {
    var network bytes.Buffer        // Stand-in for a network connection
    enc := gob.NewEncoder(&network) // Will write to network.
    dec := gob.NewDecoder(&network) // Will read from network.

    p := &P{ "P", nil }
    q := &Q{ "Q", p }
    p.Q = q

    err := enc.Encode(p)
    if err != nil {
        log.Fatal("encode error:", err)
    }
    // Decode (receive) the value.
    var p2 *P
    err = dec.Decode(&p2)
    if err != nil {
        log.Fatal("decode error:", err)
    }
    fmt.Printf("%#v", p2)
}

http://play.golang.org/p/LrO0VlLnX4

Barring rewriting the entire structure of the library to avoid cycles, is there a straightforward way to get around this problem?

Thanks

Answer 1

You can't use gob directly, but fear not brave citizen of the world!

You can implement the BinaryMarshaler/BinaryUnmarshaler interfaces on your type as a work around and gob will happily use them instead when it's encoding/decoding your type.

func (p *P) MarshalBinary() (_ []byte, err error) {
    var buf bytes.Buffer
    enc := gob.NewEncoder(&buf)
    enc.Encode(p.Name)
    if p.Q == nil {
        return buf.Bytes(), nil
    }
    isCyclic := p.Q != nil && p.Q.P == p
    enc.Encode(isCyclic)
    if isCyclic {
        p.Q.P = nil
        err = enc.Encode(p.Q)
        p.Q.P = p
    } else {
        err = enc.Encode(p.Q)
    }
    //buf.Encode
    return buf.Bytes(), err
}

func (p *P) UnmarshalBinary(data []byte) (err error) {
    dec := gob.NewDecoder(bytes.NewReader(data))
    if err = dec.Decode(&p.Name); err != nil {
        return
    }
    var isCyclic bool
    if err = dec.Decode(&isCyclic); err != nil {
        return
    }
    err = dec.Decode(&p.Q)
    if isCyclic {
        p.Q.P = p
    }
    return
}

<kbd>playground</kbd>

warning creating a new decoder/encoder every time is extremely inefficient, you might want to look into using binary.*.