Edit:
After I added a small part of file I was using (7 GB) and tried to run the program, I could see this:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/media/developer/golang/manual/examples/sp/v2/sp.v2.go:71 +0x4a9
exit status 2
Situation:
I'm completely new to GO, so I'm sorry if my question is really simple.
I am trying to stream xml
file, split documents and then parse them in different GO Routines.
Example of XML file I'm using:
<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.0.2">
<relation id="56688" user="kmvar" uid="56190" visible="true" version="28" changeset="6947637" timestamp="2011-01-12T14:23:49Z">
<member type="node" ref="294942404" role=""/>
<member type="node" ref="364933006" role=""/>
<tag k="name" v="Küstenbus Linie 123"/>
<tag k="network" v="VVW"/>
<tag k="route" v="bus"/>
<tag k="type" v="route"/>
</relation>
<relation id="98367" user="jdifh" uid="92834" visible="true" version="28" changeset="6947637" timestamp="2011-01-12T14:23:49Z">
<member type="node" ref="294942404" role=""/>
<member type="way" ref="4579143" role=""/>
<member type="node" ref="249673494" role=""/>
<tag k="name" v="Küstenbus Linie 123"/>
<tag k="network" v="VVW"/>
<tag k="operator" v="Regionalverkehr Küste"/>
<tag k="ref" v="123"/>
</relation>
<relation id="72947" user="schsu" uid="92374" visible="true" version="28" changeset="6947637" timestamp="2011-01-12T14:23:49Z">
<member type="node" ref="294942404" role=""/>
<tag k="name" v="Küstenbus Linie 123"/>
<tag k="type" v="route"/>
</relation>
<relation id="93742" user="doiff" uid="61731" visible="true" version="28" changeset="6947637" timestamp="2011-01-12T14:23:49Z">
<member type="node" ref="294942404" role=""/>
<member type="node" ref="364933006" role=""/>
<tag k="route" v="bus"/>
<tag k="type" v="route"/>
</relation>
</osm>
I have this snippet of code:
package main
import (
"encoding/xml"
"bufio"
"fmt"
"os"
"io"
)
type RS struct {
I string `xml:"id,attr"`
M []struct {
I string `xml:"ref,attr"`
T string `xml:"type,attr"`
R string `xml:"role,attr"`
} `xml:"member"`
T []struct {
K string `xml:"k,attr"`
V string `xml:"v,attr"`
} `xml:"tag"`
}
func main() {
p1D, err := os.Open("/media/developer/Transcend/osm/xml/relations.xml")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer p1D.Close()
reader := bufio.NewReader(p1D)
var count int32
var element string
channel := make(chan RS) // channel
for {
p2Rc, err := reader.ReadSlice('
')
if err != nil {
if err == io.EOF {
break
} else {
fmt.Println(err)
os.Exit(1)
}
}
var p2Rs = string(p2Rc)
if p2Rc[2] == 114 {
count++
if (count != 1) {
go parseRelation(element, channel)
}
element = ""
element += p2Rs
} else {
element += p2Rs
}
}
for i := 0; i < 5973974; i++ {
fmt.Println(<- channel)
}
}
func parseRelation(p1E string, channel chan RS) {
var rs RS
xml.Unmarshal([]byte(p1E), &rs)
channel <- rs
}
It is supposed to print each struct, but I see nothing. The program just hangs.
I tested streamer and splitter (just added fmt.Println(rs)
in function parseRelation before sending message into the channel). I could see structs. So, the problems are in sending and receiving messages.
Problem:
I have no idea how to solve this issue. Tried changing the type of the messages in the channel (from RS
to string
) and sending just one string every time. But it also didn't help (I could see nothing)