I'm trying to parse a CSV file which contains a JSON object in the last column.
Here is an example with two rows from the input CSV file:
'id','value','createddate','attributes'
524256,CAFE,2018-04-06 16:41:01,{"Att1Numeric": 6, "Att2String": "abc"}
524257,BEBE,2018-04-06 17:00:00,{}
I tried using the parser from csv
package:
func processFileAsCSV(f *multipart.Part) (int, error) {
reader := csv.NewReader(f)
reader.LazyQuotes = true
reader.Comma = ','
lineCount := 0
for {
line, err := reader.Read()
if err == io.EOF {
break
} else if err != nil {
fmt.Println("Error:", err)
return 0, err
}
if lineCount%100000 == 0 {
fmt.Println(lineCount)
}
lineCount++
fmt.Println(lineCount, line)
processLine(line) // do something with the line
}
fmt.Println("done!", lineCount)
return lineCount, nil
}
But I got an error:
Error: line 2, column 0: wrong number of fields in line,
probably because the parser ignores the JSON scope which starts with {
.
Should I be writing my own CSV parser, or is there a library that can handle this?