有效地从JSON文件中删除无效字符？

Question

I am reading in a file via the command line.

As the file is a JSON export from Oracle, it has a certain structure. This default structure is not valid JSON for some reason. Example:

// This isn't valid JSON
,"items":
[
{"id":123,"language":"ja-JP","location":"Osaka"}
,{"id":33,"language":"ja-JP","location":"Tokyo"}
,{"id":22,"language":"ja-JP","location":"Kentok"}
]}

I wish for it to only be an array of object, thus having the expected output:

// This is valid json
[
{"id":123,"language":"ja-JP","location":"Osaka"}
,{"id":33,"language":"ja-JP","location":"Tokyo"}
,{"id":22,"language":"ja-JP","location":"Kentok"}
]

Therefore, I need to remove line 1(entirely) as well as the last } from the last line of the file.

The file is being parsed via commandline from the input:

file, err := ioutil.ReadFile(os.Args[1])

I am trying to remove the invalid strings/words this way, but it does not reformat anything:

// in func main()
removeInvalidJSON(file, os.Args[1])

// later on .. 
func removeInvalidJSON(file []byte, path string) {

    info, _ := os.Stat(path)
    mode := info.Mode()

    array := strings.Split(string(file), "
")
    fmt.Println(array)

    //If we have the clunky items array which is invalid JSON, remove the first line
    if strings.Contains(array[0], "items") {
        fmt.Println("Removing items")
        array = append(array[:1], array[1+1:]...)
    }

    // Finds the last index of the array
    lastIndex := array[len(array)-1]

    // If we have the "}" in the last line, remove it as this is invalid JSON
    if strings.Contains(lastIndex, "}") {
        fmt.Println("Removing }")
        strings.Trim(lastIndex, "}")
    }

    // Nothing changed?
    fmt.Println(array)

    ioutil.WriteFile(path, []byte(strings.Join(array, "
")), mode)
}

The above function does write to the file I can see - but it does not alter the array as far as I can tell, and does not write it into the file.

How do I effectively remote the first line of the file, as well as the last false curly brace } from the file?

I unmarshall the JSON in another function: Is there a method of doing it more "cleanly" using the "encoding/json" library?

Answer 1

There are several significant issues with this code causing it to behave not as intended. I've noted these with comments below:

func removeInvalidJSON(file []byte, path string) {

    info, _ := os.Stat(path)
    mode := info.Mode()

    array := strings.Split(string(file), "
")
    fmt.Println(array)

    //If we have the clunky items array which is invalid JSON, remove the first line
    if strings.Contains(array[0], "items") {
        fmt.Println("Removing items")
        // If you just want to remove the first item, this should be array = array[1:].
        // As written, this appends the rest of the array to the first item, i.e. nothing.
        array = append(array[:1], array[1+1:]...)
    }

    // Finds the last ~index~ *line* of the array
    lastIndex := array[len(array)-1]

    // If we have the "}" in the last line, remove it as this is invalid JSON
    if strings.Contains(lastIndex, "}") {
        fmt.Println("Removing }")
        // Strings are immutable. `strings.Trim` does nothing if you discard the return value
        strings.Trim(lastIndex, "}")
        // After the trim, if you want this to have any effect, you need to put it back in `array`.
    }

    // Nothing changed?
    fmt.Println(array)

    ioutil.WriteFile(path, []byte(strings.Join(array, "
")), mode)
}

I think what you want is something more like:

func removeInvalidJSON(file []byte, path string) {
    info, _ := os.Stat(path)
    mode := info.Mode()

    array := strings.Split(string(file), "
")
    fmt.Println(array)

    //If we have the clunky items array which is invalid JSON, remove the first line
    if strings.Contains(array[0], "items") {
        fmt.Println("Removing items")
        array = array[1:]
    }

    // Finds the last line of the array
    lastLine := array[len(array)-1]

    array[len(array)-1] = strings.Trim(lastLine, "}")

    fmt.Println(array)

    ioutil.WriteFile(path, []byte(strings.Join(array, "
")), mode)
}