创建迭代JSON目录树-Golang

Question

I'm having trouble creating an iterative version of a program I wrote recursively in GoLang. The goal is to take a directory path and return a JSON tree that contains file information from that directory and preserves the directory structure. Here is what I have so far:

I've created a File struct that will contain the information of each entry in the directory tree:

type File struct {
    ModifiedTime time.Time `json:"ModifiedTime"`
    IsLink       bool      `json:"IsLink"`
    IsDir        bool      `json:"IsDir"`
    LinksTo      string    `json:"LinksTo"`
    Size         int64     `json:"Size"`
    Name         string    `json:"Name"`
    Path         string    `json:"Path"`
    Children     []File    `json:"Children"`
}

In my iterative program, I create a stack to simulate the recursive calls.

func iterateJSON(path string) {
    var stack []File
    var child File
    var file File
    rootOSFile, _ := os.Stat(path)
    rootFile := toFile(rootOSFile, path) //start with root file
    stack = append(stack, rootFile) //append root to stack 
    for len(stack) > 0 { //until stack is empty,
        file = stack[len(stack)-1] //pop entry from stack
        stack = stack[:len(stack)-1] 
        children, _ := ioutil.ReadDir(file.Path) //get the children of entry 
        for i := 0; i < len(children); i++ { //for each child
            child = (toFile(children[i], path+"/"+children[i].Name())) //turn it into a File object
            file.Children = append(file.Children, child) //append it to the children of the current file popped
            stack = append(stack, child) //append the child to the stack, so the same process can be run again
        }
    }
    rootFile.Children
    output, _ := json.MarshalIndent(rootFile, "", "     ")
    fmt.Println(string(output))
}

func toFile(file os.FileInfo, path string) File {
    var isLink bool
    var linksTo string
    if file.Mode()&os.ModeSymlink == os.ModeSymlink {
        isLink = true
        linksTo, _ = filepath.EvalSymlinks(path + "/" + file.Name())
    } else {
        isLink = false
        linksTo = ""
    }
    JSONFile := File{ModifiedTime: file.ModTime(),
        IsDir:    file.IsDir(),
        IsLink:   isLink,
        LinksTo:  linksTo,
        Size:     file.Size(),
        Name:     file.Name(),
        Path:     path,
        Children: []File{}}
    return JSONFile
}

Theoretically, the child files should be appended to the root file as we move through the stack. However, the only thing that is returned is the root file (without any children appended). Any idea as to why this is happening?

Answer 1

The main problem is that structs are not descriptor values like slices or maps, that is if you assign a struct value to a variable, it will be copied. If you assign a struct value to an element of a slice or array, the slice will be copied. They will not be linked!

So when you add your rootFile to stack, and then you pop an element from the stack (which will be equal to rootFile) and you modify the popped element, you will not observe the changes in your local variable rootFile.

Solution is simple: use pointers to structs.

You also have a mistake in your code:

child = (toFile(children[i], path+"/"+children[i].Name())) //turn it into a File object

It should be:

child = (toFile(children[i], file.Path+"/"+children[i].Name())) // ...

Tips to improve your code:

I would rather use path.Join() or filepath.Join() to join path elements:

child = toFile(children[i], filepath.Join(file.Path, children[i].Name()))

Your code might not even work if the initial path ends with a slash or backslash and you explicitly concatenate it with another slash. Join() will take care of these so you don't have to.

Don't declare all local variables ahead in the beginning of your function, only when you need them, and in the most inner block you need them. This will ensure you don't accidentally assign to the wrong variable, and you will know it is not modified outside of the innermost block (because outside of it it is not in scope) - this helps understanding your code much easier. You may also use short variable declaration.

Make use of the for ... range construct, much cleaner. For example:

for _, chld := range children {
    child := toFile(chld, filepath.Join(file.Path, chld.Name()))
    file.Children = append(file.Children, child)
    stack = append(stack, child)
}

Also make use of zero values, for example if a file is not a link, you don't need to set the IsLink and LinksTo fields as the zero values are false and "" which is what you would end up with.

And although it may not be important here, but always handle errors, print or log them as a minimum so you won't end up wasting time figuring out what is wrong if something is not what you expect (you will end up searching bugs in your code, and hours later you finally add print errors and see the bug wasn't in your code but somewhere else).

Working variant using pointers and tips mentioned above

type File struct {
    ModifiedTime time.Time `json:"ModifiedTime"`
    IsLink       bool      `json:"IsLink"`
    IsDir        bool      `json:"IsDir"`
    LinksTo      string    `json:"LinksTo"`
    Size         int64     `json:"Size"`
    Name         string    `json:"Name"`
    Path         string    `json:"Path"`
    Children     []*File   `json:"Children"`
}

func iterateJSON(path string) {
    rootOSFile, _ := os.Stat(path)
    rootFile := toFile(rootOSFile, path) //start with root file
    stack := []*File{rootFile}

    for len(stack) > 0 { //until stack is empty,
        file := stack[len(stack)-1] //pop entry from stack
        stack = stack[:len(stack)-1]
        children, _ := ioutil.ReadDir(file.Path) //get the children of entry
        for _, chld := range children {          //for each child
            child := toFile(chld, filepath.Join(file.Path, chld.Name())) //turn it into a File object
            file.Children = append(file.Children, child)                 //append it to the children of the current file popped
            stack = append(stack, child)                                 //append the child to the stack, so the same process can be run again
        }
    }

    output, _ := json.MarshalIndent(rootFile, "", "     ")
    fmt.Println(string(output))
}

func toFile(file os.FileInfo, path string) *File {
    JSONFile := File{ModifiedTime: file.ModTime(),
        IsDir:    file.IsDir(),
        Size:     file.Size(),
        Name:     file.Name(),
        Path:     path,
        Children: []*File{},
    }
    if file.Mode()&os.ModeSymlink == os.ModeSymlink {
        JSONFile.IsLink = true
        JSONFile.LinksTo, _ = filepath.EvalSymlinks(filepath.Join(path, file.Name()))
    } // Else case is the zero values of the fields
    return &JSONFile
}