go build by default includes debugging information in the executable, including the path of the source code files.
When you have a
vendor/ directory the path of the source files will be longer than when you the source files lie inside
GOPATH. As a result, the debugging information will take up more space.
Try comparing the size of the built binary by telling
go build to exclude the debugging information like this.
go build -ldflags=-s ./
That reduces the difference in size. See https://golang.org/cmd/link/ for more details about the linker flags.
(Edit: the source file paths are included when debugging information is not included as well as is evidenced by stack traces from a
panic(), so this answer is not complete)