带有子软件包的golang源的AWS CodeBuild失败

Question

When I attempt to build my golang project using CodeBuild golang image 1.10, it fails, unable to find the subpackages. Some background:

The application is organized as follows:

/go/src/company/app
/go/src/company/app/sub1
/go/src/company/app/sub2
etc...

This builds fine on my dev machine. However, when pulled by codebuild it is pulled into a different directory (/codebuild/output/srcNNN/src/<some path>) where <some path> varies according to what triggers the build.

I originally got it working by copying the code from where it is pulled to the the golang directory (/codebuild/output/srcNNN), but since the CodeBuild environment variable for the GOPATH directory inserts /go: (/go:/codebuild/output/srcNNN) in the front, I used an observed number of ../../... to copy. However, this is ugly and failed as soon as I triggered the build a different way.

My question is whether there is a good way to get this working? My next idea is to apply string manipulation to the observed path and copy there for (hopefully) more reliability. But that will only work as long as the GOPATH conforms to my assumptions.

Any ideas would be appreciated.

---

Clarification: When importing packages in the code, external packages are imported as follows:

import (
    "context"
    ...
}

Subpackages are not explicitly imported, but found when code is deployed as shown above (/go/src/company/app). However, AWS CodeBuild doesn't bring in the code this way.

Answer 1

See Update below for a complete answer if you are on golang 1.11 or higher... We no longer use my initial work around.

I was able to get an answer working. I'll post it here in case it is helpful to others, but it relies on observed behavior in AWS CodeBuild to work, so I don't think it is ideal.

In my buildspec.yaml I am able to get the build to work by:

1. Getting ${THEGOPATH} from ${GOPATH} by removing the "/go:" from the beginning
2. Copying all the code to the ${THEGOPATH}/src/<app path>
3. Copying other repositories to ${THEGOPATH}/src/<other app path>
4. Importing external dependencies as normal (in our case, go get ./... or explicit)
5. Building and forcing the output name (when launched from CodeBuild it used a different directory name)

The buildspec.yaml looks something like the following:

phases:
  install:
    commands:
      - echo GOPATH - $GOPATH
      - export THEGOPATH=`echo $GOPATH | cut -c 5-`
      - echo THEGOPATH = $THEGOPATH
      - mkdir -p ${THEGOPATH}/src/company/app1
      - mkdir -p  ${THEGOPATH}/src/company/other_repository_dependency
      - echo Copy source files to go root
      - cp -a ${CODEBUILD_SRC_DIR}/. ${THEGOPATH}/src/company/app1/${PACKAGE}
      - cp -a ${CODEBUILD_SRC_DIR_other_dep}/. ${THEGOPATH}/src/app/other_repository_dependecy/.
      - ls ${THEGOPATH}/src/
  build:
    commands:
      - echo Build started on `date`
      - echo Getting packages
      - go get ./...
      - echo DOING THE BUILD
      - go build -ldflags "<some flags>" -o "appname"
      - go test ./...
  post_build:
    commands:
      - echo Build completed on `date`
      - ls -al
      - pwd
artifacts:
  files:
    - appname

---

Update -- Better fix Today we attempted to build with go modules (available since 1.11) see here for an explanation of go modules.

Using go modules we defined the current source module app1 as company-name.com in the go.mod file like this:

module company-name.com/app1

go 1.12
require (
   ... *for example*
   github.com/golang/mock v1.3.1
   github.com/google/btree v1.0.0 // indirect
   github.com/google/go-cmp v0.3.0
   ... *etc*

We even reference our external files this way (though you'll need to figure out how to authenticate with your git repository. We used the credential helper built into the buildspec for https authentication). So, our import blocks look something like this now:

     import ( 
       "company-name.com/app1/subpackage1"
       abbrev "company-name.com/app1/subpackage2"
       "company-name.com/externalpkg"  // In another private git repo of ours
       ... //etc
     )
     ... //golang source follows here*

Finally, we added the following environment variable to the buildspec:

  variables:
    GO111MODULE: "on"
    git-credential-helper: yes

These ensured that modules worked in the path (thanks amwill04 for reminding me of my omission) and allowed the credentials for our Git repository to get setup properly.

In doing this we accomplished everything we needed:

1. By changing our references to the go module we could easily reference the subpackages as such
2. We were able to lock the version of all our dependencies
3. By implementing a simple server at company-name.com, we could reference other private modules from our application