在elasticbeanstalk上部署私有github repo golang应用程序

Question

I have been struggling trying to deploy my Golang app to AWS EB for couple days.

I am trying to deploy my app on an EB server Preconfigured Docker - Go 1.4 running on 64bit Debian/2.9.2 using the eb cli via the command eb deploy in my app folder.

After couple seconds, I've got an error message saying that my app wasn't deployed because of an error. Looking at the eb-activity.log, here what I can see:

    /var/log/eb-activity.log
    -------------------------------------
    Fetching https://golang.org/x/crypto?go-get=1
    Parsing meta tags from https://golang.org/x/crypto?go-get=1 (status code 200)
    golang.org/x/crypto (download)
    Fetching https://golang.org/x/sys/unix?go-get=1
    Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
    get "golang.org/x/sys/unix": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
    get "golang.org/x/sys/unix": verifying non-authoritative meta tag
    Fetching https://golang.org/x/sys?go-get=1
    Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
    golang.org/x/sys (download)
    github.com/randomuser/private-repo (download)
    # cd .; git clone https://github.com/randomuser/private-repo /go/src/github.com/randomuser/private-repo
    Cloning into '/go/src/github.com/randomuser/private-repo'...
    fatal: could not read Username for 'https://github.com': No such device or address
    package github.com/Sirupsen/logrus
            imports golang.org/x/crypto/ssh/terminal
            imports golang.org/x/sys/unix
            imports github.com/randomuser/private-repo/apis: exit status 128
    package github.com/Sirupsen/logrus
            imports golang.org/x/crypto/ssh/terminal
            imports golang.org/x/sys/unix
            imports github.com/randomuser/private-repo/app
            imports github.com/randomuser/private-repo/app
            imports github.com/randomuser/private-repo/app: cannot find package "github.com/randomuser/private-repo/app" in any of:
            /usr/src/go/src/github.com/randomuser/private-repo/app (from $GOROOT)
            /go/src/github.com/randomuser/private-repo/app (from $GOPATH)

I suppose there is an issue when the server tries to install the app, it seems it's trying to retrieve from my private repo on github ...

I referenced my app sub packages as github.com/randomuser/private-repo/subpackage I supposed this is why it behaves like that.

Is there a way to deploy all my code, forcing my private repo to be populated within the GOROOT src/github.com/randomuser/private-repo/ so the server doesn't have to try to get it?

I didn't found any proper example from Amazon docs (multi-packages apps) nor from Github.

Am I missing anything? Is there a better solution?

On a side note, I also tried to deploy my compiled binary directly (create a folder where I put only the binary, zip it and upload it on the ebs env) but it didn't worked neither ... Maybe this option requires yet another env config (if so, which one?).

Thanks for your help :)

Configuration

• Golang app having the following folders:

  ├── Dockerfile
  ├── server.go
  ├── Gopkg.lock
  ├── Gopkg.toml
  ├── Makefile
  ├── apis
  │   ├── auth.go
  │   ├── auth_test.go
  │   ├── ...
  ├── app
  │   ├── config.go
  │   ├── init.go
  │   ├── logger.go
  │   ├── scope.go
  │   ├── transactional.go
  │   └── version.go
  ├── config
  │   ├── dev.app.yaml
  │   ├── errors.yaml
  │   └── prod.app.yaml
  ├── daos
  │   ├── auth.go
  │   ├── auth_test.go
  │   ├── ...
  ├── errors
  │   ├── api_error.go
  │   ├── api_error_test.go
  │   ├── errors.go
  │   ├── errors_test.go
  │   ├── template.go
  │   └── template_test.go
  ├── models
  │   ├── identity.go
  │   ├── ...
  ├── services
  │   ├── auth.go
  │   ├── auth_test.go
  │   ├── ...
  ├── util
  │   ├── paginated_list.go
  │   └── paginated_list_test.go
  

• Here is the content of my server.go

  package main
  
  import (
      "flag"
      "fmt"
      "net/http"
  
      "github.com/jinzhu/gorm"
      _ "github.com/jinzhu/gorm/dialects/mysql"
  
      "github.com/randomuser/private-repo/apis"
      "github.com/randomuser/private-repo/app"
      "github.com/randomuser/private-repo/daos"
      "github.com/randomuser/private-repo/errors"
      "github.com/randomuser/private-repo/services"
  )
  
  func main() {
      // getting env from command line
      // env is either prod, preprod or dev
      // by default, env is prod
      env := flag.String("env", "prod", "environment: prod, preprod or dev")
      flag.Parse()
  
      ...
      router.To("GET,HEAD", "/ping", func(c *routing.Context) error {
         c.Abort() // skip all other middlewares/handlers
         return c.Write("OK " + app.Version)
      })
      ...
      // Serve on port 5000
  

• My Dockerfile content:

  FROM golang:1.4.2-onbuild
  
  ADD . /go/src/github.com/randomuser/private-repo
  RUN go install github.com/randomuser/private-repo
  
  EXPOSE 5000
  
  ENTRYPOINT /go/bin/private-repo
  

Answer 1

I finally managed to make it works.

So I created a brand new eb app (without docker). I then figured out that my app wasn't able to retrieve env vars set in the console somehow ... So what I did is that I forced the env variables to be passed to my app at startup time from my production config file using the build.sh script see below :

#!/bin/bash -xe
# See http://tldp.org/LDP/abs/html/options.html
# -x -> Print each command to stdout before executing it, expand commands
# -e -> Abort script at first error, when a command exits with non-zero status
#   (except in until or while loops, if-tests, list constructs)

# $GOPATH isn't set by default, nor do we have a usable Go workspace :'(
GOPATH="/var/app/current"
APP_BUILD_DIR="$GOPATH/src/to-be-defined"     # We will build the app here
APP_STAGING_DIR="/var/app/staging"            # Current directory
DEP_VERSION="v0.3.2"                          # Use specific version for stability
ENV_VAR_PREFIX="TO_BE_DEFINED_"

# Install dep, a Go dependency management tool, if not already installed or if
# the version does not match.
if ! hash dep 2> /dev/null ||\
    [[ $(dep version | awk 'NR==2{print $3}') != "$DEP_VERSION" ]]; then
    # /usr/local/bin is expected to be on $PATH.
    curl -L \
        -s https://github.com/golang/dep/releases/download/$DEP_VERSION/dep-linux-amd64 \
        -o /usr/local/bin/dep

    chmod +x /usr/local/bin/dep
fi

# Remove the $APP_BUILD_DIR just in case it was left behind in a failed build.
rm -rf $APP_BUILD_DIR

# Setup the application directory
mkdir -p $APP_BUILD_DIR

# mv all files to $APP_BUILD_DIR
# https://superuser.com/questions/62141/how-to-move-all-files-from-current-directory-to-upper-directory
mv * .[^.]* $APP_BUILD_DIR
cd $APP_BUILD_DIR

# Pull in dependencies into vendor/.
dep ensure

# Build the binary with jsoniter tag.
go build -o application -tags=jsoniter .

# Modify permissons to make the binary executable.
chmod +x application
# Move the binary back to staging dir.
# Along with the configuration files.
mkdir $APP_STAGING_DIR/bin
# By default, `bin/application` is executed. This way, a Procfile isn't needed.
mv application $APP_STAGING_DIR/bin
cp -r config $APP_STAGING_DIR
# TODO: Fix the viper not working with env var
# Generate prod config from env variables
/opt/elasticbeanstalk/bin/get-config environment --output YAML | sed s/${ENV_VAR_PREFIX}//g > $APP_STAGING_DIR/config/prod.app.yaml
# Copy .ebextensions back to staging directory.
# cp -r .ebextensions $APP_STAGING_DIR

# Clean up.
rm -rf $APP_BUILD_DIR

echo "Build successful!!"

My build.sh file is called by EBS using this Buildfile:

make: ./build.sh

Et voilà ! Everything works properly now :)