I'm failing to build my golang project using gitlab-ci. When running dep init inside of project directory it fetches only part of my dependencies, and when tries to build binary fails with error that dependencies aren't found. When I'm building my project on osx, or running docker golang image to compile project on osx it builds successfully.
My .gitlab-ci.yml file:
image: golang:1.9.2
variables:
REPO_NAME: storage
before_script:
- mkdir -p $GOPATH/src/$REPO_NAME
- ln -svf $CI_PROJECT_DIR/* $GOPATH/src/$REPO_NAME
- cd $GOPATH/src/$REPO_NAME
- go get -u github.com/golang/dep/cmd/dep
- dep init
stages:
- compile
compile:
stage: compile
script:
- go build -o $CI_PROJECT_DIR/main
artifacts:
paths:
- main
Output of gitlab-runner task:
[0KRunning with gitlab-runner 10.2.0 (0a75cdd1)
on docker-auto-scale (4e4528ca)
[0;m[0KUsing Docker executor with image golang:1.9.2 ...
[0;m[0KUsing docker image sha256:5f8b4886692c4897e1f0855043da1896fe4f1e6762fccddfa2114a2fdfa1674f for predefined container...
[0;m[0KPulling docker image golang:1.9.2 ...
[0;m[0KUsing docker image golang:1.9.2 ID=sha256:1a34fad76b34f485ebc72d32044cafe963ae00c8e80dbf4115bafabd31e93ff6 for build container...
[0;msection_start:1511994825:prepare_script
[0KRunning on runner-4e4528ca-project-4778312-concurrent-0 via runner-4e4528ca-srm-1511994698-320032ac...
section_end:1511994828:prepare_script
[0Ksection_start:1511994828:get_sources
[0K[32;1mCloning repository...[0;m
Cloning into '/builds/group-name/service-storage'...
[32;1mChecking out f59f57e4 as master...[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1511994831:get_sources
[0Ksection_start:1511994831:restore_cache
[0Ksection_end:1511994832:restore_cache
[0Ksection_start:1511994832:download_artifacts
[0Ksection_end:1511994834:download_artifacts
[0Ksection_start:1511994834:build_script
[0K[32;1m$ mkdir -p $GOPATH/src/$REPO_NAME[0;m
[32;1m$ ln -svf $CI_PROJECT_DIR/* $GOPATH/src/$REPO_NAME[0;m
'/go/src/storage/Dockerfile' -> '/builds/group-name/service-storage/Dockerfile'
'/go/src/storage/aws' -> '/builds/group-name/service-storage/aws'
'/go/src/storage/db' -> '/builds/group-name/service-storage/db'
'/go/src/storage/logHelper' -> '/builds/group-name/service-storage/logHelper'
'/go/src/storage/main' -> '/builds/group-name/service-storage/main'
'/go/src/storage/main.go' -> '/builds/group-name/service-storage/main.go'
'/go/src/storage/responses' -> '/builds/group-name/service-storage/responses'
'/go/src/storage/routers' -> '/builds/group-name/service-storage/routers'
[32;1m$ cd $GOPATH/src/$REPO_NAME[0;m
[32;1m$ go get -u github.com/golang/dep/cmd/dep[0;m
[32;1m$ dep init[0;m
Using ^1.2.0 as constraint for direct dep github.com/joho/godotenv
Locking in v1.2.0 (a79fa1e) for direct dep github.com/joho/godotenv
Using ^1.6.0 as constraint for direct dep github.com/gorilla/mux
Locking in v1.6.0 (7f08801) for direct dep github.com/gorilla/mux
Locking in v1.1 (1ea2538) for transitive dep github.com/gorilla/context
[32;1m$ go build -o $CI_PROJECT_DIR/main[0;m
aws/aws.go:16:2: cannot find package "github.com/aws/aws-sdk-go/aws" in any of:
/go/src/storage/vendor/github.com/aws/aws-sdk-go/aws (vendor tree)
/usr/local/go/src/github.com/aws/aws-sdk-go/aws (from $GOROOT)
/go/src/github.com/aws/aws-sdk-go/aws (from $GOPATH)
aws/aws.go:17:2: cannot find package "github.com/aws/aws-sdk-go/aws/credentials" in any of:
/go/src/storage/vendor/github.com/aws/aws-sdk-go/aws/credentials (vendor tree)
/usr/local/go/src/github.com/aws/aws-sdk-go/aws/credentials (from $GOROOT)
/go/src/github.com/aws/aws-sdk-go/aws/credentials (from $GOPATH)
aws/aws.go:18:2: cannot find package "github.com/aws/aws-sdk-go/aws/session" in any of:
/go/src/storage/vendor/github.com/aws/aws-sdk-go/aws/session (vendor tree)
/usr/local/go/src/github.com/aws/aws-sdk-go/aws/session (from $GOROOT)
/go/src/github.com/aws/aws-sdk-go/aws/session (from $GOPATH)
aws/aws.go:19:2: cannot find package "github.com/aws/aws-sdk-go/service/s3" in any of:
/go/src/storage/vendor/github.com/aws/aws-sdk-go/service/s3 (vendor tree)
/usr/local/go/src/github.com/aws/aws-sdk-go/service/s3 (from $GOROOT)
/go/src/github.com/aws/aws-sdk-go/service/s3 (from $GOPATH)
routers/v1/images/imageFunctions/save_images.go:23:2: cannot find package "github.com/disintegration/imaging" in any of:
/go/src/storage/vendor/github.com/disintegration/imaging (vendor tree)
/usr/local/go/src/github.com/disintegration/imaging (from $GOROOT)
/go/src/github.com/disintegration/imaging (from $GOPATH)
db/dbFunctions/db_functions.go:7:2: cannot find package "github.com/satori/go.uuid" in any of:
/go/src/storage/vendor/github.com/satori/go.uuid (vendor tree)
/usr/local/go/src/github.com/satori/go.uuid (from $GOROOT)
/go/src/github.com/satori/go.uuid (from $GOPATH)
db/db.go:13:2: cannot find package "gopkg.in/mgo.v2" in any of:
/go/src/storage/vendor/gopkg.in/mgo.v2 (vendor tree)
/usr/local/go/src/gopkg.in/mgo.v2 (from $GOROOT)
/go/src/gopkg.in/mgo.v2 (from $GOPATH)
db/db.go:14:2: cannot find package "gopkg.in/mgo.v2/bson" in any of:
/go/src/storage/vendor/gopkg.in/mgo.v2/bson (vendor tree)
/usr/local/go/src/gopkg.in/mgo.v2/bson (from $GOROOT)
/go/src/gopkg.in/mgo.v2/bson (from $GOPATH)
section_end:1511994846:build_script
[0Ksection_start:1511994846:after_script
[0Ksection_end:1511994847:after_script
[0K[31;1mERROR: Job failed: exit code 1
[0;m
When running locally on osx, I see that dep init pulls all dependencies, and after that I can successfully run go build and it will build binary.
username@hostname storage (master) $ dep init
Locking in v1.1 (1ea2538) for transitive dep github.com/gorilla/context
Locking in v1.32.0 (32e4c1e) for transitive dep github.com/go-ini/ini
Locking in master (f7e31b4) for transitive dep golang.org/x/image
Using ^1.6.0 as constraint for direct dep github.com/gorilla/mux
Locking in v1.6.0 (7f08801) for direct dep github.com/gorilla/mux
Using ^1.1.0 as constraint for direct dep github.com/satori/go.uuid
Locking in v1.1.0 (879c588) for direct dep github.com/satori/go.uuid
Using ^1.2.0 as constraint for direct dep github.com/joho/godotenv
Locking in v1.2.0 (a79fa1e) for direct dep github.com/joho/godotenv
Using ^1.12.36 as constraint for direct dep github.com/aws/aws-sdk-go
Locking in v1.12.36 (5bcc0a2) for direct dep github.com/aws/aws-sdk-go
Locking in (0b12d6b5) for transitive dep github.com/jmespath/go-jmespath
Using v2 as constraint for direct dep gopkg.in/mgo.v2
Locking in v2 (3f83fa5) for direct dep gopkg.in/mgo.v2
Using ^1.2.4 as constraint for direct dep github.com/disintegration/imaging
Locking in v1.2.4 (dd50a3e) for direct dep github.com/disintegration/imaging
Also I have tried to run on OSX locally using docker golang image and it compiles fine:
docker run --rm -v "$PWD":/go/src/storage -w /go/src/storage golang:1.9.2 go get -u github.com/golang/dep/cmd/dep && dep init
docker run --rm -v "$PWD":/go/src/storage -w /go/src/storage golang:1.9.2 go build -v -o main
Problem only exists, running on gitlab-ci.
Tried various golang images, alpine and others. Same result, it always fails with same missing dependencies.
Update and Solution example:
As user @vardius pointed and comments and later made an answer, problem was with symlinking.
Replaced my line in .gitlab-ci.yml from:
- ln -svf $CI_PROJECT_DIR/* $GOPATH/src/$REPO_NAME
to
- cp -rf $CI_PROJECT_DIR/* $GOPATH/src/$REPO_NAME/
and everything compiled.