I'm using the aws-sdk-go package in Golang to connect to Amazon S3 to provide a cloud-based storage pool. I have this working well. I would like to be able to support bulk high-speed transfers using Snowball, so I got a Snowball Edge to test this in my lab. I have not figured out how to get this working, and the documentation for Snowball Edge doesn't seem complete. This configuration may be impacted by having ordered a Snowball Edge and not just a Snowball.
The reason that I'm finding the Edge more problematic, is that a normal Snowball requires an application called snowballAdapter to be running, which looks like it handles some port mapping issues. But, this application seems to be incompatible with the Edge device, as it reports that it doesn't work with a "Snowball Edge Manifest file".
I looked at the ports that are available on the real AWS S3 and nmap reports:
nmap -v -sT -Pn s3.us-east-1.amazonaws.com
...
Scanning s3.us-east-1.amazonaws.com (52.216.161.53) [1000 ports]
Discovered open port 443/tcp on 52.216.161.53
Discovered open port 80/tcp on 52.216.161.53
Whereas on Snowball Edge, the ports are:
nmap -v -sT -Pn 192.168.1.4
....
Scanning 192.168.1.4 [1000 ports]
Discovered open port 8080/tcp on 192.168.1.4
Discovered open port 22/tcp on 192.168.1.4
Discovered open port 9091/tcp on 192.168.1.4
Discovered open port 8443/tcp on 192.168.1.4
....
PORT STATE SERVICE
22/tcp open ssh
8080/tcp open http-proxy
8443/tcp open https-alt
9091/tcp open xmltec-xmlmail
So, it seems to me that the issue may be that I have to make the aws package use port 8443 for Snowball Edge instead of 443 for the real S3. The code for connecting to S3 is pretty straight forward:
creds := credentials.NewStaticCredentials(s3Config.S3AccessKey, s3Config.S3SecretAccessKey, s3Config.S3Token)
_, err := creds.Get()
if err != nil {
return nil, nil, err
}
if len(baseFolder) > 0 {
baseFolder = baseFolder + "/"
}
cfg := aws.NewConfig().WithRegion(s3Config.S3Region).WithCredentials(creds)
svc := s3.New(session.New(), cfg)
params := &s3.ListObjectsInput{
Bucket: aws.String(s3Config.S3BucketName),
Prefix: aws.String(baseFolder),
Delimiter: aws.String("/"),
}
resp, err := svc.ListObjects(params)
So, the question is, how do I change the code to point to the Snowball Edge? I've tried mapping to the Snowball Edge in /etc/hosts from the Amazon S3 endpoint. I understand why this didn't work after discovering that the ports are different. I've played around with adding different forms of "WithEndpoint("...host...") with no success. Or, am I on the completely wrong track, and should be able to get the snowballAdapter to work with a Snowball Edge?
By the way, all the snowballEdge commands work as expected, so the device seems to be working fine, e.g.:
./snowballEdge list-access-keys
{
"AccessKeyIds" : [ "..." ]
}
./snowballEdge get-secret-access-key --access-key-id ....
[snowballEdge]
aws_access_key_id = ...
aws_secret_access_key = ...
And, I've used the correct keys associated with the device, and it does have the S3 Service configured in it:
./snowballEdge list-services
{
"ServiceIds" : [ "s3" ]
}
