How to connect to IBM Cloud object storage (Cleversafe) from Golang using the AWS S3 sdk - object-storage

Question

I'm trying to connect to IBM Cloud object storage (Cleversafe) using the following code and I get following error
"bad response: MissingRegion: could not find region configuration"
package main
import (
"fmt"
"os"
"bytes"
"net/http"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/aws/session"
)
func main() {
fmt.Printf("Starting ...")
aws_access_key_id := "some id"
aws_secret_access_key := "some key"
token := ""
creds := credentials.NewStaticCredentials(aws_access_key_id, aws_secret_access_key, token)
fmt.Printf("creds: ", creds)
_, err := creds.Get()
if err != nil {
fmt.Printf("bad credentials: %s", err)
}
//EndpointResolver endpoints.Resolver
cfg := aws.NewConfig().WithEndpoint("dal.ibmselect.objstor.com").WithCredentials(creds)
fmt.Printf("cfg: %+v \n", cfg)
//cfg := aws.NewConfig().WithEndpointResolver()
//cfg := aws.NewConfig().WithCredentials(creds)
svc := s3.New(session.New(), cfg)
fmt.Printf("svc: %+v \n", svc)
file, err := os.Open("./test.jpg")
if err != nil {
fmt.Printf("err opening file: %s", err)
}
defer file.Close()
fileInfo, _ := file.Stat()
size := fileInfo.Size()
buffer := make([]byte, size) // read file content to buffer
file.Read(buffer)
fileBytes := bytes.NewReader(buffer)
fileType := http.DetectContentType(buffer)
path := file.Name()
params := &s3.PutObjectInput{
Bucket: aws.String("cosv1-jlnab"),
Key: aws.String(path),
Body: fileBytes,
ContentLength: aws.Int64(size),
ContentType: aws.String(fileType),
}
resp, err := svc.PutObject(params)
if err != nil {
fmt.Printf("bad response: %s \n", err)
}
fmt.Printf("response %s", awsutil.StringValue(resp))
}

Answer 1

Thanks
Following is a working sample code:
package main
import (
"fmt"
"os"
"bytes"
"net/http"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/aws/aws-sdk-go/aws/session"
)
func main() {
fmt.Printf("Starting ...")
aws_access_key_id := "Access key from Bluemix credentials"
aws_secret_access_key := "Secret key from Bluemix credentials"
token := ""
creds := credentials.NewStaticCredentials(aws_access_key_id, aws_secret_access_key, token)
fmt.Printf("creds: ", creds)
_, err := creds.Get()
if err != nil {
fmt.Printf("bad credentials: %s", err)
}
//EndpointResolver endpoints.Resolver
cfg := aws.NewConfig().WithRegion("cleversafe").WithEndpoint("s3-api.us-geo.objectstorage.softlayer.net").WithCredentials(creds)
fmt.Printf("cfg: %+v \n", cfg)
svc := s3.New(session.New(), cfg)
fmt.Printf("svc: %+v \n", svc)
file, err := os.Open("./test.jpg")
if err != nil {
fmt.Printf("err opening file: %s", err)
}
defer file.Close()
fileInfo, _ := file.Stat()
size := fileInfo.Size()
buffer := make([]byte, size) // read file content to buffer
file.Read(buffer)
fileBytes := bytes.NewReader(buffer)
fileType := http.DetectContentType(buffer)
path := file.Name()
params := &s3.PutObjectInput{
Bucket: aws.String("test-ophir"),
Key: aws.String(path),
Body: fileBytes,
ContentLength: aws.Int64(size),
ContentType: aws.String(fileType),
}
resp, err := svc.PutObject(params)
if err != nil {
fmt.Printf("bad response: %s \n", err)
}
fmt.Printf("response %s", awsutil.StringValue(resp))
}

Answer 2

Essentially this happens because IBM COS and AWS S3 handle the concept of a 'region' somewhat differently. This error is being thrown because the AWS Go SDK doesn't know what value to provide for the 'region' variable used to calculate the authorization header. In this context, IBM COS doesn't care what value, as long is it isn't null. The easiest way is to provide an environment variable via the command line: export AWS_REGION=foo where foo can be any string, although it could also be set as a configuration variable at client creation within your application.
I'd recommend using something that makes sense, like us or us-south as IBM COS will probably enforce this at some point down the road by linking the concept of 'region' to the AWS LocationConstraint variable, which is currently used by IBM COS to define the storage class of a bucket, although this distinction will be made explicit at some point in the future.