How to sign amazon web service requests from the python app engine? - google-app-engine

Question

I use Amazon web service api from within my Google app engine application. Amazon have said that they will only accept signed requests from Aug 15, 2009. While they have given simple instructions for signing, I am not so knowledgeable of Python libraries for SHA256. The app engine documentation says it supports pycrypto but I was just wondering (read being lazy) if anyone has already done this. Any code snippets you could share? Any issues I might be missing here?

Answer 1

Here is an example of a REST request based on lower level (then boto) libraries. Solution was taken from http://cloudcarpenters.com/blog/amazon_products_api_request_signing.
All you need is valid entries for AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
def amazon_test_url():
import base64, hashlib, hmac, time
from urllib import urlencode, quote_plus
AWS_ACCESS_KEY_ID = 'YOUR_KEY'
AWS_SECRET_ACCESS_KEY = 'YOUR_SECRET_KEY'
TEST_ISBN = '9780735619678' #http://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read
base_url = "http://ecs.amazonaws.com/onca/xml"
url_params = dict(
Service='AWSECommerceService',
Operation='ItemLookup',
IdType='ISBN',
ItemId=TEST_ISBN,
SearchIndex='Books',
AWSAccessKeyId=AWS_ACCESS_KEY_ID,
ResponseGroup='Images,ItemAttributes,EditorialReview,SalesRank')
#Can add Version='2009-01-06'. What is it BTW? API version?
# Add a ISO 8601 compliant timestamp (in GMT)
url_params['Timestamp'] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
# Sort the URL parameters by key
keys = url_params.keys()
keys.sort()
# Get the values in the same order of the sorted keys
values = map(url_params.get, keys)
# Reconstruct the URL parameters and encode them
url_string = urlencode(zip(keys,values))
#Construct the string to sign
string_to_sign = "GET\necs.amazonaws.com\n/onca/xml\n%s" % url_string
# Sign the request
signature = hmac.new(
key=AWS_SECRET_ACCESS_KEY,
msg=string_to_sign,
digestmod=hashlib.sha256).digest()
# Base64 encode the signature
signature = base64.encodestring(signature).strip()
# Make the signature URL safe
urlencoded_signature = quote_plus(signature)
url_string += "&Signature=%s" % urlencoded_signature
print "%s?%s\n\n%s\n\n%s" % (base_url, url_string, urlencoded_signature, signature)

Answer 2

Pycrypto will work fine - it's supported on App Engine, though the public ciphers are implemented in Python rather than C. You also ought to be able to use one of the existing AWS libraries, now that urlfetch/httplib are supported on App Engine.
I have an app that uploads images to S3, and I've implemented the request signing myself, but mostly because I wrote it before urlfetch/httplib were available. It works just fine, however.

Answer 3

Got this to work based on code sample at http://jjinux.blogspot.com/2009/06/python-amazon-product-advertising-api.html
Here is a minor improved version that lets you merge a dict of call specific params with the basic params before making the call.
keyFile = open('accesskey.secret', 'r')
# I put my secret key file in .gitignore so that it doesn't show up publicly
AWS_SECRET_ACCESS_KEY = keyFile.read()
keyFile.close()
def amz_call(self, call_params):
AWS_ACCESS_KEY_ID = '<your-key>'
AWS_ASSOCIATE_TAG = '<your-tag>'
import time
import urllib
from boto.connection import AWSQueryConnection
aws_conn = AWSQueryConnection(
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=Amz.AWS_SECRET_ACCESS_KEY, is_secure=False,
host='ecs.amazonaws.com')
aws_conn.SignatureVersion = '2'
base_params = dict(
Service='AWSECommerceService',
Version='2008-08-19',
SignatureVersion=aws_conn.SignatureVersion,
AWSAccessKeyId=AWS_ACCESS_KEY_ID,
AssociateTag=AWS_ASSOCIATE_TAG,
Timestamp=time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime()))
params = dict(base_params, **call_params)
verb = 'GET'
path = '/onca/xml'
qs, signature = aws_conn.get_signature(params, verb, path)
qs = path + '?' + qs + '&Signature=' + urllib.quote(signature)
print "verb:", verb, "qs:", qs
return aws_conn._mexe(verb, qs, None, headers={})
Sample usage:
result = self.amz_call({'Operation' : 'ItemSearch' , 'Keywords' : searchString , 'SearchIndex' : 'Books' , 'ResponseGroup' : 'Small' })
if result.status == 200:
responseBodyText = result.read()
# do whatever ...

Answer 4

See http://sowacs.appspot.com/AWS/Downloads/#python for a GAE Python signing service webapp. Uses native Python libraries.

Answer 5

I wrote another simple example that uses only the core python 3 libraries (not boto) and uses version 2 of the AWS signature protocol:
http://xocoatl.blogspot.com/2011/03/signing-ec2-api-request-in-python.html
I know it won't work in GAE, but might be useful for anyone just looking for AWS authentication examples like I was.

Answer 6

I use this one using pycrypto to generate a custom policy:
import json
import time
from Crypto.Hash import SHA
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from base64 import b64encode
url = "http://*"
expires = int(time.time() + 3600)
pem = """-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----"""
key_pair_id = 'APK.....'
policy = {}
policy['Statement'] = [{}]
policy['Statement'][0]['Resource'] = url
policy['Statement'][0]['Condition'] = {}
policy['Statement'][0]['Condition']['DateLessThan'] = {}
policy['Statement'][0]['Condition']['DateLessThan']['AWS:EpochTime'] = expires
policy = json.dumps(policy)
private_key = RSA.importKey(pem)
policy_hash = SHA.new(policy)
signer = PKCS1_v1_5.new(private_key)
signature = b64encode(signer.sign(policy_hash))
print '?Policy=%s&Signature=%s&Key-Pair-Id=%s' % (b64encode(policy),
signature,
key_pair_id)
This allows me to use one key for multiple items, something like:
http://your_domain/image1.png?Policy...
http://your_domain/image2.png?Policy...
http://your_domain/file1.json?Policy...
Don't forget to enable pycrypto by adding this lines to the app.yaml
libraries:
- name: pycrypto
version: latest