Machine to machine authentication with Google Cloud Endpoints - google-app-engine

Question

CONTEXT
Have created an API using Google Cloud Endpoints (Python) with which numerous low power devices will GET/POST data.
The only communication with the API will be from these custom devices (I own both ends of the communication).
RESEARCH
Looking at authentication, was hoping it would be as simple as using SSL/TLS client certs:
Each remote device will have a client cert signed by a single project CA anyway.
The Google cloud endpoints mandate SSL.
However, only oauth2 appears to be supported; I'm looking for a 'clean' way to implement 'hands off' authentication, ideally utilising the client SSL cert I already have on the client devices.
I have investigated creating 'service' oauth2 accounts, however as I want to protect against a device spoofing another device (one set of credentials for all is not acceptable), I would need to generate a service account for each client device, which would be bulky and horrible to maintain on the API-end.
It seems i'm looming towards needing to add a layer of authentication within my code for each API method, which somewhat defeats the point of utilising the services of Google's cloud endpoints.
QUESTION... Finally
Has anyone had experience in authenticating 'hands off' machine to machine devices at scale against google's cloud endpoint?
Does anyone know of a way of using a client certificate in the Oauth2 authentication process in a way which would be supported by GCE?
Is my only option going to be custom authentication within the API methods based on some crypto data in the POST/GET headers. (or just moving to hosting an API with Apache/NGINX and client-cert auth?)
Regards,
Matt

Answer 1

I wrote you an essay:
Consider that Cloud Endpoints basically exists in the application layer of the OSI model, since it communicates via HTTPS requests (it sends HTTP requests within a TLS session). Whether or not Endpoints uses HTTP or HTTPS is not a developer-configurable option - it must be HTTPS.
It uses HTTPS in that the API server has a TLS cert which is used to authenticate the API server. Inside the secure connection, the RPC params and responses are also secured from eavesdropping. This is the extent to which Endpoints "interacts" with TLS - it uses it to establish the session and send HTTP requests inside this session.
So, already I can tell you that you will not be able to have your TLS client certs (not an often-used feature) used to authenticate API clients automatically by endpoints, in the connection setup phase. TLS client certs simply aren't looked at or requested by the Endpoints API server.
Now, while authentication of the API server itself is guaranteed through the API server's TLS cert, authentication of API clients is done via Client IDs or the Users API, which sits in your code and abstracts over the different auth options App Engine offers at present:
OAuth (2.0)
OpenID
So, in order to auth your client devices in one of these two manners and still take advantage of Cloud Endpoints, you will need to find a way for each device to perform an OAuth flow or OpenID flow, your system having provisioned an identity for the respective auth method at the time of that device's initial deployment.
Google (Apps) Accounts option
This will involve creating a Google account (Google's unified SSO) or a Google Apps account managed by a custom domain for each device, and provisioning these accounts' credentials to each respective device. You can read more about custom domain authentication and App Engine auth configuration in general here.
OpenID option (general doc on OpenID with GAE)
This will involve setting up your own OpenID provider on a GCE instance using an OpenID connect library like pyoidc, so that you can provision accounts yourself, or it could involve registering accounts with a known OpenID provider for each device. The first solution is more robust but more time-consuming (OpenID providers can go down temporarily, or deactivate forever, and then your IOT network is out of luck).
Third option using Client IDs
You can of course generate an "installed application" client ID/secret and distribute these to each device in your network. They can use this to authenticate themselves as network devices (as opposed to an attacker's laptop), and then you trust devices to accurately report their own id as a param with each API call. Depending on how hackable your devices are and how widely you intend to distribute them, this scheme doesn't necessarily prevent devices from spoofing each other's id's, although depending on the id generation scheme, you can make it very difficult (each id being a long sufficiently long hash).
If you go this route and you're really concerned about this, you can provision a client ID for each device, but who knows if you'll hit some kind of undocumented limit on number of client IDs per app, and also this will require you to either do it by hand or write a script that logs into the dev console on a headless browser and does what you need.
Fourth crazy option that actually uses the TLS client certs
If you're really set on using both TLS client certs for auth and Cloud Endpoints for your API, you could try to send the client cert in the request, since TLS is encrypting the request data (unless your attacker has found a way to efficiently solve the inverse discrete logarithm problem, in which case they'll probably be too busy attacking more important targets (no offense) and changing the infosec game forever), and then reading and auth'ing the cert in your endpoints method somehow (third party libs uploaded with your app are probably necessary for this).
Fourth realistic option if you have your heart set on TLS client certs
Switch from App Engine to Compute Engine, where you basically have a VM managed and hosted in the same data-centers. On this box, you can implement any kind of connection protocol on any port you like, so you could have incoming API requests (not Endpoints, notice) TLS-authenticated based on teh connecting device's client certs.
Good luck!