I am trying to use aurora serverless data API feature to reduce the db connection time in my serverless application. But building client is taking time.
I would like to call rds HTTP service via lambda to get/post data.
I came across some was posts but I am still getting error, missing authentication token
https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html
My sample request is below for MySQL. I have run this via AWS cloud shell.
curl --location --request POST 'https://rds-data.us-west-2.amazonaws.com/Execute' \
--header 'Content-Type: application/json' \
--data-raw '{
"continueAfterTimeout": false,
"database": "demo_data",
"includeResultMetadata": true,
"parameters": [],
"resourceArn": "arn:aws:rds:us-west-2:*******:cluster:rds-serverless",
"schema": "demo_data",
"secretArn": "arn:aws:secretsmanager:us-west-2:******:secret:serverless/user_u-cMt2Q4",
"sql": "select now()"
}'
If I can recommend a different approach--AWS has already done a lot of work for you, so you shouldn't need to construct your own access mechanism for the data api.
For instance, from the command line you can access the data api using the aws cli application (available on mac, windows, linux, etc) like this:
aws rds-data execute-statement --resource-arn "arn:aws:rds:us-east-1:123456789012:cluster:mydbcluster" \
--database "mydb" --secret-arn "arn:aws:secretsmanager:us-east-1:123456789012:secret:mysecret" \
--sql "select * from mytable"
For access to the data api from within a lambda, you would usually want to pull in the sdk for whatever language your lambda is setup to run and then to use their libraries to access the data api and other services.
If you are dead-set on using the command line from within lambda you could even create a lambda layer that includes the aws cli mentioned above and then use a system command to call the api that way, though I wouldn't recommend it.
Hopefully one of these easier solution will work better for you than curl!
Related
I wish to use the Google Cloud Platform (GCP) REST API locally, starting with the apps.services.versions.instances.list method.
The route works when I use "Try this API" here, but how would I use this method locally with curl?
"https://appengine.googleapis.com/v1/apps/$APPSID/services/$SERVICESID/versions/$VERSIONSID/instances?key=$YOUR_API_KEY" \
--compressed \
--header 'Accept: application/json' \
--header "Authorization: Bearer $YOUR_ACCESS_TOKEN"
#=>
{
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
How do I access $YOUR_API_KEY and $YOUR_ACCESS_TOKEN? I have been unable to find either in the official GCP docs.
The fastest way is use Cloud Shell:
List projects to get project id
gcloud projects list
# save you project id
PROJECT_ID="YOURS_PROJECT_ID"
Get ACCESS_TOKEN
ACCESS_TOKEN=$(gcloud auth print-access-token)
Get API_KEY
API_KEY=$(curl -X POST https://apikeys.googleapis.com/v1/projects/$PROJECT_ID/apiKeys?access_token=$ACCESS_TOKEN | jq -r ".currentKey")
Print API_KEY and ACCESS_TOKEN
echo $ACCESS_TOKEN
echo $API_KEY
To run above commands on local machine first you need authenticate using command gcloud auth login and follow instructions.
Alternatively api key could be readed or created from console go to Navigation Menu -> APIs & Services -> Credentials and next click on CREATE CREDENTIALS -> API Key.
By reading the documentation (clicking on question mark next to Credentials) we can read:
[YOUR_API_KEY] - "Include an API Key to identify your project, used to verify enablement and track request quotas."
[YOUR_ACCESS_TOKEN] - "Include an access (bearer) token to identify the user which completed the OAuth flow with your Client ID."
You no longer need an API key. It's a legacy feature of Google APIs, provide only an access token is enough.
In command line you can do this
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" https://....
All the Google Cloud APIs are compliant with access token authentication. Few are still compliant with API keys.
About APIKeys API
This API has been published in beta and now closed. At least the documentation part. I don't know if this API is stable or subject to change. You can create an API key per API like this (very similar to Bartosz Pelikan answer)
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-X POST https://apikeys.googleapis.com/v1/projects/PROJECT_ID/apiKeys
As you can see, I reuse the access token authentication mode
The above answers are using an API that isn't publicly available (I reached out to GCP support an confirmed.
I recommend using the CLI tool like so:
gcloud app instances list --service core-api --project my-project-name
docs: https://cloud.google.com/sdk/gcloud/reference/app/instances/list
You'll have to a gcloud auth first and probably set your project.
Let us assume that there is an app engine standard python app hosted at https://xyz.appspot.com and that its URLs are protected with:
login: admin
secure: always
How can I exercise the APIs using curl? I guess the real question is how can I authenticate to the app using curl. If the app is used from a browser, one is redirected to Google login but I am wondering how I can simulate the same from curl.
Any help is greatly appreciated.
Thanks,
Raghu
One way would be to do the authentication in browser first, and then copy the cookie from there to curl. For example in Chrome, you can open the devtools (F12) and select the Network tab.
When you access your secure resource it will appear there. Then you can right click -> Copy -> Copy as cURL (bash).
This will give you a cURL command that is authorized to call your secure resource.
Based on the suggestion from #Erfa, I visited the site in Chrome while keeping the dev tools open.
The browser takes you through login procedure and the site appears. At this point, right click on the GET request in "Network" tab and select "Save as HAR with Content" which saves the API information in a text file.
In the file, you will find a cookie that is being sent with the GET request. You can now use this same cookie with curl as follows:
$ curl --cookie "NAME=VALUE" <URL>
You can use a combination of Cloud Endpoints and API Key.
In this article https://cloud.google.com/endpoints/docs/frameworks/python/restricting-api-access-with-api-keys-frameworks from Google Cloud Platform you have an example of how to use curl authentication with this combination:
If an API or API method requires an API key, supply the key using a
query parameter named key, as shown in this cURL example:
curl \
-H "Content-Type: application/json" \
-X POST \
-d '{"message": "echo"}' \
"${HOST}/_ah/api/echo/v1/echo_api_key?key=${API_KEY}
where HOST and API_KEY are variables containing your API host name and API key,
respectively. Replace echo with the name of your API, and v1 with the
version of your API.
I'm trying to create a new project in the Google Cloud Platform using the Cloud Resource Manager API.
It all works fine when I use it through the API explorer however I don't quite understand how to use it as an http request outside of API Explorer.
I run the request like this:
curl -H "Content-Type: application/json" -X POST -d '{"name": "project example","projectId": "my-project-example-1234"}' https://cloudresourcemanager.googleapis.com/v1/projects?fields=response&key={MY_APY_KEY}
Response:
{
"error": {
"code": 401,
"message": "The request does not have valid authentication credentials.",
"status": "UNAUTHENTICATED"
}
}
The documentation says that this request requires an OAuth scope and that's when things get confusing to me.
Reading the documentation I could not understand how one of the required OAuth scopes can be passed with the URL when making the http request to the rest API which I'm only assuming is what I'm missing.
Rather than just tell you how to test with a working token, I'm going to try to more broadly answer what you're aiming to do.
At a pretty high level, you will need to:
Enable the Resource Manager API for your Cloud Console project.
Create an OAuth client ID for Web applications in the Cloud Console. You will need to register your authorized redirect URI. This is where your app will get the OAuth response back from Google when the end user authorizes your app. Note the client ID, you will need that next.
Start the OAuth flow by assembling your URL:
https://accounts.google.com/o/oauth2/v2/auth?
response_type=code&
client_id=<123456789example>.apps.googleusercontent.com&
scope=https://www.googleapis.com/auth/cloudplatformprojects&
redirect_uri=http://<YOUR-APP-URL>/<YOUR-OAUTH-HANDLER>
Replace in that URL the client ID and the redirect URI. I assume you'd have a button or link on your site where you would have the user click to start this flow.
Code your OAuth handler. Some more in-depth code for doing this in Go can be gleaned from this Go Sample, which was originally for G+ sign-in but much of the logic is going to be the same. You are going to get a code query parameter passed to your application, the value is a one-time authorization code that your application must exchange for your OAuth tokens that you use to make API calls on behalf of the user.
If appropriate for your app and situation, securely store your tokens for use later or for processing while your user is not active on your site (might be appropriate for batch processing).
Now that you have an access token, you can pass that to the Resource Manager API and create projects on behalf of the user. You might use the Go client library or you could call the HTTP endpoints directly in your code.
If you want more testing with curl, I'd follow the process that we wrote up accessing the App Engine Admin API. Substitute Admin API URLs and names for Resource Manager and you've got the overall flow. The difference from what's above, is I used a code flow above because I assume you want server-side and possibly refresh tokens if you need to be able to make these API calls while the user is not active on your site.
Like Alex says, you ask for scopes during OAuth authentication. One way to easily authenticate and obtain a Oauth access token is doing:
gcloud beta auth application-default login --scopes=https://www.googleapis.com/auth/cloudplatformprojects
As you can see, you can specify the scopes you want to gcloud and it will take care of authentication for you.
Then, you should be able to create a project calling:
curl -H "Content-Type: application/json" -H "Authorization: Bearer $(gcloud beta auth application-default print-access-token)" -X POST -d '{"name": "project example","projectId": "my-project-example-1234"}' https://cloudresourcemanager.googleapis.com/v1/projects?fields=response
Here, you are passing the access token obtained when you made Oauth authentication. This should be taken care of by the client libraries for you when you get the application default credentials.
I am new to Salesforce and SF workbench.
I have been given read access and login credentials to the SF workbench for one of out clients.
I know I can use SF Rest API to extract data from Salesforce object, but I guess I would need OAuth token to be able to do that.
Is there a way to extract those tokens from the workbench to do simple rest api curl commands?
If you setup a Connected App in Salesforce (any org, including a developer org) and enable OAuth then you can use the following curl command to create an OAuth token from a username and password:
curl https://login.salesforce.com/services/oauth2/token \
-d "grant_type=password" \
-d "client_id=YOUR_OAUTH_CONSUMER_KEY" \
-d "client_secret=YOUR_OAUTH_CONSUMER_SECRET" \
-d "username=YOUR_USERNAME" \
-d "password=YOUR_PASSWORD"
From Workbench if you select Session Information from the Info menu and expand the Connection folder you will see a Session Id value this is your current session token and it can be used for Curl REST commands.
You could also use the REST Explorer found under the Utilities menu to do your simple REST API commands.
Is it recommended to use the parse.com API for a mobile application while using other databases such as amazon simpleDB for most of the app back-end data?
Basically, parse.com would only be used for its login/register user system.
That leads to some questions such as "can our own server check a parse.com sessionToken validity before returning data?"
Anyway, anyone has used parse.com in combination with a bigger cloud service such as AWS?
Many Thanks
can our own server check a parse.com sessionToken validity before returning data?
I think you can achieve this using Parse's Cloud Code.
You could try something like:
Parse.Cloud.define("validateUser", function(request, response) {
var myname = request.params.myname;
var mypass = request.params.mypass;
Parse.User.logIn(myname, mypass, {
success: function(user) {
// Do stuff after successful login.
},
error: function(user, error) {
// The login failed. Check error to see why.
}
});
});
And test it with the following curl (be sure to replace the App ID and Rest API Key):
curl -X POST \
-H "X-Parse-Application-Id: <YOUR_PARSE_APP_ID_HERE>" \
-H "X-Parse-REST-API-Key: <YOUR_PARSE_REST_API_KEY_HERE>" \
-H "Content-Type: application/json" \
-d '{"myname":"johnPooter", "mypass":"pooter123"}' \
https://api.parse.com/1/functions/validateUser
As to whether it is recommended or not to use other back-ends with Parse: I think it is fine but it defeats the purpose of using Parse. Parse is good when you are trying to throw something together quickly and don't want to invest resources towards solving common back-end problems.