Anonymous caller does not have storage.objects.get - google-app-engine

On Google App Engine (GAE) written in Python.
I am trying to issue an http post to cloud-speech-to-text api and using URI audio source (Google Cloud Storage Bucket Objects).
I am using the following headers;
Authorization: BASIC encoded_base64(username:password)
But I still keep getting an error response below:
{
"error": {
"code": 403,
"message": "Anonymous caller does not have storage.objects.get access to bucket_of_secrets/four_score_seven_years.flac.",
"status": "PERMISSION_DENIED"
}
}
So I have a couple of questions;
Does BASIC Authorization Header work in Google HTTP API?
What username:password should I use? Is it my GCP account email and password? i.e. handsome_dude#gmail.com:deluded_fool
Where handsome_dude#gmail.com is the username and deluded_fool is the password.
I've tried setting the bucket objects to be public readable and of course the http call works... but I would rather avoid setting my bucket objects public readable.
Here's a sample Curl request:
curl -X POST
https://speech.googleapis.com/v1/speech:longrunningrecognize?key=<secret_api_key> -d #sample.json -H "Content-Type: application/json, Authorization: Basic base64encodedusername:password"
Here's a snippet in my python code using urlfetch:
url_post = urlfetch.fetch(url=speech_to_text_url_post, payload=json.dumps(data_to_post), method=urlfetch.POST, headers={"Content-Type" : "application/json", "Authorization" : "Basic "+encoded_user_password})

1.Does BASIC Authorization Header work in Google HTTP API?
No, It is not working on Google APIs.
You need to attach OAuth2.0 accessToken to Authorization Header as bearer token like Authorization: Bearer ${yourAccessToken}.
I have 2 recommendations to develop some application running on gae.
Use ClientLibrary to call Google APIs.
You can use AppEngineDefaultCredential to call Google APIs.
Do not forget to set permissions to your AppEngineDefaultServiceAccount (${projectId}#appspot.gserviceaccount.com) before issue your request. You can configure those permissions on IAM page in cloud console.
Also I recommend you to read this page about How to authenticate your api call.

If you want to grant public access, you can make data public.
https://cloud.google.com/storage/docs/access-control/making-data-public
But the best way is generate Signed Urls, this way you can grant limit time access to an object.
https://cloud.google.com/storage/docs/access-control/signed-urls

For me, the issue was using role "Storage Legacy Object Reader", that, for whatever reason, was suggested. Changing it to "Storage Object Viewer" solved the issue.

I landed here because I forgot to run
./google-cloud-sdk/bin/gcloud init
from the Installation Guide.

Related

Azure Log Analytics API permissions on west-us2 region

We try to create an AAD service principal for retrieving data from out Log Analytics workspace.
Our AAD region is located in Germany
Log Analytics is located in North Europe
When attempting to create API permissions, the address to the API itself is mentioning westus2.api.loganalytics.io (west US region) which is a no go for our company data privacy.
Is there any reason of this default and not editable settings ? Is there any way to overcome it ?
Well, if so, you could get the token for ARM API endpoint, then call the ARM API.
In this way, no need to add the API permission for your AD App, just make sure your AD App has an RBAC role e.g. Contributor, Log Analytics Reader in the Access control (IAM) of your workspace, if not, follow this doc to add it.
Then use the client credential flow to get the token.
POST /YOUR_AAD_TENANT/oauth2/token HTTP/1.1
Host: https://login.microsoftonline.com
Content-Type: application/x-www-form-urlencoded
grant_type=client_credentials
&client_id=YOUR_CLIENT_ID
&redirect_uri=YOUR_REDIRECT_URI
&resource=https://management.azure.com/
&client_secret=YOUR_CLIENT_SECRET
After getting the token, use it to call the api like the sample below.
GET https://management.azure.com/subscriptions/6c3ac85e-59d5-4e5d-90eb-27979f57cb16/resourceGroups/demo/providers/Microsoft.OperationalInsights/workspaces/demo-ws/api/query
Authorization: Bearer <access_token>
Prefer: response-v1=true
{
"query": "AzureActivity | limit 10"
}
For more details, refer to this link.

Google Cloud Platform REST API: Acquiring Access Token and API Key

I wish to use the Google Cloud Platform (GCP) REST API locally, starting with the apps.services.versions.instances.list method.
The route works when I use "Try this API" here, but how would I use this method locally with curl?
"https://appengine.googleapis.com/v1/apps/$APPSID/services/$SERVICESID/versions/$VERSIONSID/instances?key=$YOUR_API_KEY" \
--compressed \
--header 'Accept: application/json' \
--header "Authorization: Bearer $YOUR_ACCESS_TOKEN"
#=>
{
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
How do I access $YOUR_API_KEY and $YOUR_ACCESS_TOKEN? I have been unable to find either in the official GCP docs.
The fastest way is use Cloud Shell:
List projects to get project id
gcloud projects list
# save you project id
PROJECT_ID="YOURS_PROJECT_ID"
Get ACCESS_TOKEN
ACCESS_TOKEN=$(gcloud auth print-access-token)
Get API_KEY
API_KEY=$(curl -X POST https://apikeys.googleapis.com/v1/projects/$PROJECT_ID/apiKeys?access_token=$ACCESS_TOKEN | jq -r ".currentKey")
Print API_KEY and ACCESS_TOKEN
echo $ACCESS_TOKEN
echo $API_KEY
To run above commands on local machine first you need authenticate using command gcloud auth login and follow instructions.
Alternatively api key could be readed or created from console go to Navigation Menu -> APIs & Services -> Credentials and next click on CREATE CREDENTIALS -> API Key.
By reading the documentation (clicking on question mark next to Credentials) we can read:
[YOUR_API_KEY] - "Include an API Key to identify your project, used to verify enablement and track request quotas."
[YOUR_ACCESS_TOKEN] - "Include an access (bearer) token to identify the user which completed the OAuth flow with your Client ID."
You no longer need an API key. It's a legacy feature of Google APIs, provide only an access token is enough.
In command line you can do this
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" https://....
All the Google Cloud APIs are compliant with access token authentication. Few are still compliant with API keys.
About APIKeys API
This API has been published in beta and now closed. At least the documentation part. I don't know if this API is stable or subject to change. You can create an API key per API like this (very similar to Bartosz Pelikan answer)
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
-X POST https://apikeys.googleapis.com/v1/projects/PROJECT_ID/apiKeys
As you can see, I reuse the access token authentication mode
The above answers are using an API that isn't publicly available (I reached out to GCP support an confirmed.
I recommend using the CLI tool like so:
gcloud app instances list --service core-api --project my-project-name
docs: https://cloud.google.com/sdk/gcloud/reference/app/instances/list
You'll have to a gcloud auth first and probably set your project.

Google Vision API call via Browser for Document Text Detection from File stored in Firestore

API call from the browser (React with Firebase auth) to the Google Vision API in order to get the content of a .pdf file which is stored in the Firestore database. The result should be stored as .json in Firestore. A service account was created and it has system wide access. The expected response looks like this:
{
"name": "projects/usable-auth-library/operations/1efec2285bd442df"
}
The response I get is a 403 which indicates that something in the authenication process went wrong. In Firestore no .json with the text content is created.
The function for the call looks like this:
const test = () => {
fetch("https://vision.googleapis.com/v1/files:asyncBatchAnnotate", {
method: "post",
requests: [
{
inputConfig: {
gcsSource: {
uri: "gs://XXXX.appspot.com/images/XXXX.pdf"
},
mimeType: "application/pdf"
},
features: [
{
type: "DOCUMENT_TEXT_DETECTION"
}
],
outputConfig: {
gcsDestination: {
uri: "gs://XXXX.appspot.com/images/output"
},
batchSize: 1
}
}
]
}).then(res => console.log(res))
Any idea what I am doing wrong? Or is there a React library which handles that process out of the box or a more detailed step by step guide to make such client side calls to the API? I had a look at the npm package #google-cloud/vision but this seems not to work on the client side yet.
You can find and example here on how to use the api with curl.
curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
https://vision.googleapis.com/v1/images:annotate -d #request.json
Therefore you should use your service account to sign a jwt token and receive an access token from Google Cloud. Then you should use your access token in the post request.
Your application prepares to make authorized API calls by using
the service account's credentials to request an access token from the
OAuth 2.0 auth server.
Finally, your application can use the access token to call Google
APIs.
Using OAuth 2.0 for Server to Server Applications
Please follow Firebase Authenticate REST Requests for an example on how to generate an access token with node.js
The service account only needs to have access to the GCS bucket you interacting with. It seems like you aren't using your service account though.
Typically, you wouldn't have the service account or api key in your frontend because then someone could steal/use your service account/api key and you have to pay for the charges. Better to call Vision API from your backend.

JIRA Cloud REST API (OAuth 2.0) Error 403 on POST Requests

I am trying to connect my React app to the Jira Cloud API and can't seem to get past a 403 error.
My code currently does a Auth dance using OAuth 2.0 and returns the token and cloudid. I can use this to GET issues, however POST request (like creating an issue) return with 403. I have found here that this error is returned if the user does not have the necessary permission to access the resource or run the method.
I have ensured the user has the correct scope ([write: jira-work, read: jira-work]) and verified this is reflected in the user account (in their account > connect apps tab).
My app is not linked (via ApplicationLink) or installed (via Apps, Manage Apps), is this necessary to perform POST requests?
Here is a sample of my code:
fetch(`https://api.atlassian.com/ex/jira/${jira.cloudid}/rest/api/2/issue/`, {
method: "POST",
headers: {
"Content-Type": 'application/json',
"Authorization": `Bearer ${jira.token}`
},
body: JSON.stringify(data)
})
.then(...)
Neither api version 2 or 3 are working for this POST request. I have explored using Basic Auth however this fails due to CORS errors.
I have verified that the POST request does work in POSTMAN (using the cloudid and token).
---------------------------------------------------------------------------------------------------------------------------
UPDATE
After talking to Atlassian Staff, there is an issue within their API security:
"By trying the same thing you mentioned I think I found what the problem is. Your request likely fails with a ‘XSRF check failed’ in the browser.
I’ve already talked to one of our security engineers and we quickly dived into the implementation code to confirm why this not working and what would need to be changed on our side. We’ve also already opened a engineering ticket to get this addressed. This will likely take a few weeks to get addressed, but I’ll keep you posted if I hear any updates!"
The XSRF check failed was the main error for my 403 response. I'll post any updates I receive and answer the question when a resolution is found.
This has apparently been resolved. Follow the discussion here: https://community.developer.atlassian.com/t/jira-cloud-rest-api-oauth-2-0-error-403-on-post-requests/25621/4

Create Google Cloud Project with Cloud Resource Manager API

I'm trying to create a new project in the Google Cloud Platform using the Cloud Resource Manager API.
It all works fine when I use it through the API explorer however I don't quite understand how to use it as an http request outside of API Explorer.
I run the request like this:
curl -H "Content-Type: application/json" -X POST -d '{"name": "project example","projectId": "my-project-example-1234"}' https://cloudresourcemanager.googleapis.com/v1/projects?fields=response&key={MY_APY_KEY}
Response:
{
"error": {
"code": 401,
"message": "The request does not have valid authentication credentials.",
"status": "UNAUTHENTICATED"
}
}
The documentation says that this request requires an OAuth scope and that's when things get confusing to me.
Reading the documentation I could not understand how one of the required OAuth scopes can be passed with the URL when making the http request to the rest API which I'm only assuming is what I'm missing.
Rather than just tell you how to test with a working token, I'm going to try to more broadly answer what you're aiming to do.
At a pretty high level, you will need to:
Enable the Resource Manager API for your Cloud Console project.
Create an OAuth client ID for Web applications in the Cloud Console. You will need to register your authorized redirect URI. This is where your app will get the OAuth response back from Google when the end user authorizes your app. Note the client ID, you will need that next.
Start the OAuth flow by assembling your URL:
https://accounts.google.com/o/oauth2/v2/auth?
response_type=code&
client_id=<123456789example>.apps.googleusercontent.com&
scope=https://www.googleapis.com/auth/cloudplatformprojects&
redirect_uri=http://<YOUR-APP-URL>/<YOUR-OAUTH-HANDLER>
Replace in that URL the client ID and the redirect URI. I assume you'd have a button or link on your site where you would have the user click to start this flow.
Code your OAuth handler. Some more in-depth code for doing this in Go can be gleaned from this Go Sample, which was originally for G+ sign-in but much of the logic is going to be the same. You are going to get a code query parameter passed to your application, the value is a one-time authorization code that your application must exchange for your OAuth tokens that you use to make API calls on behalf of the user.
If appropriate for your app and situation, securely store your tokens for use later or for processing while your user is not active on your site (might be appropriate for batch processing).
Now that you have an access token, you can pass that to the Resource Manager API and create projects on behalf of the user. You might use the Go client library or you could call the HTTP endpoints directly in your code.
If you want more testing with curl, I'd follow the process that we wrote up accessing the App Engine Admin API. Substitute Admin API URLs and names for Resource Manager and you've got the overall flow. The difference from what's above, is I used a code flow above because I assume you want server-side and possibly refresh tokens if you need to be able to make these API calls while the user is not active on your site.
Like Alex says, you ask for scopes during OAuth authentication. One way to easily authenticate and obtain a Oauth access token is doing:
gcloud beta auth application-default login --scopes=https://www.googleapis.com/auth/cloudplatformprojects
As you can see, you can specify the scopes you want to gcloud and it will take care of authentication for you.
Then, you should be able to create a project calling:
curl -H "Content-Type: application/json" -H "Authorization: Bearer $(gcloud beta auth application-default print-access-token)" -X POST -d '{"name": "project example","projectId": "my-project-example-1234"}' https://cloudresourcemanager.googleapis.com/v1/projects?fields=response
Here, you are passing the access token obtained when you made Oauth authentication. This should be taken care of by the client libraries for you when you get the application default credentials.

Resources