My application was using the Watson Conversation API (21st Apr 2017 version) imported as a maven dependency:
<dependency>
<groupId>com.ibm.watson.developer_cloud</groupId>
<artifactId>conversation</artifactId>
<version>3.9.1</version>
</dependency>
I've checked the credentials (username and password) for correctness. However, it is still giving me the following error (I've replaced the actual conversation workspace id with the placeholder ACTUAL_WORKSPACE_ID in URLs):
Feb 12, 2020 6:14:40 PM okhttp3.internal.platform.Platform log
INFO: <-- 401 Unauthorized https://gateway.watsonplatform.net/conversation/api/v1/workspaces/ACTUAL_WORKSPACE_ID/message?version=2017-04-21 (342ms, 37-byte body)
Feb 12, 2020 6:14:40 PM com.ibm.watson.developer_cloud.service.WatsonService processServiceCall
SEVERE: POST https://gateway.watsonplatform.net/conversation/api/v1/workspaces/ACTUAL_WORKSPACE_ID/message?version=2017-04-21, status: 401, error: Unauthorized
Exception in thread "main" com.ibm.watson.developer_cloud.service.exception.UnauthorizedException: Unauthorized: Access is denied due to invalid credentials. Tip: Did you set the Endpoint?
at com.ibm.watson.developer_cloud.service.WatsonService.processServiceCall(WatsonService.java:410)
I observed that the newer version of APIs is available and the IAM & Resource groups have been introduced in the IBM Cloud. Is this issue a result of these changes?
Can we not use the older API any more?
IBM Cloud is switching off any services that have not been migrated to a resource group. The Watson services were part of the first batch affected. You will need to migrate your instance to IAM, then use the generated IAM key for authentication.
Related
I am using maven plugin 1.9.60 with JDK 8
Running command
mvn install appengine:update -Dappengine.additionalParams="--service_account_json_key_file=client-secret.json"
I could see the version deployed but 0% traffic. Logs says error as below. Could someone suggest how to fix this?
Beginning interaction for module default...
0% Created staging directory at: '/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg7082099622802800350.tmp'
5% Scanning for jsp files.
8% Generated git repository information file.
20% Scanning files on local disk.
25% Initiating update.
28% Cloning 34 application files.
40% Uploading 2 files.
52% Uploaded 1 files.
61% Uploaded 2 files.
68% Sending batch containing 2 file(s) totaling 1KB.
73% Initializing precompilation...
90% Deploying new version.
95% Closing update: new version is ready to start serving.
98% Uploading index definitions.
Feb. 19, 2018 12:54:48 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #0
Feb. 19, 2018 12:54:48 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #1
Feb. 19, 2018 12:54:49 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #2
Feb. 19, 2018 12:54:49 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
This is try #3
Error Details:
2018-02-19 00:54:17.700:INFO::main: Logging initialized #409ms
2018-02-19 00:54:17.816:INFO:oejs.Server:main: jetty-9.3.18.v20170406
2018-02-19 00:54:19.481:INFO:oeja.AnnotationConfiguration:main: Scanning elapsed time=1228ms
2018-02-19 00:54:19.497:INFO:oejq.QuickStartDescriptorGenerator:main: Quickstart generating
2018-02-19 00:54:19.512:INFO:oejsh.ContextHandler:main: Started o.e.j.q.QuickStartWebApp#2aceadd4{/,file:///private/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg7082099622802800350.tmp/,AVAILABLE}
2018-02-19 00:54:19.514:INFO:oejs.Server:main: Started #2231ms
2018-02-19 00:54:19.517:INFO:oejsh.ContextHandler:main: Stopped o.e.j.q.QuickStartWebApp#2aceadd4{/,file:///private/var/folders/ny/z92xw4ps0j71v43mnvjzjyd80000gn/T/appcfg7082099622802800350.tmp/,UNAVAILABLE}
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
Unable to update app: Error posting to URL: https://appengine.google.com/api/datastore/index/add?app_id=clean-aleph-191303&version=beta-001&
403 Forbidden
You do not have permission to modify this app (app_id=u'f~clean-aleph-191303').
appengine-web.xml
```
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>clean-aleph-191303</application>
<version>beta-001</version>
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
</appengine-web-app>
```
pom.xml (plugin declaration)
```
<plugin>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-maven-plugin</artifactId>
<version>1.9.60</version>
</plugin>
```
Complete bitbucket repo can be found here.
https://bitbucket.org/rohgoyal/springbootgcp-example/src
The issue you are experiencing is due to a missing role on the service account. It probably has the "App Engine Deployer" role assigned but it is missing the "Datastore Index Admin" role and this causes the 403s while POSTing to appengine.google.com/api/datastore/index/
After adding that role, you'll need to regenerate the private key (client-secret.json as you've named it) because the old one won't have the new permissions.
Some more information on roles needed to deploy GAE apps can be found on Deploying using IAM roles section.
Note: Thanks for providing a full repro. It is always helpful.
It is insanely hard, to deploy an app to Google App Engine, using Google Cloud SDK.
I had tried the below 2 commands
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud config set project jstock-android
Updated property [core/project].
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud app deploy app.yaml --log-http --verbosity=debug
DEBUG: Running [gcloud.app.deploy] with arguments: [--log-http: "true", --verbosity: "debug", DEPLOYABLES:1: "['app.yaml']"]
DEBUG: No staging command found for runtime [python27] and environment [STANDARD].
DEBUG: API endpoint: [https://appengine.googleapis.com/], API version: [v1]
=======================
==== request start ====
uri: https://appengine.googleapis.com/v1/apps/jstock-android?alt=json
method: GET
== headers start ==
Authorization: Bearer ya29.GlxEBb1XVP1JK93-ARiaN_ZgiMbvZmw5KWfvJVfibDJ4FK_ZaMRoU1jVDTiWzsY606GSduJKJd9Nm8zA-_Iql5mGn4AMk4QVl8mPRycfekeZnOOHtbUvpkBMgOLOQA
accept: application/json
accept-encoding: gzip, deflate
content-length: 0
user-agent: google-cloud-sdk x_Tw5K8nnjoRAqULM9PFAC2b gcloud/184.0.0 command/gcloud.app.deploy invocation-id/c9ae232d33b346d787b95a36e28c38c0 environment/None environment-version/None interactive/True python/2.7.13 (Windows NT 10.0.16299)
== headers end ==
== body start ==
== body end ==
==== request end ====
---- response start ----
-- headers start --
-content-encoding: gzip
alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
cache-control: private
content-length: 335
content-type: application/json; charset=UTF-8
date: Tue, 16 Jan 2018 19:16:21 GMT
server: ESF
status: 403
transfer-encoding: chunked
vary: Origin, X-Origin, Referer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
-- headers end --
-- body start --
{
"error": {
"code": 403,
"message": "Operation not allowed",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"resourceType": "gae.api",
"description": "The \"appengine.applications.get\" permission is required."
}
]
}
}
-- body end --
total round trip time (request+response): 1.796 secs
---- response end ----
----------------------
DEBUG: (gcloud.app.deploy) Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 797, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 757, in Run
resources = command_instance.Run(args)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 65, in Run
parallel_build=False)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 543, in RunDeploy
app = _PossiblyCreateApp(api_client, project)
File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 703, in _PossiblyCreateApp
api_client._FormatApp()))) # pylint: disable=protected-access
HttpException: Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
ERROR: (gcloud.app.deploy) Permissions error fetching application [apps/jstock-android]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
C:\Users\yccheok\Desktop\jstock-android-appengine>
Then, I went through https://cloud.google.com/appengine/docs/admin-api/accessing-the-api , it mentioned I need to use Admin API. So, I do it step by step carefully.
Step 1
Step 2
It mentions Admin API is enabled. Now I need credential.
Step 3
Step 4
OK. Now they mention I don't need create new credential. I can use Application Default Credentials ?!
Step 5
So, I went to https://developers.google.com/identity/protocols/application-default-credentials?hl=en_GB . I learn that I need to run
C:\Users\yccheok\Desktop\jstock-android-appengine>gcloud auth application-default login
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/auth?redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&prompt=select_account&response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform&access_type=offline
Credentials saved to file: [C:\Users\yccheok\AppData\Roaming\gcloud\application_default_credentials.json]
These credentials will be used by any library that requests
Application Default Credentials.
Step 6
Step 7
Still, after completing the above 7 steps, I still get the exact same error message, when trying to run
gcloud app deploy app.yaml --log-http --verbosity=debug
Can anyone let me know, what step I'm still require, in order to deploy my Python app to Google App Engine, using Google Cloud SDK?
The Admin API is for programmatically deploying the app, not for deploying using gcloud app deploy, for which you don't even need the Admin API enabled for your app.
From Deploying a Python App:
To programmatically deploy your apps, use the Admin API.
Before you begin
Before you can deploy your app:
The Owner of the GCP project must create the App Engine application.
Ensure that your user account includes the required privileges.
(but I can see how the above could be mis-intrepreted as an invitation to use the Admin API)
Most likely the account actually used by gcloud app deploy is missing or doesn't have the required permissions.
You can check the acount used with gcloud auth list. In my case the account is my email address, not a service account (I'm not sure if a service account can be used).
Use gcloud auth login (and maybe gcloud auth revoke) if you need a different account.
And you can check the account's privileges (if any) on the project/app on the IAM Page.
One additional note on this,
When you enable the App Engine API and the cloud builder api, make sure the Cloud Build Service Account also has access to the project.
I ran into that problem after enabling the correct apis.
This was using a build trigger. I could deploy locally from a command line because I was authenticated as myself. However, if you are using a build trigger, it will use the build service account, which needs access.
Hope this helps.
I had this issue. In my case it was solved by setting the project using the project ID, rather than the project name. See this answer
gcloud app deploy ERROR: Permissions error fetching application [apps/<PROJECT_NAME>]
We are using a service account to deploy our app to App Engine using Travis.
On every merged PR, Travis pulls the code from our GitHub repository, and pulls a Docker image which contains Google Cloud SDK and executes the gcloud app deploy command. We use a Service Account to perform the deployment with "Project Owner" role.
Everything used to work fine until I added a new service to the project which automates SSL certificate generation and renewal, along with a dispatch.yaml file to route traffic incoming from Let's Encrypt for domain verification. I needed to add more permissions to allow updating the SSL certificates we use for our custom domain. I removed the current service account, and created a new one with a new private key. I created a new role with the required permissions to update and view SSL certificates in addition to the previous permissions (all appengine.* permissions). I assigned the new role and the Project Owner role to the new account. After these changes, the deployment fails with the following error when executing the deploy command:
Permissions error fetching application [apps/hollowverse-c9cad]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
I used the same service account on my local machine with logging level set to debug. I got this error:
DEBUG: HttpError accessing <https://appengine.googleapis.com/v1/apps/hollowverse-c9cad?alt=json>: response: <{'status': '403', 'content-length': '335', 'x-xss-protection': '1; mode=block', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'vary': 'Origin, X-Origin, Referer', 'server': 'ESF', '-content-encoding': 'gzip', 'cache-control': 'private', 'date': 'Wed, 02 Aug 2017 14:33:50 GMT', 'x-frame-options': 'SAMEORIGIN', 'alt-svc': 'quic=":443"; ma=2592000; v="39,38,37,36,35"', 'content-type': 'application/json; charset=UTF-8'}>, content <{
"error": {
"code": 403,
"message": "Operation not allowed",
"status": "PERMISSION_DENIED",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"resourceType": "gae.api",
"description": "The \"appengine.applications.get\" permission is required."
}
]
}
}
>
DEBUG: (gcloud.beta.app.deploy) Permissions error fetching application [apps/hollowverse-c9cad]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.
The description says that appengine.applications.get is required to perform the deployment. Looking at the permissions granted to the role assigned to the Travis account we use to deploy, appengine.applications.get is clearly granted:
I assigned every possible App Engine and Project role to the account, but deployment still fails with the same error. However, using the default service account, which is automatically created for every new project on GCP, seems to be working.
I removed the current service account, and created a new one with a new private key.
This is where it went wrong. The new account had the same ID as the previous one. Although I could not find this behavior documented anywhere, it looks like that once an ID is used for a service account, it cannot be used again for a new account, even if the previous one is removed.
We created a new account with a new ID (travis2#hollowverse-c9cad.iam.gserviceaccount.com) instead of travis#hollowverse-c9cad.iam.gserviceaccount.com) and the issue is now resolved.
I am using Google AppEngine to run an application. I am now trying to setup the Google Cloud Datalab. I have the Compute Engine enabled. When I try to deploy, I get the following error in the log:
...
Jan 26 21:48:20 datalab-deploy-main-20160126-21-46-14 startupscript: Verifying that Managed VMs are enabled and ready.
Jan 26 21:48:22 datalab-deploy-main-20160126-21-46-14 startupscript: If this is your first deployment, this may take a while...#015If this is your first deployment, this may take a while.../#015If this is your first deployment, this may take a while...-#015If this is your first deployment, this may take a while...done.
Jan 26 21:48:22 datalab-deploy-main-20160126-21-46-14 startupscript: WARNING: If this is your first deployment, please try again.
Jan 26 21:48:22 datalab-deploy-main-20160126-21-46-14 startupscript: ERROR: (gcloud.preview.app.deploy) Server responded with code [400]:
Jan 26 21:48:22 datalab-deploy-main-20160126-21-46-14 startupscript: Bad Request Unexpected HTTP status 400.
Jan 26 21:48:22 datalab-deploy-main-20160126-21-46-14 startupscript: Failed Project Preparation (app_id='s~prjectId'). App Engine service account has insufficient permissions for project. Developer or owner required.
Any ideas what to do?
We have a project that runs on App Engine and creates files on Cloud Storage. The two are connected as being part of the same cloud platform project.
In App Engine we have a "Google APIs Console Project Number", and in Cloud Console -> Credentials we have that project number listed under "Client ID" (1[..........].apps.googleusercontent.com) and "Email Address" (1[..........]#developer.gserviceaccount.com).
Every morning, we have some cron jobs that upload files to our Cloud Storage bucket. This has worked flawlessly since September 2013 but as of this morning (Oct 16, 2014) we're getting "permission denied" errors from Cloud Storage.
We're using the cloudstorage client library, which raises cloudstorage.ForbiddenError. Here's the log & exception output:
Expect status [201] from Google Storage. But got status 403.
Path: u'/bucketname/icon_20141016.png'.
Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': 'image/png', 'accept-encoding': 'gzip, *'}.
Response headers: {'alternate-protocol': '443:quic,p=0.01', 'content-length': '151', 'via': 'HTTP/1.1 GWA', 'x-google-cache-control': 'remote-fetch', 'vary': 'Origin', 'server': 'UploadServer ("Built on Oct 9 2014 15:35:27 (1412894127)")', 'date': 'Thu, 16 Oct 2014 11:56:10 GMT', 'content-type': 'application/xml; charset=UTF-8'}.
Extra info: None.
Since we're using the Cloud platform connection between the two services, I feel like I can only diagnose the problem on my production App Engine instance. I would prefer not to deploy new versions and risk breaking a production server. This also appears to be a Cloud Storage issue this morning, but the only status page I could find says everything is working fine.
As #tx802 suggested, I checked the bucket ACLs carefully.
$ gsutil getacl gs://bucket
<Entry>
<Scope type="UserByEmail">
<EmailAddress>1[..........]#developer.gserviceaccount.com</EmailAddress>
</Scope>
<Permission>FULL_CONTROL</Permission>
</Entry>
I looked at the App Engine application settings and saw the service account is actually appname#appspot.gserviceaccount.com, so I gave that account full control:
$ gsutil chacl -u appname#appspot.gserviceaccount.com:FC gs://bucket
I'm not sure what changed since yesterday's cron run, but now it succeeds.