I've been following the steps in the Google tutorial, Using Pub/Sub with Cloud Run tutorial.
All ok until allowing Google's service account to create auth tokens (Integrating with Pub/Sub 2.b.)
Here I run the command:
gcloud projects add-iam-policy-binding PROJECT_ID \
--member=serviceAccount:service-PROJECT_NUMBER#gcp-sa-pubsub.iam.gserviceaccount.com \
--role=roles/iam.serviceAccountTokenCreator
PROJECT_ID and PROJECT_NUMBER are replaced by their actual values of course.
However I get the error:
ERROR: (gcloud.projects.add-iam-policy-binding) INVALID_ARGUMENT: Service account service-PROJECT_NUMBER#gcp-sa-pubsub.iam.gserviceaccount.com does not exist.
(I've replaced my actual project number by PROJECT_NUMBER here for privacy, believe me I've double-checked that it's the correct ID and number.)
There's a note in this guide about IAM changes taking several minutes to propagate but I'm seeing the same error hours after the previous command completed successfully.
I've also tried adding this through the Cloud Console IAM UI but I get a consistent error:
using IAM UI.
I'm already using Pub/Sub in this project and it's still working with a Pull subscription.
Update
I've finished setting this up in another project following the same instructions and that's working fine, so I'm certain this isn't a misunderstanding of the tutorial, there's something wrong with this project (or rather, the Google-managed service account for this project).
I was able to force the account to be created with the command (docs):
gcloud beta services identity create --project PROJECT_ID --service pubsub
After this the add-iam-policy-binding command completed without error.
Related
I have this app that's been running on Google App Engine for a good couple of years. Today I needed to modify its app.yaml file, but when I ran "gcloud app deploy" I got this error:
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [7] Failed to create cloud build: service account "redacted#cloudbuild.gserviceaccount.com" has insufficient permission to execute the build on project "redacted"
It appears that the default service account no longer has the necessary permissions. In fact, it may have been deleted completely, because I don't see it listed under gcloud projects get-iam-policy xxxx. What I see instead is this:
serviceAccount:service-redacted#gcp-sa-cloudbuild.iam.gserviceaccount.com
with a correct-looking role cloudbuild.serviceAgent.
I tried googling any changes related to GAE service accounts but didn't find any useful references. I followed the solution to this question, but it didn't help.
The question is, how do I change the permissions of the service account mentioned in the original error message, or, how do I tell GAE to use another (correct) service account when deploying my app?
According to [this][1] documentation
...To deploy new versions, a member must have the Service Account User (roles/iam.serviceAccountUser) role on the App Engine default service account, and the Cloud Build Editor (roles/cloudbuild.builds.editor) and Cloud Storage Object Admin (roles/storage.objectAdmin) roles on the project.....
Using the IAM page you can add the email address of the person deploying (the one you have authenticated via gcloud) and add the above roles to that email
[1]: https://cloud.google.com/iam/docs/understanding-roles?hl=en#app-engine-roles
I am trying to use Jenkins for CI/CD.
I've developed a Python flask application. I am deploying this application into Google App Engine. So far I am using gcloud app deploy app.yaml command to deploy application to Google App Engine.
Code for this application is present in Google Cloud Source Repository.
Since the authentication to git(Google Cloud Source Repository) requires Google OAuth, I've installed Google OAuth Credentials Plugin
Now I am facing two issues
When I use "Google Service Account from metadata" Credential Kind, I am not seeing the credentials listed in "Source Code Management".
when I use "Google Service Account from private key", I am able to see credentials. But when I run my jenkins job I am getting below error
FATAL: Could not call
com.google.jenkins.plugins.source.GoogleRobotUsernamePassword.writeObject()
: Failed to serialize
com.google.jenkins.plugins.source.GoogleRobotUsernamePasswordModule$ForRemote#credentials
for class
com.google.jenkins.plugins.source.GoogleRobotUsernamePasswordModule$ForRemote
---- Debugging information ---- message : Could not call com.google.jenkins.plugins.source.GoogleRobotUsernamePassword.writeObject()
cause-exception : java.lang.RuntimeException cause-message :
Failed to serialize
com.google.jenkins.plugins.source.GoogleRobotUsernamePasswordModule$ForRemote#credentials
for class
com.google.jenkins.plugins.source.GoogleRobotUsernamePasswordModule$ForRemote
------------------------------- java.lang.UnsupportedOperationException: Refusing to marshal
org.joda.time.DateTime for security reasons; see
https://jenkins.io/redirect/class-filter/ at
hudson.util.XStream2$BlacklistedTypesConverter.marshal(XStream2.java:543)
at
com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)
at
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)
at
com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)
Question: How can I authenticate Google Cloud Source repository in Jenkins? What are the steps|plugins required to use Google Cloud Source repository in Jenkins?
Solution
You probably need to add a missing scope to the Compute Engine VM Instance running Jenkins that would give it access to Cloud Source Repository. You can follow the documentation or this steps, which ever you find convenient.
Go to Cloud Deployment Manager
Click on the name of the Jenkins deployment in question
Click on the instance name in the left info pane and it will redirect you to VM instance details
Stop the instance
Press Edit and set the right access scope for Cloud Source Repository
After your start the VM instance, try adding your git repository again and once you select the credentials, either "Google Service Account from metadata" or from service account, everything should work properly.
Elaboration
I stumbled upon the "Invalid authentication credentials." issue while trying to add a Cloud Source Repository after deploying Jenkins from Launcher.
In my case the reason why it happened was that during the process of deployment the Cloud API access scope for Cloud Source Repositories on the Compute Engine VM instance was set to Disabled, which prevented any interaction from that instance even if a service account had all the necessary roles/permissions.
Here are the scopes that are reconfigured by Launcher:
scopes:
- 'https://www.googleapis.com/auth/cloud.useraccounts.readonly'
- 'https://www.googleapis.com/auth/devstorage.read_only'
- 'https://www.googleapis.com/auth/logging.write'
- 'https://www.googleapis.com/auth/monitoring.write'
{% if enableComputeApi %}
- 'https://www.googleapis.com/auth/compute'
{% endif %}
- 'https://www.googleapis.com/auth/cloudruntimeconfig'
Adding the following scope to the VM instance running Jenkins was enough to fix the error:
https://www.googleapis.com/auth/source.read_only
Extra:
List of scopes for Google APIs.
I updated the plugin to 0.6 version.
In 0.6 version, I have the same problem in Jenkins "Free-style" project.
Bit I can success to checkout in Jenkins "pipeline" project.
I just copy the "checkout syntax" in others project and replace 2 parts of the syntax to the right:
credentialsid -> replace to your right credentials id that you created Google Service Account from private key in jenkins.
url ->replace to your google source repository url
I have mistakenly deleted the default "app engine service account" for my project - {id}#appspot.gserviceaccount.com
Is there a way to recover it without resetting/recreating my project on Google Cloud Platform?
Unfortunately at this time, there is no way to recover the default App Engine service account. The solution is as you say creating a new project and redeploying your code there.
Should you wish to see this deletion prevented or default service account recreation made possible in the future, I strongly urge you to file a feature request on the App Engine public issue tracker while providing this example as a business case.
UPDATE: As filed by the OP, a feature request now exists on the App Engine public issue tracker as Issue 13085. Please star this issue to receive updates on its progress.
UPDATE 2: As part of the App Engine Admin API, one can now use the apps.repair API to attempt to address issues of default Cloud Storage buckets and App Engine service accounts. I would recommend trying this API before creating a new project and redeploying. I'm leaving this as an addendum and not the primary solution as it's not guaranteed to solve the issue.
UPDATE 3 It is now possible to undelete service accounts. As per the Undeleting a service account documentation, a service account may be restored if:
The service account was deleted less than 30 days ago
and
There is no existing service account with the same name as the deleted service account.
You can now recover the deleted service accounts from https://cloud.google.com/iam/reference/rest/v1/projects.serviceAccounts/undelete
you have to get the UniqueID of the service account from https://console.cloud.google.com/home/activity
Recover
You can undelete service accounts. You will need the service account's unique ID. If you don't have it, you can find it on Google Cloud Logging.
You can find Logging service here on the side menu:
Then you will need to filter by date and type service account to find the exact moment the service was deleted.
Then you can either
Option 1: Use Google Cloud Command Line
You can run the command line by installing it on your computer (https://cloud.google.com/sdk/docs/install). Or you can run it online using the Active Shell offered by Google Cloud Platform.
The command you want to run is the following.
gcloud beta iam service-accounts undelete 12345678901234567890
Option 2: Use Google Cloud API
Using curl, call the API with the following command.
You will need to change API_KEY, PROJECT_ID and SERVICE_ACCOUNT_UID for real values.
curl -X POST \
-H "Authorization: Bearer API_KEY \
-H "Content-Type: application/json; charset=utf-8" \
-d "" \
"https://iam.googleapis.com/v1/projects/PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT_UID:undelete"
You can get the API_KEY from Google Cloud Command Line:
gcloud auth application-default print-access-token
Again you can either have gcloud installed on your local machine or you can use it online with the Active Shell.
If you don't know the default service account UniqueID, there is another solution.
Simply disable the App Engine Admin API and enable the same after some time. This will create a new default service account. Please note that this will delete all the associated resources.
(To enable/disable an API: Menu -> API & Services -> Dashboard -> Enable APIs and Services -> Search for that API -> Enable/Disable)
Your Account is deleted less than 30 days you can recover it via cloud console
Go to Cloud Console Open Terminal :
Write :
gcloud beta iam service-accounts undelete 100214681451516381413
100214681451516381413 this is the Deleted Account UID .
If You Don't know the Deleted Account ID then GO IAM and Search With Your Deleted Email Address you will find-out the UID there .
Thanks .
I searched the web for a while, but can't seem to find the right answer.
I have created a VM Instance on Google Compute Engine and I am running Jenkins on it. This instance checks a code repository I have and when a change occurs, I want to run the following command
gcloud --project=test preview app deploy -q app.yaml --version=dev
When I want to trigger a build, or it triggers it by himself, I get the following error:
Beginning deployment...
ERROR: Error Response: [403] Request had insufficient authentication scopes.
ERROR: (gcloud.preview.app.deploy) Could not retrieve the default Google Cloud Storage bucket for [test].
Please try again or use the [bucket] argument.
The VM instance does have acces to the storage as you can see in the following image, so I don't understand why this error pops up.
Go to the App Engine developer console https://appengine.google.com/ and in
Application Settings > Cloud integration > and turn ON the Google Cloud Storage
so this will set up the default bucket for you. This something that the new Google Developer Console will do automatically.If the project is created on the old console you have to do it manually.
Using Eclipse, I am experiencing an error when trying to deploy a rather basic web app with JAX-RS and JAXB. It runs okay locally, but when trying it on the remote servers I get the message shown below...
'Deploying to Google' has encountered a problem / This application does not exist
Below shows my appengine-web.xml
The XML file illustrates that I am using the same name in the xml as what's specified in the project properties...
The output window show...
------------ Deploying frontend ------------
Preparing to deploy:
Created staging directory at: '/var/folders/n8/6by626014jbfc0dwmxnb0ly00000gn/T/appcfg2754901216637807129.tmp'
Scanning for jsp files.
Scanning files on local disk.
Initiating update.
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=hillingarincident&version=0&
404 Not Found
This application does not exist (app_id=u'hillingarincident').
Debugging information may be found in /private/var/folders/n8/6by626014jbfc0dwmxnb0ly00000gn/T/appengine-deploy447984481661870877.log
The referenced debug logs show...
Unable to update:
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=hillingarincident&version=0&
404 Not Found
This application does not exist (app_id=u'hillingarincident').
at com.google.appengine.tools.admin.AbstractServerConnection.send1(AbstractServerConnection.java:293)
at com.google.appengine.tools.admin.AbstractServerConnection.send(AbstractServerConnection.java:253)
at com.google.appengine.tools.admin.AbstractServerConnection.post(AbstractServerConnection.java:232)
at com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.java:644)
at com.google.appengine.tools.admin.AppVersionUpload.beginTransaction(AppVersionUpload.java:449)
at com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload.java:124)
at com.google.appengine.tools.admin.AppAdminImpl.doUpdate(AppAdminImpl.java:371)
at com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:53)
at com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngineBridgeImpl.java:433)
at com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(DeployProjectJob.java:148)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Any answers will be appreciated. At one point my browser was not logged in to the target google account, so I swapped to the correct one a little later, Google does render the application name as expected.
Okay, this was simple in the end! Eclipse performs an auto-login to the Google account, unfortunately I created the Eclipse project whilst being logged in to one Google account and then tried to specify the application name afterwards.
You'll see in the bottom-right (or bottom-left in some versions) a Google icon with the name of the user that you are logged in as. If that's not the account where your application is defined, then simply logout of that account, then login as the correct Google account.
Now there's no error :-)
I know this question is super old but I had this issue all day and finally I found a solution. Maybe it will help someone out in the future.
After you create a project in Google Cloud Platform, you must go to google cloud shell in your project and run the command
gcloud beta app create
After you run this command, you will get prompted to choose a region. Then go back to eclipse and try deploying it. It worked for me.
There are not just 1 way can cause this problem. For me, I have this problem when I create the project using Maven. But I don't have the same issue if I directly create the project from the Google plugin.
There might be another issue, when you register with Google App Engine, you receive email indicating your activation. If you have not received the email yet, this problem could occur too.
Another issue could be to use the gmail account for the Google App Engine to avoid any such errors.