Known issue:
Installing google-cloud-sdk (linux package or from tarball) has a quirk where you cannot create projects from the command line before accepting the terms of service.
Steps to reproduce:
Download sdk, untar, move folder to home directory and add google sdk root directory to PATH using install.sh
Initialize and login with: $ google init
Create a project from the CL: $ gcloud projects create --set-as-default
This will spit out an error like:
ERROR: (gcloud.projects.create) Operation [cp.5641973328385684887] failed: 9: Callers must accept Terms of Service
I hazard a guess that accepting the terms of service have not been built into the command line initialization yet. Omitting such a fundamental step to an installation process should be illegal, with consequences ranging from death by a thousand key-stokes, to 'build an operating system in headfuck'... but that's just me...
We find the solution in the most unholiest of places: the google cloud control interface (cloud console).
Go to your cloud console
Create a project by selecting 'select a project' (top-left next to "Google Cloud Platform" and then 'create project' (top-right in the popup window).
This will prompt the terms of service agreement and you may carry on after agreeing to the terms of service.
I hope this helps whoever else stumbles upon this most infuriating of errors.
Live long and prosper
Bitshift
Let's not be so dramatic with "death by a thousand key-strokes". This is a security measure that should be implemented. Security is not always convenient but can save your checking/credit account a lot of grief.
Imagine this theoretical scenario. You provide me with a service account that has the roles to create a project. I create a new project. This project is created under your Google Billing Account. I know what I am doing with Google IAM so I remove you from the new project and make myself the Project Owner. Now you have no access to the new project but your credit card is paying the bills for my project. I think you would then be screaming "death by a million key-strokes".
There are two types of projects:
Independent projects not part of an organization.
Projects that are part of an organization.
If you are part of a Google Cloud Organization, you can easily create projects up to your quota limit (default is 5). No prompting, accepting TOS, etc. Using the CLI to create a new project is effortless.
If you are not part of an Google Cloud Organization then you are basically creating a new account, you need to set up account billing, accept terms of service, etc. This means that you should not use the CLI to create a new project as the CLI does not prompt you for the items that a new project requires. Why, the CLI should be using a service account. The service account is not the IAM member that owns the account. This forces you to log into the Google Cloud Console using your User Credentials to create the new project.
For anyone getting this message when trying to create a dialogflow agent:
Go to https://console.cloud.google.com, login and accept the displayed terms and conditions.
Afterwards it worked for me...
Related
I want to access datastore (and storage) data of an AppEngine project via google-cloud-datastore and google-cloud-storage with an Python program on my own server.
This works with my AppEngine staging server, by creating a service account and giving it owner access (to the project).
Doing the same thing with the production AppEngine instance fails with
google.api_core.exceptions.PermissionDenied: 403 Missing or insufficient permissions.
Part of the problem might be, that I might be using the wrong project to create the service account with. There are more than one project with the same name in my cloud console. How do I identify the correct one?
How do I get more details about the problem?
First, note that the Datastore and the Cloud Storage are 2 different products with 2 different accessing methods.
The Datastore is closely tied to the GAE project - each project has its own datastore. The external access procedure in general is captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?.
When switching the project (staging to production in your case) there are 2 things to keep in mind:
as you observed, you need to change the project you're accessing.
you also need to change the credentials you load and use for access to match the project you select, as each project has it own service account key configured in the above-mentioned procedure
For the google-cloud-datastore library both of these are simultaneously configured via the datastore.Client() call parameters (emphasis mine):
class google.cloud.datastore.client.Client(project=None,
namespace=None, credentials=None, _http=None, _use_grpc=None)
project (str) – (Optional) The project to pass to proxied API methods.
credentials (Credentials) – (Optional) The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed),
falls back to the default inferred from the environment.
The Cloud Storage is completely independent from GAE, the GAE project/credentials you use (if any) have no bearing on bucket/object access restrictions whatsoever. There's nothing you need to do from the google-cloud-storage library perspective when switching from one GAE project to another
To eliminate the confusion created by multiple projects having the same name just go to the IAM & admin Settings page, select the respective projects from the drop-down list on the top blue bar and rename them using meaningful names (click in the Project name box to edit the name, then click SAVE). Then re-check if you're using the right keys for the desired project.
I am using a Custom Domain for a Google Application project. I have multiple projects and I use the same domain for each, although only one mapping is active at any time. Historically this has been as simple as Verifying the domain on the latest project and then Adding the domain. The domain has then automatically switched to the new project.
I have not used this approach for some months and when I tried it recently I got the messages
www.xxxxx.com is already mapped to a project.
xxxxx.com is already mapped to a project.
Research on StackOverflow suggests the use of the following command
gcloud beta app domain-mappings create xxxxx.com
This does look the right thing to do, unfortunately the response to the command is:
ERROR: (gcloud.beta.app.domain-mappings.create) App [aaaaaa] is the subject of a conflict: Domain 'xxxxx.com' is already mapped to another application. You must delete the existing domain mapping before you can re-map the domain, or you may specify 'DomainOverrideStrategy.OVERRIDE' on the request to force overwrite the existing mapping. Domain 'xxxxx.com' is currently mapped to an application on which you do not have permissions.
I do not want to use the delete approach since there will be a gap in service before the re-map. I would like to use the OVERRIDE option but I cannot work out how to add it to the gcloud command and I cannot locate any documentation.
Update 6Nov17.
In the absence of an answer I have used the delete approach and it worked as expected. The re-map was possible immediately after the mapping was deleted for the current app. Unfortunately for some users access to the web page was not possible, or error messages were returned, for a while. After about 10-15 minutes normal service was resumed. For my web site a gap of 10-15 minutes is manageable. This will not be true for many sites and I anticipate that Google Cloud will tidy up this procedure before it exits Beta.
If for some reason it is not possible to access the current app to delete the mapping then I guess deleting the app's subdomain information at the domain registrar will have the same effect, although it may be difficult to predict when the delete of the associated mapping will happen.
On the plus site the new automatic SSL provision worked flawlessly.
I have a website that is currently running under GAE... unfortunately, I, nor anyone on the team, does not have access the local environment that it was created from.... Is it possible to create a local environment or at least get a copy of the application files and database from an existing GAE installation?
What you need is the application source code, not the "local environment".
Ideally this source code would be on a version control system (ie GIT,SVN), Google cloud platform provides free GIT repositories for your projects so you might try looking there first. There's also a tool for both Java and python that allow you to download the source of a deployed version, provided you are authenticated as either the dev who uploaded it or a project owner. EDIT: as stated by Dan Cornilescu this feature can be disabled.
As for the database info there's plenty of tools available to "export" your GAE datastore info, just consider for your project that it might be easier to do the queries manually than actually implementing this tools.
Thanks for help... But unfortunately, this code is not in GIT. Furthermore,
being new to Google hosting, I wasn't clear on my setup... My web instance is actually running within Compute Engine not Application Engine. Be that as it may, with some additional search, I was first able to find out how to browse my filesystem by accessing the VM Instances menu option under the Compute Engine section of the Google Cloud Platform interface. On the VM Instances page, it will show your instance and an option to the left side of the instance to connect with a drop down box that will allow you to open a browser window that shows the instance's file system. In addition to this, I found this link https://www.youtube.com/watch?v=9ssfE6ODpak that shows how to configure Filezila FTP client to access your server instance - very helpful. From there, I was able to download all of my site files from the var/www directory. Now, onto extracting my data... Thanks again!
I believe there is an undocumented Google API available to create and manage Google Cloud Console (and App Engine) projects on behalf of third party users.
Does anyone know how to use it?
I think older versions of the Google Eclipse Plugin obtained an OAuth2 token in the (undocumented) scope https://www.googleapis.com/auth/appengine.admin, and this allowed it to generate a Cloud Console project on your behalf. The latest version doesn't seem to do this. App Engine's own appcfg.py also uses this scope, but doesn't seem to do much more than deploy the code - I'm looking to change core settings for the project, such as Name, Redirect URLs, and Web Origins.
Any information would be appreciated.
I maintain a WordPress plugin providing secure Google Apps Login for end users, and currently have to give detailed instructions to admins for creating a new Cloud Console project manually, and entering settings such as Redirect URL. Ideally, I would create a simple on-line service to do all of this for them.
Thank you!
It is possible to programmatically create a new Developer Console project on behalf of a Google Account (yes, you read that right). You do so in a very roundabout way:
Request the https://www.googleapis.com/auth/drive.scripts scope from the user (standard OAuth 2.0 flow).
Use the Drive API's drive.insert method to create a new file with a mimetype of application/vnd.google-apps.script.
Somehow try to get the project ID, maybe by uploading some Apps Script code? This is the part that I was never able to figure out.
A little known fact is that every Google Apps Script project has a hidden Developer Console project associated with it. This project is not shown in the list of projects, but it does exist. It is created automatically when the user starts a new Apps Script project, and the drive.insert method is enough to cause this to happen.
How do you get to the hidden project? Well, the only way I know of is to open the Apps Script project from the Drive website, open the "Resources > Advanced Google Services" dialog, and click the link to the Developer Console. You'll find the project ID in the URL.
Aside from not being shown in your list of projects and not being able to use App Engine, this is a normal Developer Console project. You can add additional OAuth client credentials, service accounts, Compute Engine instances, etc. And of course once you have a project ID, all of the various management APIs will work: creating new virtual machines, making use of a service account's impersonation ability, etc.
I recall seeing an option on the google app engine admin page that would allow me to duplicate/copy a deployment to a different appid. Was I imagining this? I do have a copy of all the files needed for deployment but I do not currently have access to a computer with admin privileges in order to install the GAE SDK.
Not relevant to this question, but FYI I need to do this simply because there is a major bug that's preventing me from logging in to a particular part of the admin interface and google tech support has failed to resolve the issue, so I'm just trying a workaround.
Copy old settings to new app via the admin console admin>application settings>Duplicate Application Settings. (you'll choose new appid here too)
Then upload a version of your code to the new appId (so you'll need gae sdk or use online sdk devtable).
Then go to datastore admin in the old app, select all the entities and choose "copy to another app" You'll need datastore admin to be enabled on the old app (and new app I think).