Suppose I uploaded another version (say, "version: 2" in file app.yaml ) of my Google App Engine application. Version 1 is still the default and version 2 is for testing. How do I run it then?
Once you upload a version on Appengine, you can switch between them easily.
Say that your app name is myapp, currently running version 1. You also have uploaded a version called 2-testing. Your default app (with version 1) can be reached by accessing myapp.appspot.com
If you wanted to access your versions explicitly, you joust need to access <version_name>-dot-myapp.appspot.com. Following the example it would be:
1-dot-myapp.appspot.com or 2-testing-dot-myapp.appspot.com
The -dot- is equivalent to <version>.<appname> but allows you to correctly serve a secure application with SSL
You can mark any version you want as default (serving myapp.appspot.com) using the admin console
edit: this is the official documentation page talking about domains and subdomains in Appengine
Under versions in your admin console you can find the live uri of a version, if you select the version.
And you can use traffic splitting, where you can use your own client ip or a cookie to test a version.
Docs: https://developers.google.com/appengine/docs/adminconsole/trafficsplitting
Related
I've recently done a UI update for my website, the backend is all identical. I want to access the latest deploy (with the no-promote flag) from 'beta.my-website.com' but still access the 'normal' site from 'my-website.com'. Ideally this would dynamically serve the latest version, but if not could I set it statically to the URL with project ID? I have bought my domain from 123reg. Is there a way to direct to the latest version regardless of it's traffic serving status?
TL;DR Can you set up a 'beta' site at beta.mywebsite.com while still being able to access the live version on www.mywebsite.com on google cloud?
Any help would be greatly appreciated.
Thanks!
I don't think that can be done like that.
You could deploy to a different appengine project by putting a different project id and then have beta.mywebsite.com route to that project instead. The nice thing about this option is everything else would be isolated (i.e. no chance of your beta changes causing bad data in your production database, or tasks in your queues)
Another option would be to deploy to a different service within your project and use your dispatch.yaml to route beta.mywebsite.com to it https://cloud.google.com/appengine/docs/standard/python3/reference/dispatch-yaml
A third option would be to put beta as your version-id, and have your DNS redirect beta.mywebsite.com to beta-dot-your_project_id.appspot.com. However, when the user goes to beta.mywebsite.com they would ultimately see beta-dot-your_project_id.appspot.com in the URL bar. Also, when you're ready to go live, you would want to redeploy and promote that new version id and not promote the beta version-id.
I have been using the Google App Engine Launcher's "Deploy" button. I realize that the Launcher is going away at the end of July 2020, so I'm trying to learn to use gcloud. My web site has a custom url, www.xxx.org, but "gcloud app deploy" uploads to xxx.appspot.com instead of to www.xxx.org, presumably because "target url" is set to xxx.appspot.com. Can someone please tell me what gcloud command I need to use to change "target url" to www.xxx.org, or some other scheme that would work? I've wandered through lots of gcloud and GAE documentation and web searches without being able to figure out what I need to do.
There is a very good and complete guide about mapping a custom domain to your application in the official documentation. As #DanCornilescu said,it needs to be configured, but it is not necessary to be done before deployment.
Basically, you need to follow the steps from the guide:
1) Verify that you are the owner of your domain through Webmaster
Central
2)Ensure that your domain has been verified, otherwise you will not be able to proceed with the following steps. Note that only verified domains will be displayed.
3)If you need to delegate the ownership of your domain to other users or service accounts, you can add permission through the Webmaster Central page
4)After you verify ownership of your domain, you can map that domain to your App Engine app
5)Add the resource records that you receive to the DNS configuration of your domain registrar
6)Test for success by browsing to your app via its new domain URL, for example www.example.com
In case you need to do more complex mappings for different urls and subdomains you can always write down the routing rules in the dispatch.yaml file and they will be mapped successfully.
Also, to add a bit more of information about the dipatch.yaml file, it will handle the routing in the App as opposed with the app.yaml which will take care of the App's settings. Both .yamls can exist in the same application and can be deployed at the same time by using the following gcloud command:
gcloud app deploy app.yaml dispatch.yaml
The deployment is always done to .appspot.com, the custom domain is automatically updated if it is already configured. If it is not then it needs to be configured (it can be done after the deployment as well).
I have a problem deleting app engine version.
In my Google app engine console, under the section "app engine"->"Version" I have different version of the same project. I delete one of these, i.e. test.myapp.appspot.com. and I can't see this version anymore in the list of the versions.
But if I try to make a call to an API of this version I have a response like this is still alive.
How can I completely delete this version? Like I never deploy it...
App Engine automatically redirects to the default version if the version you're navigating to isn't known (or has been deleted).
E.g.
If you're default version is:
https://1-0-0-dot-yourapp.appspot.com/
And you navigate to:
https://foo-dot-yourapp.appspot.com/
It'll automatically load the default version of the app (and not render a 500 error), despite the url showing https://foo-dot-yourapp.appspot.com/
This is specifically a question relating to server to server authorisation between a python Google AppEngine app and Google's BigQuery, but could be relevant for other cloud services.
tldr; Is it possible to get the App Engine local development server to authenticate with the remote BigQuery service? Better yet is there a local BigQuery?
I understand that AppAssertionCredentials does not currently work on the local development server, though that in itself is very frustrating.
The alternative method which works for standard python code, outside of the local development server sandbox, detailed here does not work for the local development server because even with PyCrypto enabled the sandbox does not allow some posix modules e.g. 'pwd'.
I have got AppAssertionCredentials working on the remote server and the SignedJwtAssertionCredentials method working in native python locally, so the service accounts are set up properly.
The imports fail within oauth2client/crypt.py within the try/except blocks - after commenting them out the sandbox whitelist exceptions are easily seen.
I've fiddled around with adding 'pwd' to the whitelist, then another problem crops up, so I scurried back out of that rabbit hole.
I've tried including PyCrypto directly into the project with similar results.
I've also tried with OpenSSL with similar results.
I have looked for a local appengine specific PyCrypto to no avail, have I missed one? I should say this is on Mac OSX - perhaps I should fire up a linux box and give that a go?
A recent release of Google App Engine SDK added support for the AppAssertionCredentials method on the development server. To use this method locally, add the following arguments to dev_appserver.py:
$ dev_appserver.py --help
...
Application Identity:
--appidentity_email_address APPIDENTITY_EMAIL_ADDRESS
email address associated with a service account that
has a downloadable key. May be None for no local
application identity. (default: None)
--appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH
path to private key file associated with service
account (.pem format). Must be set if
appidentity_email_address is set. (default: None)
To use these:
In Google Developer Console, select a project then navigate to "API & auth" -> "Credentials" -> "Create new client ID".
Select "Service account" and follow the prompts to download the private key in PKCS12 (.p12) format. Take note of the email address for the service account.
Make sure you add that service account email address to the "Permissions" tab for any project that contains data it needs to access, by default it is added to the project team in which it was created.
Convert the PKCS12 format to PKCS1 format using the following command:
$ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem
Start dev_appserver.py as:
$ dev_appserver.py --appidentity_email_address xxxx#developer.gserviceaccount.com --appidentity_private_key_path /path/to/secret.pem ...
Use appidentity module and AppAssertionCredentials in the same manner locally as you normally would in production.
Please ensure that /path/to/secret.pem is outside of your application source directory so that it is not accidentally deployed as part of your application.
So searching deeper for PyCrypto and local appengine sandbox lead me onto this thread and response specifically...
https://code.google.com/p/googleappengine/issues/detail?id=1627#c22
This is fixed in 1.7.4. However, you must use easy_install -Z
(--always-unzip) to install PyCrypto. The default zipfile option in
OSX 10.8 is incompatible with the sandbox emulation in the
dev_appserver.
The solution turns out to be very straight forward...
I used:
sudo easy_install pycrypto
and it should have been:
sudo easy_install -Z pycrypto
as per the thread above. Using PIP will work as well:
pip install pycrypto
or a manual download and install of pycrypto will also work. I tested all three.
If you have installed pycrypto with easy_install and without -Z flag then you may want to install pip just so you can easily uninstall pycrypto...
easy_install pip
for the record I built and installed libgmp, as pil and the manual install showed this warning...
warning: GMP or MPIR library not found; Not building
Crypto.PublicKey._fastmath.
Although this gave me fastmath, it was not essential to solve the problem as the Crypto libs gracefully fail to slowmath.
Another point that tripped me up for a bit was I removed pycrypto from app.yaml whilst testing to see if OpenSSL might give me all I need.
So dont forget to add...
- name: pycrypto
version: latest
into app.yaml under the libraries: section.
With this missing the native _counter library was not imported hence Counter failed etc.
Also for the record any talk of having to move Crypto into the app folders themselves or out of the default Mac OS X location of /Library/Python/2.7/site-packages/Crypto was only valid in earlier versions of the dev server.
Similarly there is now no need to edit any _WHITE_LIST_C_MODULES lists (which is in sandbox.py in appengine 1.8 onwards, which also includes the regex which allows Crypto.Util._counter etc)
The other bit of the puzzle in case you get here before discovering the key issue is that the key file you download from the console is PKCS12 and is downloaded as hex text, so I converted that to binary and then converted that to a PEM so I could include it in the source code.
I struggled with this one for a day or two. And I was finally able to get localhost working with server to server authentication, a service account and a .p12 cert.
If it's at all helpful to anyone, here's a simple gist: https://gist.github.com/dandelauro/7836962
I agree with the first post - the localhost/production impedance is a real pain in the a**. AppAssertionCredentials is the right way to go on production and I don't want to have two different code paths between production and localhost. So the development environments need to be adjusted to be able to perform the required authentication without affecting the main code path.
E.g., perhaps a developer could log in with their own Google account using appcfg.py and then that auth would be cached for a period such that AppAssertionCredentials would work out. The developer's Google account could be granted permissions on the appropriate environments (dev and test for us, e.g.)
re: "local BigQuery" - we have some initial stuff in place that uses SQLLite to simulate BigQuery interactions for unit tests and other offline/local testing, but of course, it's not a great simulation. I agree that all the Cloud Platform products need to spend as much time thinking about the development-time experience as App Engine has.
Is it possible to get the App Engine local development server to authenticate with the remote BigQuery service?
I think it's impossible to use AppAssertionCredentials as authentication method between BigQuery service and your local App Engine server currently.
Alternatively, I'm using OAuth2 authentication which is associated with specific user(this user must be registered in your project at google api console) to access BigQuery from local App Engine server.
For getting user OAuth2 authentication, I use oauth2client.client module in the app code.
I hope this will be helpful to your problem.
Updated:
This is what I'm doing for getting the user OAuth2 authorization.
Edited:
Added missing import statement.
Thanks mattes!
import os
import webapp2
import httplib2
from oauth2client.client import OAuth2Credentials
from oauth2client.appengine import StorageByKeyName, CredentialsModel, OAuth2DecoratorFromClientSecrets
from google.appengine.api import users
oauth2_decorator = OAuth2DecoratorFromClientSecrets(
os.path.join(os.path.dirname(__file__), 'client_secrets.json'),
scope='https://www.googleapis.com/auth/bigquery')
oauth2_decorator._kwargs = {'approval_prompt': 'force'}
class TestPage(webapp2.RequestHandler):
#oauth2_decorator.oauth_required
def get(self):
user_id = users.get_current_user().user_id()
credentials = StorageByKeyName(CredentialsModel, user_id, 'credentials').locked_get()
http = credentials.authorize(httplib2.Http()) # now you can use this http object to access BigQuery service
application = webapp2.WSGIApplication([
('/', TestPage),
(oauth2_decorator.callback_path, oauth2_decorator.callback_handler()),
], debug=True)
The Google App Engine documentation says that the URL to access a specific version of an app deployed is .latest..appspot.com. The GAE admin webpage however links to ..appspot.com in the Versions tab and omits the 'latest'.
Any idea what that 'latest' is for? Is it actually not needed? Is omitting it not taking you the latest code deployed for that specific version?
References:
https://developers.google.com/appengine/docs/java/config/appconfig
The *.latest.app.appspot.com convention was originally the only way to access a specific version of your app. This was changed to simply accessing them directly as subdomains a while ago; latest was kept as a special case so as not to break old URLs.