How to Add Environment Variables to Google App Engine - google-app-engine

I have deployed my Django Project to Google App Engine and I need to add environment variables.
The docs say to add them to app.yaml but that seems like bad practice because app.yaml should be in your git repository.
Is there any way to add environment variables to App Engine the same way you can add them in Cloud Run > Services > Variables & Secrets ?

Google Secret Manager is available, since this spring:
Enable Secret Manager API
Add the Secret Manager Secret Accessor role to the App Engine SA
Create secretes from the GCP Web UI or programmatically(code examples are from official documentation):
def create_secret(project_id, secret_id):
"""
Create a new secret with the given name. A secret is a logical wrapper
around a collection of secret versions. Secret versions hold the actual
secret material.
"""
# Import the Secret Manager client library.
from google.cloud import secretmanager
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the parent project.
parent = client.project_path(project_id)
# Create the secret.
response = client.create_secret(parent, secret_id, {
'replication': {
'automatic': {},
},
})
# Print the new secret name.
print('Created secret: {}'.format(response.name))
Consume the secrets from the app instead of the environment variables:
def access_secret_version(project_id, secret_id, version_id):
"""
Access the payload for the given secret version if one exists. The version
can be a version number as a string (e.g. "5") or an alias (e.g. "latest").
"""
# Import the Secret Manager client library.
from google.cloud import secretmanager
# Create the Secret Manager client.
client = secretmanager.SecretManagerServiceClient()
# Build the resource name of the secret version.
name = client.secret_version_path(project_id, secret_id, version_id)
# Access the secret version.
response = client.access_secret_version(name)
# Print the secret payload.
#
# WARNING: Do not print the secret in a production environment - this
# snippet is showing how to access the secret material.
payload = response.payload.data.decode('UTF-8')
print('Plaintext: {}'.format(payload))

If you are using a continuous deployment process you could rewrite (or created) the app.yaml to include variables relevant to each deployment target within the CD build system.
We rewrite several files as part of our deployment process to App engine using Bitbucket pipelines. Variables can be defined at a workspace level (across multiple repositories), within a repository, and also for each deployment target defined. These variables can be secured so they are not readable.
build: &build
- step:
name: Update configuration for deployment
script:
- find . -type f -name "*.yaml" -exec sed -i "s/\[secret-key-placeholder\]/$SECRET_KEY/g" {} +
Refer to https://support.atlassian.com/bitbucket-cloud/docs/variables-in-pipelines/#Deployment-variables

Related

deploy Huggingface model to SageMaker endpoint

I need to deploy a large language model (t0pp) on a SageMaker endpoint. I modified the official example to look like this:
from sagemaker.huggingface import HuggingFaceModel
import sagemaker
role = sagemaker.get_execution_role()
hub = {
'HF_MODEL_ID':'bigscience/T0', # model_id from hf.co/models
'HF_TASK':'text2text-generation' # NLP task you want to use for predictions
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
env=hub,
role=role, # iam role with permissions to create an Endpoint
transformers_version="4.6", # transformers version used
pytorch_version="1.7", # pytorch version used
py_version="py36", # python version of the DLC
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.m5.xlarge"
)
but I'm getting this error
UnexpectedStatusException: Error hosting endpoint huggingface-pytorch-inference-2022-09-21-15-44-30-116: Failed. Reason: The primary container for production variant AllTraffic did not pass the ping health check.
Any idea what is going wrong here?

Create new instances of a Google App Engine Service

I have a Google App Engine Java 11 service using the Standard Environment.
I have deployed it specifying in the corresponding app.yaml file manual scaling, setting the number of instances to 1.
Is there a way that I can increase the number of instances for this service without having to upload again all the files in the service?
So I have one instance. Now I want 2 instances. How do I do this?
Have not found a way in either the console or in the gcloud utilities to do this.
Also, just calling gcloud app deploy with the modified app.yaml file creates a broken version of the service.
app.yaml:
service: headergrabber
runtime: java11
instance_class: B8
manual_scaling:
instances: 1
Use REST API to patch the number of instances of your manual scaling app.
Here's the HTTP request:
PATCH https://appengine.googleapis.com/v1/{name=apps/*/services/*/versions/*}
You will have to pass manualScaling.instances field to update to the number of instance you prefer.
Here's an example with curl using a token that should only be used for local testing. I tested it on my side and it works:
curl -X PATCH -H "Content-Type: application/json" \
-d "{ 'manualScaling': { 'instances': 2 } }" \
-H "Authorization: Bearer $(gcloud auth application-default print-access-token)" \
https://appengine.googleapis.com/v1/apps/PROJECT-ID/services/SERVICE/versions/VERSION?updateMask=manualScaling.instances
Where:
PROJECT_ID = Project ID
SERVICE = Service Name
VERSION = Version Name
You need to login to your account and set the project if you're using Cloud SDK or you can run the command on Cloud Shell.
An alternative is to use a client library so you can write applications that can update your App Engine instance.
Take note that doing this would require App Engine Admin API enabled on your project. This API provides programmatic access to several of the App Engine administrative operations that are found in the Google Cloud Console.

Consume SQS tasks from App Engine

I'm attempting to integrate with a third party that is posting messages on an Amazon SQS queue. I need my GAE backend to receive these messages.
Essentially, I want the following script to launch and always be running
import boto3
sqs_client = boto3.client('sqs',
aws_access_key_id=KEY,
aws_secret_access_key=SECRET,
region_name=REGION)
while True:
sqs_client.receive_message(QueueUrl=QUEUE_URL, WaitTimeSeconds=60)
for message in msgs_response.get('Messages', []):
deferred.defer(process_and_delete_message, message)
My main appengine web app is on Automatic Scaling (with the 60-second &10-minute task timeouts), but I'm thinking of setting up a micro-service set to either Manual Scaling or Basic Scaling because:
Requests can run indefinitely. A manually-scaled instance can choose to handle /_ah/start and execute a program or script for many hours without returning an HTTP response code. Task queue tasks can run up to 24 hours.
https://cloud.google.com/appengine/docs/standard/python/an-overview-of-app-engine
Apparently both Manual & Basic Scaling also allow "Background Threads", but I am having a hard-time finding documentation for it and I'm thinking this may be a relic from the days before they deprecated Backends in favor of Modules (although I did find this https://cloud.google.com/appengine/docs/standard/python/refdocs/modules/google/appengine/api/background_thread/background_thread#BackgroundThread).
Is Manual or Basic Scaling suited for this? If so, what should I use to listen on sqs_client.receive_message()? One thing I'm concerned about is this task/background thread dieing and not relaunching itself.
This maybe a possible solution:
Try to use a Google Compute Engine micro instance to run that script continuously and send a REST call to your app engine app. Easy Python Example For Compute Engine
OR:
I have used modules that run instance type B2/B1 for long running jobs; and I have never had any trouble; but those jobs do start and stop. I use the basic scaling: with max_instances set to 1. The jobs I have run take around 6 hours to complete.
I ended up creating a manual scaling app engine standard micro-service for this. This micro-service has handeler for /_ah/start never returns and runs indefinitely (many days at a time) and when it does get stopped, then app engine restarts it immediately.
Requests can run indefinitely. A manually-scaled instance can choose
to handle /_ah/start and execute a program or script for many hours
without returning an HTTP response code. Task queue tasks can run up
to 24 hours.
https://cloud.google.com/appengine/docs/standard/python/an-overview-of-app-engine
My /_ah/start handler listens to the SQS queue, and creates Push Queue tasks that my default service is set up to listen for.
I was looking into the Compute Engine route as well as the App Engine Flex route (which is essentially Compute Engine managed by app engine), but there were other complexities like not getting access to ndb and the taskqueue sdk and I didn't have time to dive into that.
Below are all of the files for this micro-service, not included is my lib folder that contains the source code for boto3 & some other libraries I needed.
I hope this helpful for someone.
gaesqs.yaml:
application: my-project-id
module: gaesqs
version: dev
runtime: python27
api_version: 1
threadsafe: true
manual_scaling:
instances: 1
env_variables:
theme: 'default'
GAE_USE_SOCKETS_HTTPLIB : 'true'
builtins:
- appstats: on #/_ah/stats/
- remote_api: on #/_ah/remote_api/
- deferred: on
handlers:
- url: /.*
script: gaesqs_main.app
libraries:
- name: jinja2
version: "2.6"
- name: webapp2
version: "2.5.2"
- name: markupsafe
version: "0.15"
- name: ssl
version: "2.7.11"
- name: pycrypto
version: "2.6"
- name: lxml
version: latest
gaesqs_main.py:
#!/usr/bin/env python
import json
import logging
import appengine_config
try:
# This is needed to make local development work with SSL.
# See http://stackoverflow.com/a/24066819/500584
# and https://code.google.com/p/googleappengine/issues/detail?id=9246 for more information.
from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
import sys
# this is socket.py copied from a standard python install
from lib import stdlib_socket
socket = sys.modules['socket'] = stdlib_socket
except ImportError:
pass
import boto3
import os
import webapp2
from webapp2_extras.routes import RedirectRoute
from google.appengine.api import taskqueue
app = webapp2.WSGIApplication(debug=os.environ['SERVER_SOFTWARE'].startswith('Dev'))#, config=webapp2_config)
KEY = "<MY-KEY>"
SECRET = "<MY-SECRET>"
REGION = "<MY-REGION>"
QUEUE_URL = "<MY-QUEUE_URL>"
def process_message(message_body):
queue = taskqueue.Queue('default')
task = taskqueue.Task(
url='/task/sqs-process/',
countdown=0,
target='default',
params={'message': message_body})
queue.add(task)
class Start(webapp2.RequestHandler):
def get(self):
logging.info("Start")
for loggers_to_suppress in ['boto3', 'botocore', 'nose', 's3transfer']:
logger = logging.getLogger(loggers_to_suppress)
if logger:
logger.setLevel(logging.WARNING)
logging.info("boto3 loggers suppressed")
sqs_client = boto3.client('sqs',
aws_access_key_id=KEY,
aws_secret_access_key=SECRET,
region_name=REGION)
while True:
msgs_response = sqs_client.receive_message(QueueUrl=QUEUE_URL, WaitTimeSeconds=20)
logging.info("msgs_response: %s" % msgs_response)
for message in msgs_response.get('Messages', []):
logging.info("message: %s" % message)
process_message(message['Body'])
sqs_client.delete_message(QueueUrl=QUEUE_URL, ReceiptHandle=message['ReceiptHandle'])
_routes = [
RedirectRoute('/_ah/start', Start, name='start'),
]
for r in _routes:
app.router.add(r)
appengine_config.py:
import os
from google.appengine.ext import vendor
from google.appengine.ext.appstats import recording
appstats_CALC_RPC_COSTS = True
# Add any libraries installed in the "lib" folder.
# Use pip with the -t lib flag to install libraries in this directory:
# $ pip install -t lib gcloud
# https://cloud.google.com/appengine/docs/python/tools/libraries27
try:
vendor.add('lib')
except:
print "Unable to add 'lib'"
def webapp_add_wsgi_middleware(app):
app = recording.appstats_wsgi_middleware(app)
return app
if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
print "gaesqs development"
import imp
import os.path
import inspect
from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
# Use the system socket.
real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
imp.load_source('socket', psocket)
os.environ['HTTP_HOST'] = "my-project-id.appspot.com"
else:
print "gaesqs prod"
# Doing this on dev_appserver/localhost seems to cause outbound https requests to fail
from lib import requests
from lib.requests_toolbelt.adapters import appengine as requests_toolbelt_appengine
# Use the App Engine Requests adapter. This makes sure that Requests uses
# URLFetch.
requests_toolbelt_appengine.monkeypatch()

Testing Google Cloud PubSub push endpoints locally

Trying to figure out the best way to test PubSub push endpoints locally. We tried with ngrok.io, but you must own the domain in order to whitelist (the tool for doing so is also broken… resulting in an infinite redirect loop). We also tried emulating PubSub locally. I am able to publish and pull, but I cannot get the push subscriptions working. We are using a local Flask webserver like so:
#app.route('/_ah/push-handlers/events', methods=['POST'])
def handle_message():
print request.json
return jsonify({'ok': 1}), 200
The following produces no result:
client = pubsub.Client()
topic = client('events')
topic.create()
subscription = topic.subscription('test_push', push_endpoint='http://localhost:5000/_ah/push-handlers/events')
subscription.create()
topic.publish('{"test": 123}')
It does yell at us when we attempt to create a subscription to an HTTP endpoint (whereas live PubSub will if you do not use HTTPS). Perhaps this is by design? Pull works just fine… Any ideas on how to best develop PubSub push endpoints locally?
Following the latest PubSub library documentation at the time of writing, the following example creates a subscription with a push configuration.
Requirements
I have tested with the following requirements :
Google Cloud SDK 285.0.1 (for PubSub local emulator)
Python 3.8.1
Python packages (requirements.txt) :
flask==1.1.1
google-cloud-pubsub==1.3.1
Run PubSub emulator locally
export PUBSUB_PROJECT_ID=fake-project
gcloud beta emulators pubsub start --project=$PUBSUB_PROJECT_ID
By default, PubSub emulator starts on port 8085.
Project argument can be anything and does not matter.
Flask server
Considering the following server.py :
from flask import Flask, jsonify, request
app = Flask(__name__)
#app.route('/_ah/push-handlers/events', methods=['POST'])
def handle_message():
print(request.json)
return jsonify({'ok': 1}), 200
if __name__ == "__main__":
app.run(port=5000)
Run the server (starts on port 5000) :
python server.py
PubSub example
Considering the following pubsub.py :
import sys
from google.cloud import pubsub_v1
if __name__ == "__main__":
project_id = sys.argv[1]
# 1. create topic (events)
publisher_client = pubsub_v1.PublisherClient()
topic_path = publisher_client.topic_path(project_id, "events")
publisher_client.create_topic(topic_path)
# 2. create subscription (test_push with push_config)
subscriber_client = pubsub_v1.SubscriberClient()
subscription_path = subscriber_client.subscription_path(
project_id, "test_push"
)
subscriber_client.create_subscription(
subscription_path,
topic_path,
push_config={
'push_endpoint': 'http://localhost:5000/_ah/push-handlers/events'
}
)
# 3. publish a test message
publisher_client.publish(
topic_path,
data='{"test": 123}'.encode("utf-8")
)
Finally, run this script :
PUBSUB_EMULATOR_HOST=localhost:8085 \
PUBSUB_PROJECT_ID=fake-project \
python pubsub.py $PUBSUB_PROJECT_ID
Results
Then, you can see the results in Flask server's log :
{'subscription': 'projects/fake-project/subscriptions/test_push', 'message': {'data': 'eyJ0ZXN0IjogMTIzfQ==', 'messageId': '1', 'attributes': {}}}
127.0.0.1 - - [22/Mar/2020 12:11:00] "POST /_ah/push-handlers/events HTTP/1.1" 200 -
Note that you can retrieve the message sent, encoded here in base64 (message.data) :
$ echo "eyJ0ZXN0IjogMTIzfQ==" | base64 -d
{"test": 123}
Of course, you can also do the decoding in Python.
This could be a known bug (fix forthcoming) in the emulator where push endpoints created along with the subscription don't work. The bug only affects the initial push config; modifying the push config for an existing subscription should work. Can you try that?
I failed to get PubSub emulator to work on my local env (fails with various java exceptions). I didn't even get to try various features like push with auth, etc. So I end up using ngrok to expose my local dev server and used the public https URL from ngrok in PubSub subscription.
I had no issue with whitelisting and redirects like described in the Q.
So might be helpful for anyone else.

Passwordless continuous deployment from CircleCI to AppEngine

The CircleCI appengine documentation suggests using a password to do deployment. How can I use the oauth2 flow instead of using passwords? I don't want to share my Google password.
Do I generate a ~/.appcfg_oauth2_tokens_java file, from token data stored as environment variables in CircleCI? Is there a simpler way?
I solved the issue this way:
deployment:
appengine:
branch: master
commands:
- erb .appcfg_oauth2_tokens_java.json > ~/.appcfg_oauth2_tokens_java # requires ENV in circle ci
- mvn -DskipTests=true appengine:update # tests have already been run
.appcfg_oauth2_tokens_java.json:
{
"credentials": {
"ubuntu": {
"access_token": "<%= ENV["GOOGLE_ACCESS_TOKEN"] %>",
"expiration_time_millis": 1431552739090,
"refresh_token": "<%= ENV["GOOGLE_REFRESH_TOKEN"] %>"
}
}
}
Then in CircleCI, configure the ENV variables for the two tokens. I got the tokens by locally running mvn appengine:update and going through the oAuth2 dance. Note: You may have to remove your existing ~/.appcfg_oauth2_tokens_java file first.
Reading the AppEngine SDK docs, it sounds like that would be a good approach. There is not a built-in way to do that on CircleCI.
If you don't want to use any user-related credential, you can leverage service accounts, like mentioned in this blog post:
Continuous Deployment with Google App Engine and CircleCI
I solved it this way on the latest GAE SDK 1.9.34 for Java.
Assuming you have a Base64 encoded ENV Variable with your JSON key for a service account you've created on the GCloud project:
dependencies:
pre:
- echo $GOOGLE_CLIENT_SECRET | base64 --decode > ${HOME}/client-secret.json
And then in the deployment section:
- $HOME/appengine-java-sdk-$APP_ENGINE_VERSION/bin/appcfg.sh -A $GCLOUD_PROJECT -M $GCLOUD_MODULE -V $BUILD_VERSION --service_account_json_key_file=$HOME/client-secret.json update $WAR_FOLDER
The --service_account_json_key_file doesn't seem to appear as an option when you use appcfg.sh help but it is there, and does work.

Resources