The CircleCI appengine documentation suggests using a password to do deployment. How can I use the oauth2 flow instead of using passwords? I don't want to share my Google password.
Do I generate a ~/.appcfg_oauth2_tokens_java file, from token data stored as environment variables in CircleCI? Is there a simpler way?
I solved the issue this way:
deployment:
appengine:
branch: master
commands:
- erb .appcfg_oauth2_tokens_java.json > ~/.appcfg_oauth2_tokens_java # requires ENV in circle ci
- mvn -DskipTests=true appengine:update # tests have already been run
.appcfg_oauth2_tokens_java.json:
{
"credentials": {
"ubuntu": {
"access_token": "<%= ENV["GOOGLE_ACCESS_TOKEN"] %>",
"expiration_time_millis": 1431552739090,
"refresh_token": "<%= ENV["GOOGLE_REFRESH_TOKEN"] %>"
}
}
}
Then in CircleCI, configure the ENV variables for the two tokens. I got the tokens by locally running mvn appengine:update and going through the oAuth2 dance. Note: You may have to remove your existing ~/.appcfg_oauth2_tokens_java file first.
Reading the AppEngine SDK docs, it sounds like that would be a good approach. There is not a built-in way to do that on CircleCI.
If you don't want to use any user-related credential, you can leverage service accounts, like mentioned in this blog post:
Continuous Deployment with Google App Engine and CircleCI
I solved it this way on the latest GAE SDK 1.9.34 for Java.
Assuming you have a Base64 encoded ENV Variable with your JSON key for a service account you've created on the GCloud project:
dependencies:
pre:
- echo $GOOGLE_CLIENT_SECRET | base64 --decode > ${HOME}/client-secret.json
And then in the deployment section:
- $HOME/appengine-java-sdk-$APP_ENGINE_VERSION/bin/appcfg.sh -A $GCLOUD_PROJECT -M $GCLOUD_MODULE -V $BUILD_VERSION --service_account_json_key_file=$HOME/client-secret.json update $WAR_FOLDER
The --service_account_json_key_file doesn't seem to appear as an option when you use appcfg.sh help but it is there, and does work.
Related
I am trying to modularize our code, and running into difficulties installing private packages on app engine.
We have a repository on GCP in the same project, so I would think this is not too difficult.
My requirement.txt looks like:
--extra-index-url=https://us-central1-python.pkg.dev/myproject/my-python-repo/simple/
Flask==2.0.1
my-new-package
I can pip install the package locally, and it uses the keyrings.google-artifactregistry-auth package to authenticate.
The deployment fails with:
File "/opt/python3.8/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 218, in ask_input
return input(message)
EOFError: EOF when reading a line
This is clearly pip asking for a username, where i
Is google's own keyring package not available in its own environment? This Suggests so. Of course adding it to requirements.txt has no effect, as it is too late.
How can I install packages properly?
A possible solution is to:
--extra-index-url=_json_key_base64:KEY#us-central1-python.pkg.dev/myproject/my-python-repo/simple/
Where KEY is the result of base64 -w 0 < credentials.json
Of course this then requires some rewriting of the requirements.txt file in your deployment pipeline to ensure service account credentials are secured in repository variables.
I'm actually working on the Continuous Delivery pipeline for the UI project made in ReactJs with the use of the Azure Static Web App.
I want to create and deploy the static web app to Azure based on the git repo located in Azure DevOps.
The reason behind this is I see a huge opportunity to create a Pull Request Environment pipeline for the system I work on every day with the usage of static web apps which seem to be a super cheap and fast solution! Then the pipeline would allow testing the Pull Request changes in isolation before releasing to DEV, QA,... Prod environments.
Anyway, straight to the point.
The official Microsoft documentation provides only an example of how to do this for GitHub repo but I cannot find any info on how this can be achieved when using Azure DevOps git repo:
az staticwebapp create \
-n my-first-static-web-app \
-g <RESOURCE_GROUP_NAME> \
-s https://github.com/<YOUR_GITHUB_ACCOUNT_NAME>/my-first-static-web-app \
-l <LOCATION> \
-b main \
--app-artifact-location "build" \
--token <YOUR_GITHUB_PERSONAL_ACCESS_TOKEN>
I thought that the way the az staticwebapp create works in the provided example should be analogical with the Azure DevOps.
I thought that equivalent of YOUR_GITHUB_PERSONAL_ACCESS_TOKEN in Azure DevOps would be an access token that can be generated:
When I try to run the following code:
az staticwebapp create -l westus2 -n appNameTest1 -g TestPrEnvResourceGroup -s "https://dev.azure.com/myOrganisationName/myProjectName/_git/myRepoName -b "main" --token "generatedTokenValuefwfsdgsgsd"
I'm getting the following exception;
Command group 'staticwebapp' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus Operation returned an invalid status 'Bad Request'
Also, I don't think it can matter but the TestPrEnvResourceGroup is created under the UK West location.
It doesn't tell me much, like if the token or some other parameter is wrong...
Any ideas?
Cheers
Have you tried this way?
You need to generate a deployment token from the Azure Static Web App and have it in the YAML
I tried the same for Angular app and it worked pretty well
azure_static_web_apps_api_token: $(deployment_token)
UPDATE : As per the Github issue , currently this is not supported.
We currently don’t support automatically creating Azure DevOps
pipelines. This is the supported path for using Azure DevOps:
https://learn.microsoft.com/en-us/azure/static-web-apps/publish-devops
You can vote for the feature here
I have a django project I want to put into maintenance mode before I update (migrate) the database.
So, my github workflow
deploys my project with a variable MAINTENANCE_MODE set to true. This new deploy I understand will reboot any running instances, ensuring all instances only show my 'Site down for maintenance' 503.html page and won't be interacting with the database.
I launch a django VM in github actions, run my migrate, run collectstatic.
I set MAINTENANCE_MODE to false, I deploy a second time. This will re-enable production server with new code that now accesses a migrated database.
My question is, I am trying to use a single app.yaml file for both deploys. To pass the MAINTENANCE_MODE variable from github actions workflow to the app.yaml file, how can I do this?
I know you can import secrets like so:
runtime: python38
instance_class: F2
env_variables:
DB_URL: $ {{ secrets.DB_URL }}
But I don't know how to modify a secret in the workflow. Perhaps its not a secret but some other type of variable one can set in the workflow and access in the app.yaml?
So it appears that Google App Engine's yaml files do not support dynamic environment variable substitution. Static substitution (like using github's secrets) works, because github compiles the file with the github environment variable before the workflow runs, but there's no clear way to modify a file with a variable that is going to change during a workflow.
A method however that does work is to compile a new GAE yaml file during the workflow. Here's what I came up with in the end...
- name: Put in Maintenance mode
run: |
MAINTENANCE_MODE=1 envsubst < app_eng_staging.yml.template > app.yaml
cat app.yaml
gcloud app deploy --project staging-project --quiet
- name: Collectstatic and migrate
env:
RUNNING_ENVIRONMENT: 'Staging_Server'
DJANGO_DEBUG: 'False'
run: |
pipenv run python manage.py collectstatic --noinput
pipenv run python manage.py migrate
- name: Turn off Maintenance and Deploy
run: |
MAINTENANCE_MODE=0 envsubst < app_eng_staging.yml.template > app.yaml
gcloud app deploy --project staging-project --quiet
The trick is to use the linux envsubst command. We start with a app_eng_staging.yml.template file, which looks like so:
runtime: python38
instance_class: F2
env_variables:
RUNNING_ENVIRONMENT: 'Staging_Server'
StagingServerDB: $ {{ secrets.STAGINGSERVER_DB }}
MAINTENANCE_MODE: ${MAINTENANCE_MODE}
FRONTEND_URL: $ {{ secrets.STAGING_FRONTEND_URL }}
envsubst then populates ${MAINTENANCE_MODE} with the value 1 and the result is saved to a new file app.yaml.
After we finish working with out database migration, we can use envsubst to create a new app.yaml with MAINTENANCE_MODE set to zero (off), and re-deploy.
Neat huh?
There is a feature in GitHub Action for this called "GAE environment variable compiler"
Please read here
I am trying to access my Google Cloud Bucket from development environment but when I write import statement I get error
from google.cloud import storage
Command I use to run server with bucket flag
dev_appserver.py app.yaml --default_gcs_bucket_name ABC-test-bucket
Error I get
File "C:\Users\ABC\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\setuptools-0.6c11\pkg_resources.py", line 565, in resolve
raise DistributionNotFound(req) # XXX put more info here
DistributionNotFound: google-cloud-storage
I think I completed all the steps like creating bucket, downloading client library using pip i.e.
pip install GoogleAppEngineCloudStorageClient -t <your_app_directory/lib>
I am new to GAE projects(Using Webapp2 Python framework for server) so I will really appreciate any pointers and help
The library you're using is not the correct one. You need the one below, as per the documentation:
pip install --upgrade google-cloud-storage
I need to download the application files that are deployed using command prompt into google appengine server.
if anybody know about this plz tell me!
thanks in advance!
SDK 1.4.0 has this sort of feature (Python only):
The developer who uploaded an app version can download that version's
code using the appcfg.py
download_app command. This feature can
be disabled on a per application
basis in the admin console, under the
'Permissions' tab. Once disabled,
code download for the application
CANNOT be re-enabled.
Note that although the 'download_app' command is only available in the Python SDK, it can be used to download Java applications from App Engine. If you have a Java app that you want to download, just do:
Install Python (version 2.x only), if necessary.
Install App Engine 1.4.0 (or higher), Python version.
Run the python download_app command as described in the documentation, like appcfg.py download_app -A <your app name> -V <version to download> <path to download to>.
Your Java app will download. Hooray!
It's not possible to download your code from App Engine, unless you included code to support it when you uploaded your app. You can download static files by simply fetching them, of course.
To achieve this, you need to use a source control system, such as SVN or Git, and store/retrieve your code from there - App Engine isn't intended to act as source control.
You can use appcfg.sh / appcfg.cmd in your GAE SDK, as pointed out in this official guide.
But notice that as of 2012-05-07, at least for the Windows version appcfg.cmd, the command's syntax is
appcfg -A yourappid -V yourversionnumber download_app c:\path\to\download\directory
Also notice that when you run this command and you get asked for your password and you happen to have activated Google's 2-step verification, you must not provide your Google password, but instead must provide an application-specific password (which you can generate on this side)
As explained in this post: http://www.labnol.org/software/download-appengine-files/19348/
App Engine 1.5.0 introduced the feature to download your source code.
The command is:
appcfg.py download_app –A MyAppName -V 1 c:\AppEngine\SourceCode
You can use appcfg.py with download_data option, see docs.
You can download an application's source code by running appcfg.py with the download_app action in the Python SDK command-line tool:
appcfg.py download_app -A <your_app_id> -V <your_app_version> <output-dir>