I have a Google Cloud Platform application that runs on several Google App Engine Standard instances. One app has a cron.xml with some cron jobs defined pointing to REST endpoints on that app.
Now I want to create a cron job in another app, so I created another cron.xml, all the REST endpoints and deployed. After deployment I realized the deployment erased the cron jobs defined by the first app. I read somewhere that you can only have one cron.xml defined PER PROJECT and not PER APP. Is this correct? I have been reading the documentation and a book on Google App Engine and could not find an answer.
cron.yaml/cron.xml apply per application.
I just got an answer from Google Cloud support. Apparently there can be only one CRON file PER PROJECT. Each upload of a cron file will override the previous upload.
A workaround for this would be add a TARGET tag for the URLs that are not in the same App as the CRON file. Adding target: will re-route the request to the appName app.
Related
I'm a little bit confused between gcp components, here is my use case :
daily, I need to fetch data from an external API (the API return json data), store it in GCS then load it in Bigquery, I already created the python script fetching the data and store it in GCS and i'm confused which component to use for deployment :
Cloud run : from the doc it is used for deploying services, so I think its a bad choose
Cloud function: I think it works, but it is used for even based processing (through single purpose function...)
composer :(I'll use composer to orchestrate tasks, such as preprocessing of files in GCS, load them to BQ, transfert them to an archive Bucket) through kubernetesPodOperator, create a task that trigger the script to get the data
compute engine: I don't think that its the best chose since there are better ones
app engine: also I don't think it a good idea since it is used to deploy and scale web app ...
(correcte me if i'm wrong in what I said, ) so my question is : what is the GCP component used for this kind of task
Cloud run : from the doc it is used for deploying services
app engine: also I don't think it a good idea since it is used to deploy and scale web app ...
I think you've misunderstood. Both Cloud run and Google App Engine (GAE) are serverless offerings from Google Cloud. You deploy your code to any of them and you can invoke their urls which in turn will cause your code to execute and do stuff like go fetch data from somewhere and save it somewhere.
Google App Engine has a shorter timeout than Cloud Run (can't remember if Cloud Run has time out). So, if your code will take a long time to run, you don't want to use Google App Engine (unless you make it a background task) and if you don't need a UI, then you don't need GAE.
For your specific scenario, you can deploy your code to Cloud Run and use Cloud Scheduler to schedule it to be invoked at specific times. We have that architecture running in a similar scenario (we have a task that runs once daily; it's deployed to Cloud Run; Google Scheduler invokes the endpoint, it runs and saves data to datastore linked to an App Engine App). We wrote a blog article on deploying to Cloud Run and another on securing your cloud run (based off our experience in the earlier described scenario)
GAE Timeout:
Every request to a Google App Engine (Standard) must complete within 1 - 10 minutes for automatic scaling and up to 24 hours for basic scaling (see documentation). For Google App Engine Flexible, the timeout is 60 minutes (documentation).
So I have this github repo that updates every 24 hours and I want to pull a file every 24 hours; from this repo to my Google Cloud app engine environment; is there any way I can pull that file every 24 hours using the App Engine cron?
The app is running on an app engine.
Yes, it's possible. But, you have not given us enough info to help. For instance, what language are you using?
If you have a url to a file, you can fetch that from your app. Then, you can save that file to your GCS storage bucket and access it there from your app.
You can use Cloud Build triggers based on GH commits to get the updated file and trigger the app to be deployed to GAE. If the file is updated at the exact same time everyday, you can use Cloud Scheduler to run like a cron job.
I would like to know how to deploy the application from bitbucket using pipelines to multiple Google Cloud projects.
Here is our current set up and it is working fine.
On Bitbucket, the application repo with development/UAT test/production branches, once the pull request approved and merged into development/production, it shall deploy to the GCP app engine through the pipelines.
The problem now, we want to isolate each client in GCP which mean each client will have its own GCP project, cloud SQL, App engines, storage bucket, etc...
I need some advice on how to change the deployment workflow in bitbucket and pipelines, so will work for the new set up.
For the branches setup on bitbucket, I'm thinking like below, but if I go for option2, then it seems too much if got more clients.
Option 1 (repo branches)
development/
UAT test/
validation/
production
Option 2 (repo branches)
development/
UAT test client1/
UAT test client2/
validation_client1/
validation_client2/
production_client1/
production_client2/
The first step, I know I have to create different app.yaml for each app engine service for each client, so it can deploy the app engine service to different CGP projects/bucket/SQL instance.
Also just found out the bitbucket-pipelines.yml only support 10 steps, if I create so many branches then it will over the limits for sure.
Does anyone have any suggestions about how should be set up?
Thanks,
You could create Cloud build triggers to specific bitbucket branches or repos (whatever your branching model is defined) and deploy the app engine implementation to the App engine service on the same project, and if you need to customize other steps, you could use custom steps as described here. Finally you can take a look at how to create a basic configuration file for Cloud Build if you are not very familiar with this product
I have built an app using Google Apps Script (GAS). It displays a form. When the user submits the form, the submitted data is written to a Google docs spreadsheet. I have deployed the app using the built in Deploy as web app option in the GAS script builder page
What I can't seem to find out is whether it is free to build and deploy web apps using Google Apps Script, or is it the case that one needs to pay?
I did come across a paid service called Google App Engine, but I am not sure if this is relevant to Google Apps Script.
Thanks.
Google Apps Script is a javascript cloud scripting language and it is free to use as long as you do not need higher quotas than defined here: https://developers.google.com/apps-script/guides/services/quotas
If you need higher quotas than listed there - I would suggest you take into consideration to build your own Google Appengine Application for your service.
However if you did not hear about that since now you should first do some examples listed here https://cloud.google.com/appengine/docs - and get familiar with the Platform as a Service Google offers. It is also free of any charge as long as your application applies to the free quotas.
Important fact: Every Google Apps Script has it's own Developers Console Project assigned to it - however it is not neccessary to configure anything on the Console for App Script to work properly. You can review your assigned Appengine Project by
using the Menu: Resources - Developers Console Project... and click on the link that looks similar to this -
https://console.developers.google.com/project/project-id-YOUR_PROJECT_ID
I am getting very confused with the whole cloud console api console thing.
I just created a new Cloud Console project in order to setup some GCE instances. It seems to have auto-created an App Engine app. That is fine, but when I go to enable billing, it tells me that it cannot enable billing for app engine apps and that I need to do that in App Engine Console.
Earlier this week, I tried to setup a project to include both an existing App Engine app and Compute Engine. I was asked to got through the request access process like before GCE was available.
I created a project in the Cloud Console by starting in the App Engine console and choosing Cloud Integration. I said there was an error creating the project, but it seems like it created it. However, when I go to that project in Cloud Console, there is no option to even add Compute Engine.
Two questions:
(1) Is there something outlining the differences between Cloud Console and API Console and when it is most appropriate to use one over the other?
(2) How do I get a project (in either one I suppose) that includes an existing App Engine app and Compute Engine resources that I want to create in this project?
Thanks for your help.
-- Jay
Cloud Console is essentially a planned replacement for the API console,
however it may not have all functionality yet
Try going to appengine admin console application settings, and at the bottom of
the page cloud integration. Click on create project. Once it succeeds go to the cloud
console and enable billing for Compute Engine