I have created a simple API using Cloud Endpoints, which makes some translations using Google Translate API, caches them in Datastore and returns the list of results to Client.
The problem is that each time I make a call, which uses Google Translate API I receive 502 Bad Gateway error.
I've been debugging my endpoint from cloud console and the response is created successfully, so the 502 must be thrown when sending the result back to client. Another strange thing here is that I don't see 502 on any logging tool in Cloud Console.
I was trying to change region from europe-west to us-central, but no luck so far.
Does anyone have any ideas, what may be wrong here?
My cofnig of GAE:
runtime: java7
env: standard
threadsafe: true
instance_class: B2
inbound_services:
- warmup
handlers:
- url: '(/.*/)'
application_readable: false
static_files: "__static__\\1index.html"
require_matching_file: true
upload: __NOT_USED__
- url: (/)
application_readable: false
static_files: "__static__\\1index.html"
require_matching_file: true
upload: __NOT_USED__
- url: '(/.*)'
application_readable: false
static_files: "__static__\\1"
require_matching_file: true
upload: __NOT_USED__
- url: /
script: unused
- url: '/.*/'
script: unused
- url: '/_ah/.*'
script: unused
basic_scaling:
idle_timeout: 900s
max_instances: 2
When changing configuration of GAE and removing basic-scaling I get from Google translate following error:
com.google.cloud.translate.TranslateException: com.google.apphosting.api.ApiProxy$CancelledException: The API call urlfetch.Fetch() was cancelled because the overall HTTP request deadline was reached.
at io.stringx.repository.DatastoreTranslationRepository.translate (DatastoreTranslationRepository.java:53)
It turned out that Google Cloud Endpoints have 60s deadline limit, so I had to make my requests in chunks:
https://cloud.google.com/appengine/articles/deadlineexceedederrors#UrlFetch_Delays
Related
I am deploying a web application to Google App Engine Standard edition using Java 8.
It is a documentation web site plus servlets that includes a huge amount of static web content (built with Jekyll).
I recently started getting this error trying to deploy the app:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files.
New versions are limited to 10000 files for this app.
- '#type': type.googleapis.com/google.rpc.BadRequest
fieldViolations:
- description: This deployment has too many files.
New versions are limited to 10000 files for this app.
field: version.deployment.files[...]
My src/main/webapp folder has 5016 files.
But after running:
mvn clean install appengine:stage
my target/appengine-staging folder has 10113 files.
I see that the files are in 2 locations, both
target/appengine-staging
and
target/appengine-staging/__static__
Why do they appear twice?
Is this normal?
Or am I doing something wrong?
How can I avoid this duplication?
Or do I have to limit my files to under 5000?
Thanks for any advice.
Note: The generated app.yaml looks like this:
runtime: java8
instance_class: F1
inbound_services:
- warmup
derived_file_type:
- java_precompiled
threadsafe: True
auto_id_policy: default
beta_settings:
'source_reference': 'https://...'
api_version: 'user_defined'
handlers:
- url: (/.*/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/.*)
static_files: __static__\1
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: /.*
script: unused
login: optional
secure: always
skip_files: app.yaml
I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto
I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.
the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?
I followed this steps to deploy my angular 8 application in Google Cloud
https://medium.com/#karthiksagar/how-to-deploy-angular-8-app-on-google-cloud-platform-gcp-cdd79e5cc5cf
My app.yaml is as follows
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: sofbox-angular/index.html
upload: sofbox-angular/index.html
- url: /
static_dir: sofbox-angular
I have placed the app.yaml file inside dist/ directory as mentioned.
I am able to access the main url- http://www.myridemate.com/
but when clicked on blog the link is not accessible - http://www.myridemate.com/blog
It gives below error
Error: Not Found The requested URL /blog was not found on this server.
Tried adding extra handler for /blog in app.yaml and it didn't work.
This same code is working in my local and also in hostgator - https://www.myridemate.alexvijayraj.com/blog
Not sure what I am missing here. Is there some extra configuration required in app.yaml or anything in GCP?
Error Message in App Engine Logs:
{
"time": "2020-09-17T03:58:28.682252Z",
"severity": "WARNING",
"logMessage": "Static file referenced by handler not found: sofbox-angular/blog"
}
The answer in this thread worked for me. Angular 6 routes not found on Google App Engine
Remember to remove the two handlers that are already on the .yaml file (-url: ...)
handlers:
- url: /(.*\.(js|css|svg|png)(|\.map))$
static_files: dist/\1
upload: dist/(.*)(|\.map)
- url: /.*
static_files: dist/index.html
upload: dist/.*
I'm getting the following error when trying to deploy my app to Google App Engine using gcloud app deploy.
error [INTERNAL]: An internal error occurred while processing task /appengine-flex-v1/insert_flex_deployment/flex_create_resources>2020-05-22T15:14:57.416Z3210.jc.5: Deployment Manager operation thesis-lock/operation-1590160497681-5a63e1799a578-3c148be2-663d8bc4 errors: [code: "RESOURCE_ERROR"
location: "/deployments/aef-flex-20200522t171231/resources/aef-flex-20200522t171231"
message: "{\"ResourceType\":\"compute.beta.regionAutoscaler\",\"ResourceErrorCode\":\"403\",\"ResourceErrorMessage\":{\"code\":403,\"errors\":[{\"domain\":\"usageLimits\",\"message\":\"Exceeded limit \'QUOTA_FOR_INSTANCES\' on resource \'aef-flex-20200522t171231\'. Limit: 8.0\",\"reason\":\"limitExceeded\"}],\"message\":\"Exceeded limit \'QUOTA_FOR_INSTANCES\' on resource \'aef-flex-20200522t171231\'. Limit: 8.0\",\"statusMessage\":\"Forbidden\",\"requestPath\":\"https://compute.googleapis.com/compute/beta/projects/.../regions/europe-west1/autoscalers\",\"httpMethod\":\"POST\"}}"
I've been able to deploy previously without any problems or errors in exactly the same way. I haven't changed my app.yaml. I have checked my quota in the Console yet I can find no quota that have been exceeded. The documentation does not provide any insights.
Any ideas as to what I can do?
I have found some similar questions on SO, but none of them seem to point to this issue specifically and none of the proposed solutions to those questions seem to work.
I have gone through the same. After digging it for 3 days. I found a solution which worked for me. I have opted to use standard environment instead of env: flex (flex environment). I have changed my config file i.e., app.yaml like the following.
a detailed form with each file type.
runtime: python27
threadsafe: true
api_version: 1
handlers:
- url: /(.+\.js)
static_files: app/\1
upload: app/(.+\.js)
- url: /(.+\.css)
static_files: app/\1
upload: app/(.+\.css)
- url: /(.+\.png)
static_files: app/\1
upload: app/(.+\.png)
- url: /(.+\.jpg)
static_files: app/\1
upload: app/(.+\.jpg)
- url: /(.+\.svg)
static_files: app/\1
upload: app/(.+\.svg)
- url: /favicon.ico
static_files: app/favicon.ico
upload: app/favicon.ico
- url: /(.+\.json)
static_files: app/\1
upload: app/(.+\.json)
- url: /(.+)
static_files: app/index.html
upload: app/index.html
- url: /
static_files: app/index.html
upload: app/index.html
If you are going to deploy nodejs application change runtime to nodejs(version) delete threaddSafe and api_version rest are same.
I have a lot of trouble finding how to map multiple domains to multiple services in the GAE. Here is the configuration :
One application is a Go API, deployed in GAE in the standard environment
The second application is an Angular application, also deployed in GAE in the standard environment but as another service.
Here are the app.yaml files :
Go application app.yaml
runtime: go
api_version: go1.9
handlers:
- url: /.*
script: _go_app
Angular application app.yaml
service: stage
runtime: python27
api_version: 1
threadsafe: true
skip_files:
- ^(?!dist) # Skip any files not in the dist folder
handlers:
# Routing for bundles to serve directly
- url: /((?:inline|main|polyfills|styles|vendor)\.[a-z0-9]+\.bundle\.js)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for a prod styles.bundle.css to serve directly
- url: /(styles\.[a-z0-9]+\.bundle\.css)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for typedoc, assets and favicon.ico to serve directly
- url: /((?:assets|docs)/.*|favicon\.ico)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Any other requests are routed to index.html for angular to handle so we don't need hash URLs
- url: /.*
secure: always
redirect_http_response_code: 301
static_files: dist/index.html
upload: dist/index\.html
http_headers:
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: DENY
I have a domain and want to bind the Go API to api.domain.com and the Angular app to domain.com.
By going to App Engine > Settings > Custom Domains I managed to add the domain for my API and it is perfectly working.
But now, I cannot find a way to map domain.com to my Angular application. Going to the same settings does not gives me an option to map a different service to my domain.
Thanks for the help and have a nice day !
To map subdomains you can use a dispatch.yaml file. An example:
dispatch:
- url: "example.com/*"
service: default
- url: "api.example.com/*"
service: otherservice
And then run $ gcloud app deploy dispatch.yaml (it can be in any directory).
Once you have example.com added under App Engine > Settings > Custom Domains for the default service, you can add the subdomain api.example.com for the other service. Later you need to add the new subdomain DNS records to you domain registrar as pointed out in the console configuration.
You want to first map your naked domain.com to your app.
Then choose to add another domain and you'll have the option to add the api (or any other) domain.com subdomain to a specific service.
Note that you have a conflicting/overlapping handler pattern in the 2 services: - url: /.*, this won't work as GAE won't know to which service to direct such requests to, they'll all end up sent to the same service. You need to partition your requests URL namespaces in a non-overlapping manner and you'll likely need a dispatch.yaml file as well. See Mapping subdomain to a service in Google App Engine project for details.