Google App Engine High Costs - google-app-engine

I recently asked a question: Deploying to google app engine failed
I got my app deployed, however, we are incurring a large sum for this project. (wiki.js using third-party DB mLab).
I'm wondering if it has to do with the config I put in app.yaml, namely, the memory expansion and resources
This is what the google support person said: to add:
resources:
cpu: 2
memory_gb: 4.0
disk_size_gb: 20
health_check:
enable_health_check: False
My app.yaml (from google console) is:
runtime: nodejs
api_version: '1.0'
env: flexible
threadsafe: true
automatic_scaling:
min_num_instances: 2
max_num_instances: 20
cpu_utilization:
target_utilization: 0.5
resources:
cpu: 2
memory_gb: 4
disk_size_gb: 20
health_check:
enable_health_check: false

Google Cloud Support person here again!
Find a detailed description of costs going to Google Cloud Platform Console home --> Billing --> View detailed charges or following this guide. Update the post with the expensive fields without sharing private data, please, so I can understand what is going on.
Find general information about quotas here.
P.S.: I found a couple of solutions and I suggested you to TRY them since they weren't working for me.

Related

Google App Engine: unpredictable cost and discrepancy between app engine dashboard vs billing export

I have been exploring the App Engine settings for a small data science web application for 2 weeks. Since it is a personal project that bills my own wallet, I tried a few different parameters in app.yaml to reduce the "frontend instances" cost. Several changes in, I got unexpected ~10x cost surge!!! It was painful!!! In order to not waste it, I decided to learn something here to understand the behaviour :)... Don't worry, I had temporarily shut down my app ;)
Version 1 app.yaml:
service: my-app
runtime: python37
instance_class: F4
env: standard
automatic_scaling:
min_idle_instances: 1
max_idle_instances: 1
default_expiration: "1m"
inbound_services:
- warmup
entrypoint: gunicorn -b 0.0.0.0:8080 main:server
Version 1, billing result (usage.amount_in_pricing_units exported from billing account): ~100hr/day, the same as Front end Instance Hours shown from App Engine billing status.
This is understandable, because I had a F4 instance constantly runing idle that would translate into 24*4=96 frontend instance hours. Adding the instance usage from actual requests (from me only), ~100hr/day seems reasonable.
Version 2, where I intended to lower the instance class and number of instances and also made longer the default_expiration and hoping it would help the app to start quicker and some other stuff that I thought wouldn't affect much....
service: my-app
runtime: python37
instance_class: F2
env: standard
automatic_scaling:
min_instances: 1
max_instances: 1
target_cpu_utilization: 0.85
max_concurrent_requests: 80
max_pending_latency: 6s
default_expiration: "3h"
inbound_services:
- warmup
entrypoint: gunicorn -b 0.0.0.0:8080 main:server
Version 2, billing result (usage.amount_in_pricing_units exported from billing account): ~800hr+/day, ouch!!! In contrast, the Front end Instance Hours from App Engine dashboard billing status is less than 60hr/day as expected. This is where I got lost:
Why the usage from billing is so much larger than the App Engine Dashboard where do those usage come from?
Where to find and track indicators of those unaccounted usage in App Engine Dashboard etc?
2020-01-16 Solution for issue #1.
While I was waiting for Google Billing Support to come back to me, I found this:
Pricing of Google App Engine Flexible env, a $500 lesson
Namely, the past deployed versions of the app also eating frontend instance hours, which needed real world confirmation. (To my surprise, this has nothing to do with app.yaml file!!) So I deleted all the past versions of the app and let it run for two days while observing instance hours and billing records with the following app.yaml file.
service: my-app
runtime: python37
instance_class: F2
env: standard
automatic_scaling:
min_instances: 1
max_instances: 2
max_idle_instances: 1
target_cpu_utilization: 0.85
max_concurrent_requests: 80
max_pending_latency: 6s
default_expiration: "1m"
inbound_services:
- warmup
entrypoint: gunicorn -b 0.0.0.0:8080 main:server
This should have always one F2 instance running and goes up to maximum 2 instances. This time both app engine and exported billing usage hours agreed on 50 hours frontend instance hours. Yes!!! The daily cost is cut down to 1/16.
This solves the cost question #1, but #2 remains to be answered. It is very problematic that app engine dashboard is not showing all the billed usage of frontend instances. Yesterday I heard from Google Billing Support Team, the answer is not helpful (mainly talking about instance numbers in app.yaml, which doesn't help), they seem oblivious about this issue, I will have to let them know.
2020-01-31 Followup on issue #2.
Google Billing Support Team responded swiftly, acknowledged the discrepency between App Engine Dashboard vs Billing Export and agreed to ajust the billing for me. Effectively, the bills during the spiky days were refunded as a result. Kudos to them!

Google App engine Deoployment receives ERROR Only when specifiying CPUs above 1 in app.yaml

I have a Flask app that deploys fine in the Google App Engine Flexible environment but some new updates have made it relatively resource intensive (Was receiving a [CRITICAL] Worker Timeout message.) In attempting to fix this issue I wanted to increase the number of CPUs for my app.
app.yaml:
env: flex
entrypoint: gunicorn -t 600 --timeout 600 -b :$PORT main:server
runtime: python
threadsafe: false
runtime_config:
python_version: 2
automatic_scaling:
min_num_instances: 3
max_num_instances: 40
cool_down_period_sec: 260
cpu_utilization:
target_utilization: .5
resources:
cpu: 3
After some time I receive:
"Updating service [default] (this may take several minutes)...failed.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment."
Is there some sort of permission issue preventing me from increasing the CPUs? Or is my app.ymal invalid?
You can not set the number of cores(CPU) to odd numbers except 1. It should be even.

Google cloud appengine 1.9.54 Release

We are deploying ASP.NET Core application on Appengine Flex and in Instances Summary on Dashboard page appears strange 1.9.54 appengine release as well as Flex release. What that might be?
Our app.yaml:
env: flex
runtime: aspnetcore
resources:
cpu: 8
memory_gb: 14.4
automatic_scaling:
min_num_instances: 8
max_num_instances: 20
cool_down_period_sec: 180
cpu_utilization:
target_utilization: 0.5
Your app's dashboard summary indicates your app has both:
standard env GAE instances (1.9.54 being the version of the GAE sandbox/infra running those instances), possibly from older service version(s) not yet deleted
flexible env GAE instances
You can play with the Service and/or Version selection boxes above the summary to identify which service/version those instances correspond to.

Task Queue Issue - Endpoint v2, Google App Engine

We are facing issues with taskqueues after recently updating the Endpoints API to version 2 in Google App Engine - Python. Following are the issues faced with respect to taskqueus,
Taskqueue doesn't get added to the queue at all , Just gets ignored and never executed.
Taskqueue get terminated with error - "Process terminated because the backend was stopped."
The most critical error is the first one where the task is just ignored and not added to the queue itself.
Details on the codebase and logs are attached along.
It would be great if someone can help us out here.
app.yaml (Server Settings)
#version: 1
runtime: python27
api_version: 1
threadsafe: true
instance_class: F4
automatic_scaling:
min_idle_instances: 1
max_idle_instances: 4 # default value
min_pending_latency: 500ms # default value
max_pending_latency: 900ms
max_concurrent_requests: 50
queue.yaml
- name: allocateStore
rate: 500/s
bucket_size: 500
max_concurrent_requests: 1000
retry_parameters:
task_retry_limit: 0
Adding task to queue:
taskqueue.add(queue_name='allocateStore', url='/tasksStore/allocateStore')
Thanks,
Navin Lr

When deploying my app in flexible environment getting error beta setting machine_type cannot be set in an App Engine Flexible Environment

When trying to deploy my app engine using flexible environment then i am getting error.
ERROR: (gcloud.preview.app.deploy) INVALID_ARGUMENT:
The beta setting machine_type cannot be set in an App Engine Flexible Environment deployment.
My app.yaml is given below
runtime: nodejs
#vm: true
env: flex
# [END runtime]
network:
instance_tag: app-tag
name: network-tag
instance_class: F1
automatic_scaling:
min_num_instances: 1
max_num_instances: 2
cool_down_period_sec: 60
beta_settings:
machine_type: f1-micro
handlers:
- url: /.*
script: IGNORED
secure: always
# Temporary setting to keep gcloud from uploading node_modules
skip_files:
- ^node_modules$
Also can anyone please tell me what is the difference between vm: true and env: flex because both set app engine environment to flexible ??
When changing from vm: true to env: flex you're actually switching to the latest infra version, see Upgrading to the Latest App Engine Flexible Environment Beta Release.
The machine type is no longer configured that way. Instead you'd configure a custom instance shape via its resources:
Resource settings
These settings control the computing resources. App Engine assigns a
machine type based on the amount of CPU and memory you've
specified. The machine is guaranteed to have at least the level of
resources you've specified, it might have more.
You can specify up to eight volumes of tmpfs in the resource settings.
You can then enable workloads that require shared memory via tmpfs and
can improve file system I/O.
For example:
resources:
cpu: 2
memory_gb: 1.3
disk_size_gb: 10
volumes:
- name: ramdisk1
volume_type: tmpfs
size_gb: 0.5

Resources