I know that there was some functionality for controlling the max idle instances and max latency after which a new instance was started. Now I can't find those in the new console.
Is this completely gone, or has the mechanism changed?
Those settings have all moved into your application's configuration files. Here are the python docs on it https://cloud.google.com/appengine/docs/python/modules/#Python_Instance_scaling_and_class
Related
Here is the deal;
I'm dealing with adding new worker (embbeded) to on running the cluster (flink statefun 2.2.1).
As you see the new task manager can be registered to the cluster;
Screenshot of new deployed taskmanager
But it doesn't initialize (it doesn't deploying sources);
What am I missing here?? (master and workers has to same jar files too? or it should be enough deploying taskmanager with jar file)
Any help would be appreciated,
Thx.
Flink supports two different approaches to rescaling: active and reactive.
Reactive mode is new in Flink 1.13 (released just this week), and works as you expected: add (or remove) a task manager, and your application will adjust to the new parallelism. You can read about elastic scaling and reactive mode in the docs.
Reactive mode is currently a work in progress, but might need your needs.
In broad strokes, for active mode rescaling you need to:
Do a stop with savepoint to bring down your current job while taking a snapshot of its state.
Relaunch with the new parallelism, using the savepoint as the starting point.
The exact details depend on how your cluster is deployed.
For a step-by-step tutorial, see Upgrading & Rescaling a Job in the Flink Operations Playground.
The above applies to rescaling statefun embedded functions. Being stateless, remote functions can be rescaled more straightforwardly.
We have a service running on Google App Engine.
If that service does not receive a traffic for some time then all instances are killed and the next call takes a few additional seconds to start the application.
We are thinking about specifying a min_instances option in app.yaml to always keep at least one instance alive.
We deploy new versions of that service quite frequently and keeping old versions for some time. Those old versions are not serving traffic and kept just in case.
What we would like to do is to always keep at least one instance of default service version alive and leave all other non-default versions with default behavior – we want them to be scaled automatically to 0 instances if they do not receive any traffic.
I didn't find such option in the documentation (https://cloud.google.com/appengine/docs/standard/python3/config/appref#scaling_elements) and didn't come to any workarounds.
I am thinking about creating a cron job (https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml) which will periodically "ping" only default version of my application periodically thus making it always asleep. But I am not sure if it is good solution.
Are there any better solutions to such case?
Thanks!
min_idle_instances config option seems to solve my problem.
Note following in the documentation: "This setting only applies to the version that receives most of the traffic" which is almost exactly my case:
automatic_scaling:
min_idle_instances: 1
I'm trying to debug 502 errors coming out of the nginx container with my AppEngine Flex setup.
I noticed that the logs indicate liveness and rediness checks being spammed very rapidly (see attached).
For clarification this is currently running a single instance in manual_scaling mode.
check_interval_sec is set for 30s on liveness_check and 5 sec on rediness_check.
Can anyone provide insight into what is going on here?
It looks like you setup readiness and liveness checks too aggressively in your app.yaml. Please keep in mind that the checks works for every instance so if you have a lot of instances, it will occur frequently.
If you only have one instance setup, then the behavior contradict what the documentation described. Please file an issue with us on the issue tracker.
On our development environment, we have been charged about USD100 every month for an instance we didn't know existed (and of course we are not using), and we cant find in the entire AppEngine or Console Engine.
Also, the usage report shows no activity for the whole month, but we are still getting the charges.
The instance is: Flex Instance Core Hours Sao Paulo
I found similar posts in stackoverflow, so, here are the questions:
- is this some bad strategy from Google???
- where can I see this instance to stop it or delete it?
- where can I see who started this instance and when?
Of course, I called google support and no answer received.
Many thanks!
Google Cloud Platform Support here! I found your ticket and see that you were provided an answer there already. In addition to what Dan described in his answer, if your app has currently the "Serving" status it will still run with the corresponding instances regardless of any requests incoming or not. As long as the version is serving it will continue to bill for hours that you are using. Also, if you are using automatic scaling with a minimum number of instances:
that specified number of instances run as resident instances while any
additional instances are dynamic
(Instance scaling description in GCP docs)
You can use basic or manual scaling if this is not what you're interested in.
Check the App Engine Versions pages for all your projects, you should find at least one with Flexible environment. The Deployed column should indicate who deployed it and when.
Based on that information you can decide to keep or delete the respective version(s). Simply stopping the instance may not be sufficient, depending on the scaling configuration for that service version GAE may automatically start one or more new instances.
You should also check the App Engine Instances pages for your projects and cross-reference that with the versions info to make sure no undesired instances are accidentally left behind (at least in the standard environment they are normally stopped when the respective versions are deleted, not entirely certain the same is true for the flex environment)
The running flexible environment instances are billed by the hour, even if they receive no requests, which could explain why you're seeing charges without any activity.
Apparently, the source of this instance was a firebase setting we made to make some test, and it automatically creates this instance. I shut off the billing account for this space, and instantly I received an email from firebase saying it detected changes that make some functions unavailable.
I have problem with automatic scaling at GAE.
One of my application module have instance class F4 and folowing automatic scaling settings.
<automatic-scaling>
<min-idle-instances>2</min-idle-instances>
<max-idle-instances>5</max-idle-instances>
<min-pending-latency>automatic</min-pending-latency>
<max-pending-latency>30ms</max-pending-latency>
<max-concurrent-requests>100</max-concurrent-requests>
</automatic-scaling>
I set maximum instances as 5 but when we run some jmeter tests with 300 users running instance number is increased to more than 40.
So it's looks that GAE don't handle this parameter correctly.
Does anybody have experience with this problem?
If you REALLY wanted to use the max instances cap, you have to use the app engine Application Settings UI at appengine.google.com. You can't set the max instances in app.yaml (yet).