I am stuck in deciding between choosing Google App Engine Standard Vs. Flexible environment for a real world production. I want to use Java definitely. Need to use Firebase(latest version) for Authentication and Push notification; I'm not sure whether new Firebase is compatible with standard or flexible.
per the caution note in the following link, my impression is that recent Firebase is compatible is with Flexible Environment only.
https://cloud.google.com/solutions/mobile/firebase-app-engine-android-studio
All things being equal any standard environment app can also run in a flexible environment with minimal app changes. The reverse is not true - the standard environment restrictions are stricter than those for the flexible environment (hence the flexible in its name).
UPDATE: the above is not correct, the language specific section of the Migrating an Existing App Engine App guide should be checked for which standard env APIs are explicitly listed as compatible or incompatible. In some cases that could clarify the decision right away.
Assuming for the remaining of the answer that the choice between environments remains open after this check.
So - to get unstuck - I'd initially shoot for the standard environment (simpler to setup/manage and also potentially free, depending on the app's usage) and only decide to switch to flexible-only env if/when I hit an unavoidable issue caused by one of a standard environment restrictions and which is not an issue in the flex environment. If no such issue is hit you practically maintain the option to switch between the 2 deployment options as you desire.
As for your impression from the tutorial doc - I think it's unfounded, based on the Costs section...
Both Firebase and App Engine have free levels of usage. If your usage
of these services is less than the limits specified in the Firebase
free plan and the App Engine free quota, there is no charge for doing
this tutorial.
... in combination with the Pricing row Comparing environments table...
... as the standard env has a free daily Instance hours quota but the Compute Engine Pricing uses the Machine type billing model and doesn't have a free quota except for the initial limited free trial.
Which overall tells me that the standard environment is used in the tutorial :)
Confirmed in the Configuring the App Engine backend to use manual scaling section as the app's config doesn't have the <vm>true</vm> setting used to select the flexible environment.
Your preference should be app engine standard environment for the time being, unless you specifically need a feature only offered in flexible. Currently, there is very little documentation to go with Flexible app engine, and converting nearly any app from one to the other is NOT a simple task.
Anything you read that says "its easy to convert from standard to flexible" or vice versa is from BEFORE december 6'th, when vm:true was deprecated (along with all of the compat runtimes which let you use standard code on flexible app engine). Flexible now has it's own libraries for most things, and it's different enough to require nearly a complete re-write of many of your methods.
For example, when I converted my cloud storage controller to flexible, not a single line of code was useful beyond method names as the new library primarily utilizes Blobs instead of StorageObjects. (java)
Related
Sorry, if this is a naive question, but i've watched bunch of talks from google's staff and still don't understand why on earth i would use AE instead of CF?
If i understood it correctly, the whole concept of both of these services is to build "microservice architecture".
both CF and AE are stateless
both suppose to execute during limited period of time
both can interact with dbs and other gcp apis.
Though, AE must be wrapped into own server. Basically it utilizes a lot of complexities on top of the same capabilities as CF. So, when should i use it instead of CF?
Cloud Functions (CFs) and Google App Engine (GAE) are different tools for different jobs. Using the right tool for the job is usually a good idea.
Driving a nail using pliers might be possible, but it won't be as convenient as using a hammer. Similarly building a complex app using CFs might be possible, but building it using GAE would definitely be more convenient.
CFs have several disadvantages compared to GAE (in the context of building more complex applications, of course):
they're limited to Node.js, Python, Go, Java, .NET Core, and Ruby. GAE supports several other popular programming languages
they're really designed for lightweight, standalone pieces of functionality, attempting to build complex applications using such components quickly becomes "awkward". Yes, the inter-relationship context for every individual request must be restored on GAE just as well, only GAE benefits from more convenient means of doing that which aren't available on CFs. For example user session management, as discussed in other comments
GAE apps have an app context that survives across individual requests, CFs don't have that. Such context makes access to certain Google services more efficient/performant (or even plain possible) for GAE apps, but not for CFs. For example memcached.
the availability of the app context for GAE apps can support more efficient/performant client libraries for other services which can't operate on CFs. For example accessing the datastore using the ndb client library (only available for standard env GAE python apps) can be more efficient/performant than using the generic datastore client library.
GAE can be more cost effective as it's "wholesale" priced (based on instance-hours, regardless of how many requests a particular instance serves) compared to "retail" pricing of CFs (where each invocation is charged separately)
response times might be typically shorter for GAE apps than CFs since typically the app instance handling the request is already running, thus:
the GAE app context doesn't need to be loaded/restored, it's already available, CFs need to load/restore it
(most of the time) the handling code is already loaded; CFs' code still needs to be loaded. Not too sure about this one; I guess it depends on the underlying implementation.
App Engine is better suited to applications, which have numerous pieces of functionality behaving in various inter-related (or even unrelated) ways, while cloud functions are more specifically single-purpose functions that respond to some event and perform some specific action.
App Engine offers numerous choices of language, and more management options, while cloud functions are limited in those areas.
You could easily replicate Cloud Functions on App Engine, but replicating a large scale App Engine application using a bunch of discrete Could Functions would be complicated. For example, the backend of Spotify is App Engine based.
Another way to put this is that for a significantly large application, starting with a more complex system like App Engine can lead to a codebase which is less complex, or at least, easier to manage or understand.
Ultimately these both run on similar underlying infrastructure at Google, and it's up to you to decide which one works for the task at hand. Furthermore, There is nothing stopping you from mixing elements of both in a single project.
Google Cloud Functions are simple , single purpose functions which are fired while watching event(s).
These function will remove need to build your own application servers to handle light weight APIs.
Main use cases :
Data processing / ETL : Listen and respond to Cloud Storage events, e.g. File created , changed or removed )
Webhooks : Via a simple HTTP trigger, respond to events originating from 3rd party systems like GitHub)
Lightweight APIs : Compose applications from lightweight, loosely coupled bits of logic
Mobile backend: Listen and respond to events from Firebase Analytics, Realtime Database, Authentication, and Storage
IoT: Thousands of devices streaming events and which in-turn calls google cloud functions to transform and store data
App Engine is meant for building highly scalable applications on a fully managed serverless platform. It will help you to focus more on code. Infrastructure and security will be provided by AE
It will support many popular programming languages. You can bring any framework to app engine by supplying docker container.
Use cases:
Modern web application to quickly reach customers with zero config deployment and zero server management.
Scalable mobile backends : Seamless integration with Firebase provides an easy-to-use frontend mobile platform along with the scalable and reliable back end.
Refer to official documentation pages of Cloud functions and App Engine
As both Cloud Functions and App Engine are serverless services, this is what I feel.
For Microservices - We can go either with CF's or App Engine. I prefer CF's though.
For Monolithic Apps - App engine suits well.
Main differentiator as #Cameron points out, is that cloud functions reliably respond to events. E.g. if you want to execute a script on a change in a cloud storage bucket, there is a dedicated trigger for cloud functions. Replicating this logic would be much more cumbersome in GAE. Same for Firestore collection changes.
Additionally, GAE’s B-machines (backend machines for basic or manual scaling) have conveniently longer run times of up to 24 hours. Cloud functions currently only run for 9 minutes top. Further, GAE allows you to encapsulate cron jobs as yamls next to your application code. This makes developing a server less event driven service much more clean.
Of course, the other answers covered these aspects better than mine. But I wanted to point out the main advantages of Cloud Functions being the trigger options. If you want functions or services to communicate with each other, GAE is probably the better choice.
We're building an application using Google App Engine. From what I've seen there are 2 types of environments, Standard and Flexible, with huge differences. The problem is I can't seem to find any usage for the Standard environment apart from the faster instance creation compared to the Flexible environment and its ability to scale to 0 instances.
Assuming that our application will never have extreme traffic spikes but more like sinusoidal changes and will always have traffic (will never have to scale down to 0 instances) is there any reason to choose the Standard environment over Flexible?
Aside from traffic spikes, there are a few other reasons to consider standard:
1) Some services are not available/haven't been implemented yet on Flexible (like memcache)
2) The documentation for flexible is currently lacking. Most of what you can find on the web right now will be for standard.
3) Integration with android studio. You can deploy and test your app completely within android studio. However, this is not necessarily that big of a deal - once you get to know your way around the gcloud console you can do some pretty amazing things. Changes to your code are applied almost instantly without re-compiling anything. Just use "gradle jettyRun". Also, while running locally you can test against live app engine resources like the datastore in whatever project you choose (it could be a test project or even your live production project).
I think one clear advantage of using the standard environment is the free usage. You will get about 28h per day of free usage for the standard enviroment but you do not have any free usage for the flexible environment. You will always have to pay when you are using the flexible instance.
here you will find an overview of standard enviroment free usage limits
https://cloud.google.com/free/docs/always-free-usage-limits
Google Cloud Functions seems very interesting as it is serverless and zero maintenance solution. But, when is it appropriate to use Google Cloud Functions over Google App Engine?
Update:
As of June 12, 2018, Node.js 8.x is supported in Google App
Engine Standard environment along with the Flexible Environment.
Short answer: It depends upon your need.
Long answer: Here's the checklist
Runtime
Cloud Functions supports only Node.js at the moment and there aren't any plans, as far as I know, to introduce new runtimes there. If you're good with that, you can put Cloud Functions in your options.
App Engine does support Node.js, although it's only available in the Flexible environment. App Engine Standard Environment supports Python 2.7, Java 8, Java 7, PHP 5.5, Go 1.8 and 1.6, while App Engine Flexible Environment supports Python, Java, Node.js, Go, Ruby, PHP, or .NET. You can also provide your own runtime using a dockerfile in Flexible environment. So if you want to develop your application in anything other than Node.js, App Engine is the better option there.
Serverless Architecture
Are you looking for a serverless architecture? Are you frustrated with managing instances and having them scale up or down? Do you want to spend no time to manage your server? Go for Cloud functions if you answer yes to all of these questions.
Are you looking for fine grain control on no. of instances and billing of those. Do you want to have separate versions and want better control of those. Look for App Engine in this case.
Microservice
Can you break your code into smaller independant functions? Go for Cloud Functions.
App Engine do support Microservice architecture using same code base, but different yaml files to split the services, but it's upto you if you want to break them into services or not. We are running all our code into one monolithic application for last few years and it's still working good on App Engine.
Database
Is your app data stored in Firebase? Then Cloud functions can be used easily there. If not, App Engine is the better alternative. App Engine can connect to Firebase too, in case you're wondering.
There're other things to consider too, such as pricing and if you're looking to migrate existing application or if you're writing things from scratch. You can in fact, use both of the options. We are using App Engine (Python) Standard Environment for our application, but we have recently migrated few of our long running tasks on Cloud functions and they are working amazingly.
In my opinion App Engine is the answer to most of the things, where as Cloud Functions are made for specific requirements.
When what you desire is to execute a function (some logic of some sort) in response to an event originated in the cloud and you don't want to build (and be billed for) a full web application for just that.
From Product Overview:
Cloud computing has made possible fully serverless models of computing
where logic can be spun up on-demand in response to events originating
from anywhere. Construct applications from bite-sized business logic
billed to the nearest 100 milliseconds, only while your code is
running. Serve users from zero to planet-scale, all without managing
any infrastructure.
From What are Google Cloud Functions?
Google Cloud Functions is a serverless execution environment for
building and connecting cloud services. With Cloud Functions you write
simple, single-purpose functions that are attached to events emitted
from your cloud infrastructure and services. Your Cloud Function is
triggered when an event being watched is fired. Your code executes in
a fully managed environment. There is no need to provision any
infrastructure or worry about managing any servers.
If you already have a GAE app related to the piece of logic you want to implement it's probably simpler to just do it inside the app :)
Google's AppEngine has upgraded their Developer's Console and it includes a monitoring tool called Cloud Trace. Is this an improved version of the AppStats profiling tool? If so, can I remove the configuration files that were previously required, like appengine_config.py and the console path in app.yaml?
builtins:
- appstats: on
Cloud Trace is in beta, meaning "it's not covered by any SLA or deprecation policy and may be subject to backward-incompatible changes", while AppStats is a supported component of App Engine.
The implication, to me, is that you probably should not yet depend on Cloud Trace in production applications, but that it's an excellent idea to experiment with it in development versions -- if you use a lot of RPC traces/stats (a great way to keep your apps in fine shape!-), it's probably worth your while to experiment, both with using Cloud Trace on its own and with "doubling up" with both it and AppStats.
This way you will learn whether, in its current beta state, Cloud Trace supplies all the information you find relevant to tuning your apps' use of RPCs, as well as whether also having AppStats produces any change in Cloud Trace's functionality (per the docs, it shouldn't [*], but, beta does mean beta:-) in your apps (which is the most relevant to you:-).
[*] so to explicitly answer the Q "can I remove the configuration files that were previously required": the answer in theory is "yes!"; in practice, "it's beta, do try it in a development, not production, version, and see if it breaks" -- as for most any new feature in beta:-)
Having both on lets you compare and contrast the measurements they offer, checking if there's any systematic or random difference that could affect your use of said measurements (again, there shouldn't be, but, etc, etc:-).
Since AppStats is a supported component, it is subject to the usual deprecation policy, and thus it will be around for a minimum of one year after it's officially announced as deprecated (if such an announcement ever happens), so there is really no urgency for you to consider migrating production apps off it at this time.
Is there any way to apply Pending Latency and Idle Instances Application Settings to non-default versions of an application?
We have both Java and Python versions that must share the datastore/memcache, so we badly want to control these settings for both versions of the app.
If not possible immediately, any idea when GAE team might add Application Setting support for non-default versions?
these settings are global for your app and can't be setup differently for other versions than the dafault one.
can't really answer your second question. did you try to check out the issue tracker?
http://code.google.com/p/googleappengine/issues/list
you could file a feature request and see what happens
Non-default versions were intended to be for staged deployments to live and are not actually intended to be production apps. Thus, it's not likely that any such request would be accepted by Google. Additionally, it's not clear why you have a Python and a Java version of this app. Your users won't technically know the difference, so in the end it's probably best to pick one platform, Java or Python, and stick with it.
However, I imagine there is something you aren't telling us, so if you do need to maintain both applications for whatever reason, you could use REST to access your datastore from a completely separate app. With memcache, you could reduce any latency by caching frequently accessed data.
This would allow both apps to access the same datastore while also making both apps production apps with their own settings.