Google's AppEngine has upgraded their Developer's Console and it includes a monitoring tool called Cloud Trace. Is this an improved version of the AppStats profiling tool? If so, can I remove the configuration files that were previously required, like appengine_config.py and the console path in app.yaml?
builtins:
- appstats: on
Cloud Trace is in beta, meaning "it's not covered by any SLA or deprecation policy and may be subject to backward-incompatible changes", while AppStats is a supported component of App Engine.
The implication, to me, is that you probably should not yet depend on Cloud Trace in production applications, but that it's an excellent idea to experiment with it in development versions -- if you use a lot of RPC traces/stats (a great way to keep your apps in fine shape!-), it's probably worth your while to experiment, both with using Cloud Trace on its own and with "doubling up" with both it and AppStats.
This way you will learn whether, in its current beta state, Cloud Trace supplies all the information you find relevant to tuning your apps' use of RPCs, as well as whether also having AppStats produces any change in Cloud Trace's functionality (per the docs, it shouldn't [*], but, beta does mean beta:-) in your apps (which is the most relevant to you:-).
[*] so to explicitly answer the Q "can I remove the configuration files that were previously required": the answer in theory is "yes!"; in practice, "it's beta, do try it in a development, not production, version, and see if it breaks" -- as for most any new feature in beta:-)
Having both on lets you compare and contrast the measurements they offer, checking if there's any systematic or random difference that could affect your use of said measurements (again, there shouldn't be, but, etc, etc:-).
Since AppStats is a supported component, it is subject to the usual deprecation policy, and thus it will be around for a minimum of one year after it's officially announced as deprecated (if such an announcement ever happens), so there is really no urgency for you to consider migrating production apps off it at this time.
Related
We're building an application using Google App Engine. From what I've seen there are 2 types of environments, Standard and Flexible, with huge differences. The problem is I can't seem to find any usage for the Standard environment apart from the faster instance creation compared to the Flexible environment and its ability to scale to 0 instances.
Assuming that our application will never have extreme traffic spikes but more like sinusoidal changes and will always have traffic (will never have to scale down to 0 instances) is there any reason to choose the Standard environment over Flexible?
Aside from traffic spikes, there are a few other reasons to consider standard:
1) Some services are not available/haven't been implemented yet on Flexible (like memcache)
2) The documentation for flexible is currently lacking. Most of what you can find on the web right now will be for standard.
3) Integration with android studio. You can deploy and test your app completely within android studio. However, this is not necessarily that big of a deal - once you get to know your way around the gcloud console you can do some pretty amazing things. Changes to your code are applied almost instantly without re-compiling anything. Just use "gradle jettyRun". Also, while running locally you can test against live app engine resources like the datastore in whatever project you choose (it could be a test project or even your live production project).
I think one clear advantage of using the standard environment is the free usage. You will get about 28h per day of free usage for the standard enviroment but you do not have any free usage for the flexible environment. You will always have to pay when you are using the flexible instance.
here you will find an overview of standard enviroment free usage limits
https://cloud.google.com/free/docs/always-free-usage-limits
I am stuck in deciding between choosing Google App Engine Standard Vs. Flexible environment for a real world production. I want to use Java definitely. Need to use Firebase(latest version) for Authentication and Push notification; I'm not sure whether new Firebase is compatible with standard or flexible.
per the caution note in the following link, my impression is that recent Firebase is compatible is with Flexible Environment only.
https://cloud.google.com/solutions/mobile/firebase-app-engine-android-studio
All things being equal any standard environment app can also run in a flexible environment with minimal app changes. The reverse is not true - the standard environment restrictions are stricter than those for the flexible environment (hence the flexible in its name).
UPDATE: the above is not correct, the language specific section of the Migrating an Existing App Engine App guide should be checked for which standard env APIs are explicitly listed as compatible or incompatible. In some cases that could clarify the decision right away.
Assuming for the remaining of the answer that the choice between environments remains open after this check.
So - to get unstuck - I'd initially shoot for the standard environment (simpler to setup/manage and also potentially free, depending on the app's usage) and only decide to switch to flexible-only env if/when I hit an unavoidable issue caused by one of a standard environment restrictions and which is not an issue in the flex environment. If no such issue is hit you practically maintain the option to switch between the 2 deployment options as you desire.
As for your impression from the tutorial doc - I think it's unfounded, based on the Costs section...
Both Firebase and App Engine have free levels of usage. If your usage
of these services is less than the limits specified in the Firebase
free plan and the App Engine free quota, there is no charge for doing
this tutorial.
... in combination with the Pricing row Comparing environments table...
... as the standard env has a free daily Instance hours quota but the Compute Engine Pricing uses the Machine type billing model and doesn't have a free quota except for the initial limited free trial.
Which overall tells me that the standard environment is used in the tutorial :)
Confirmed in the Configuring the App Engine backend to use manual scaling section as the app's config doesn't have the <vm>true</vm> setting used to select the flexible environment.
Your preference should be app engine standard environment for the time being, unless you specifically need a feature only offered in flexible. Currently, there is very little documentation to go with Flexible app engine, and converting nearly any app from one to the other is NOT a simple task.
Anything you read that says "its easy to convert from standard to flexible" or vice versa is from BEFORE december 6'th, when vm:true was deprecated (along with all of the compat runtimes which let you use standard code on flexible app engine). Flexible now has it's own libraries for most things, and it's different enough to require nearly a complete re-write of many of your methods.
For example, when I converted my cloud storage controller to flexible, not a single line of code was useful beyond method names as the new library primarily utilizes Blobs instead of StorageObjects. (java)
Where can I find the SLA and deprecation policy of GAE Go runtime.
As per the features page,
The runtime is experimental,
Go
Experimental The Go runtime executes your code in a Go 1.1 environment. This runtime is Experimental. Release Notes Documentation SDK
But, core features are GA (Data storage, retrieval, and search, ...).
So, where can I see if Google decide not to support Go runtime in app engine, what deprecation policy would be in effect.
Thanks.
Update: In April 2018 I no longer see any signs that Go support is experimental. You should research the current state of things rather than relying on the years-old answer below.
I think there is no deprecation policy for the Go runtime, not even a hard commitment to keep the service running, based on language in section 7.3 of the TOS: 'This Deprecation Policy doesn't apply to versions, features, and functionality labeled as "experimental."' I'm not a lawyer.
In reality I don't think there are any signs Google is likely to shut it down for the short term. They haven't deprecated a language so far, the runtime has been out for a while and is still updated (SDK released 2011, and is actively updated as of Nov '13), it has users (goread.io, at least a couple little projects in Google), and Go is a Google creation. But no one has promised continued support or backwards compatibility.
It's quite reasonable to be wary of lock-in with any platform/vendor, especially if they have no contractual obligations to you. It might be wise to ensure you can eventually port your app if needed (to Heroku, your own set of VMs, etc) by not relying too heavily on the quirks of GAE's datastore and other quirks.
I know this has been asked one way or another before, but most of the main issues to do with GAE stability seem to have been asked around the end of 2008, early 2009, or aren't directly related to games at scale (which I'm interested in).
Basically, I have been arguing back and forth with my business partner about whether to use GAE or AWS for the back-end of our social game engine, and now it's crunch time. I love GAE (Java) for so many reasons, and although it used to be unstable, it's pretty good now. The main argument in favour of AWS is the fact that AWS has proven itself with multiple games running tens of millions of active users per day. The obvious pin-up child for AWS is Zynga, with its Farmville peaking at 80+million DAU. And that's just one of the hugely successful games running on the AWS infrastructure. Remarkable achievement.
So, one way or another it's KNOWN to work. GAE on the other hand doesn't have any examples that I could find doing these sorts of numbers. Not even close. So can I trust it? Is there a single example of a large social game with 2 million+ Daily Active Users, using GAE?
The main considerations for our social game back-end are:
Reliable CDN (Amazon CloudFront/S3 is excellent for this, as is Google's obviously excellent DataStore).
Ability to scale without falling over (AWS-EC2 is proven here, GAE doesn't seem to have examples of large game apps which can run into the 1000s of requests per second. GAE used to be quite unstable in this regard and so is my main concern).
Reliable no-SQL database. (AWS-SimpleDB and Google's DataStore are both excellent for this. We really don't need SQL).
Support/someone to call/contact if there is a problem. (This is one of the biggest worries with GAE. I have no idea who I can call, or if it's even possible. AWS has an SLA and support.)
I look forward to your thoughts, but please also note, this is not intended to start any sort of flame war. I love both systems, but both have their positives and negatives, but I'm about to make an architectural decision that likely won't be undone moving forward.
Regards,
Shane
I've never worked with AWS-EC2 so I'm going to share my knowledge just on the Google App Engine side.
Google App Engine is not meant to be a CDN; though it can serve static content through its powerful infrastructure providing caching close to the users, it does not guarantee the same kind of high quality and high availability service of a real CDN because it's not part of its duties.
Further data:
Maximum size of a file using the BlobStore service: 2 Gigabytes
Maximum size of a static file: 10 Megabytes
Currently App Engine always returns 200 status for static files even on Conditional gets (you have to rely on third party caching library like cirruxcache for example).
Recently Google App Engine team has shut down the App Gallery for one simple reason: too many Toy Apps!
Google wants to counteract this tendency showing successful businesses case studies; here are some of them:
BuddyPoke (viral Facebook app with 65 million installs)
WalkScore (serves 3 million request a day to thousands of real estate partner sites)
Webfillings
Snapabug
Optimizely
Ubisoft Facebook TikTok game
Other interesting case studies here
"We are well aware of downtimes and reliability issues, and are working hard to solve them: Improving App Engine reliability is our number one priority" was recently said by a Google Developer Relations Manager here.
App Engine is still in beta and is an evolving platform so you have to be prepared to deal with downtimes and issues.
Google App Engine team has just launched a preview of App Engine for Business providing 99.9% uptime service level agreement and premium developer support available.
Here is my opinion for what it's worth:
I'm aware that it's a tough call; having read a lot of articles about GAE I have mixed feelings about it because you can go from the recent catastrophic Carlos Ble report to the happy experience of Flower Garden or Gri.pe.
App Engine for Business looks promising and I would consider it in the case of a serious business project plan.
The fresh SDK 1.4.0 is huge and it clearly shows that the Team is really pushing hard to fix some annoying issues (Warmup requests) and relaxing some limitations (10 minutes process on TaskQueue).
Last thing to consider: if you are going to have big numbers, the Google App Engine Team will probably take your app as a successfull case study to follow with a boost of free and powerful Hype.
BuddyPoke is one example of a large-scale social app running on GAE. How large I'm not sure. This article says 30m daily page views (not users):
http://googleappengine.blogspot.com/2008/10/app-engine-case-studies.html
Their facebook page says 2.7 million monthly (not daily) users:
http://www.facebook.com/buddypoke
Although, they are also on a heap of other social networks:
http://www.buddypoke.com/
Personally I decided to go with GAE, for a couple of main reasons:
The unit of scalability is a single request, not a whole instance like it is with AWS.
I can work at a higher level, without having to worry about configuring instances.
If your point 4 is a big one for you, then you may be better off with AWS. With GAE there appears to be nothing you can do, and no-one you can contact.
About a week ago I had an issue with my app - it had suddenly started failing in Google's code, in a location which had been working fine for the last 5 days, ie since I had last uploaded my app. The only way to report issues to Google seems to be via their production issue template, here:
http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue
I reported the issue, and didn't hear anything. Since it's running on Google's servers I was unable to resort to any 'usual' emergency tactics like restarting a server. An hour later and the problem resolved itself - I'm not sure if someone at Google saw my message and fixed something, or if it just went away. I updated my bug report to say the problem was fixed, but even now a week later the issue hasn't been closed or even acknowledged. Also since the issue has to be posted publicly, my app is now getting random hits from bots.
Admittedly my app is currently only in beta and so only has a hundred or so users, and so it wasn't a major incident for me. If I was getting thousands / millions of hits, maybe either Google would have noticed the problem themselves earlier, or they would have paid more attention to my bug report.
On your point 3, even my small app with a small amount of traffic throws occasional data store errors (even during times which aren't reported on the availability charts as outages).
Having said this, I still like GAE (I am using the Python version), and plan to stick with it. The promise of GAE is its scalability - although it falls over occasionally now for my small traffic, it shouldn't fall over any more when it scales to much more traffic (ie your point 2), provided I've coded it correctly to avoid contention. I'll see how it goes.
Finally regarding your point 1, the blobstore and/or static files are more like a CDN on GAE, than the datastore. However for very large amounts of traffic, a real CDN may be cheaper. It's also not necessarily a CDN, see Google app engine & CDN.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
What alternatives are there to GAE, given that I already have a good bit of code working that I would like to keep. In other words, I'm digging python. However, my use case is more of a low number of requests, higher CPU usage type use case, and I'm worried that I may not be able to stay with App Engine forever. I have heard a lot of people talking about Amazon Web Services and other sorts of cloud providers, but I am having a hard time seeing where most of these other offerings provide the range of services (data querying, user authentication, automatic scaling) that App Engine provides. What are my options here?
AppScale
AppScale is a platform that allows users to deploy and host their own Google App Engine applications. It executes automatically over Amazon EC2 and Eucalyptus as well as Xen and KVM. It has been developed and is maintained by AppScale Systems. It supports the Python, Go, PHP, and Java Google App Engine platforms.
http://github.com/AppScale/appscale
In the mean time...
...it is amost 2015 and it seems that containers are the way to go forward. Alternatives to GAE are emerging:
Google has released Kubernetes, container scheduling software developed by them to manage GCE containers, but can be used on other clusters as well.
There are some upcoming PaaS on Docker such as
http://deis.io/
http://www.tsuru.io/
even Appscale themselves are supporting Docker
Interesting stuff to keep an eye on.
I don't think there is another alternative (with regards to code portability) to GAE right now since GAE is in a class of its own. Sure GAE is cloud computing, but I see GAE as a subset of cloud computing. Amazon's EC2 is also cloud computing (as well as Joyent Accelerators, Slicehost Slices), but obviously they are two different beasts as well. So right now you're in a situation that requires rethinking your architecture depending on your needs.
The immediate benefits of GAE is that its essentially maintenance free as it relates to infrastructure (scalable web server and database administration). GAE is more tailored to those developers that only want to focus on their applications and not the underlying system.In a way you can consider that developer friendly. Now it should also be said that these other cloud computing solutions also try to allow you to only worry about your application as much as you like by providing VM images/templates. Ultimately your needs will dictate the approach you should take.
Now with all this in mind we can also construct hybrid solutions and workarounds that might fulfill our needs as well. For example, GAE doesn't seem directly suited to this specific app needs you describe. In other words, GAE offers relatively high number of requests, low number of cpu cycles (not sure if paid version will be any different).
However, one way to tackle this challenge is by building a customized solution involving GAE as the front end and Amazon AWS (EC2, S3, and SQS) as the backend. Some will say you might as well build your entire stack on AWS, but that may involve rewriting lots of existing code as well. Furthermore, as a workaround a previous stackoverflow post describes a method of simulating background tasks in GAE. Furthermore, you can look into HTTP Map/Reduce to distribute workload as well.
As of 2016, if you're willing to lump PaaS (platform as a service) and FaaS (function as a service) in the same serverless computing category, then you have a few FaaS options.
Proprietary
AWS Lambda
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.
AWS Step Functions complements AWS Lambda.
AWS Step Functions makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function lets you scale and change applications quickly. Step Functions is a reliable way to coordinate components and step through the functions of your application. Step Functions provides a graphical console to arrange and visualize the components of your application as a series of steps. This makes it simple to build and run multi-step applications. Step Functions automatically triggers and tracks each step, and retries when there are errors, so your application executes in order and as expected. Step Functions logs the state of each step, so when things do go wrong, you can diagnose and debug problems quickly. You can change and add steps without even writing code
Google Cloud Functions
As of 2016 it is in alpha.
Google Cloud Functions is a lightweight, event-based, asynchronous compute solution that allows you to create small, single-purpose functions that respond to cloud events without the need to manage a server or a runtime environment. Events from Google Cloud Storage and Google Cloud Pub/Sub can trigger Cloud Functions asynchronously, or you can use HTTP invocation for synchronous execution.
Azure Functions
An event-based serverless compute experience to accelerate your development. It can scale based on demand and you pay only for the resources you consume.
Open
Serverless
The Serverless Framework allows you to deploy auto-scaling, pay-per-execution, event-driven functions to any cloud. We currently support Amazon Web Service's Lambda, and are expanding to support other cloud providers.
IronFunctions
IronFunctions is an open source serverless computing platform for any cloud - private, public, or hybrid.
It remains to seen how well FaaS competes with CaaS (container as a service). The former seems more lightweight. Both seem suited to microservices architectures.
I anticipate that functions (as in FaaS) are not the end of the line, and that many years forward we'll see further service abstractions, e.g. test-only development, followed by plain-language scenarios.
Alternatives:
1. AppScale
2. Heroku.
Ref: Alternative for Google AppEngine?
Amazon's Elastic Compute Cloud or EC2 is a good option. You basically run Linux VMs on their servers that you can control via a web interface (for powering up and down) and of course access via SSH or whatever you normally set up...
And as it's a linux install that you control, you can of course run python if you wish.
Microsoft Windows Azure might be worth consideration. I'm afraid I haven't used it so can't say if it's any good and you should bear in mind that it's a CTP at the moment.
Check it out here.
A bit late, but I would give Heroku a go:
Heroku is a polyglot cloud application platform. With Heroku, you
don’t need to think about servers at all. You can write apps using
modern development practices in the programming language of your
choice, back it with add-on resources such as SQL and NoSQL databases,
Memcached, and many others. You manage your app using the Heroku
command-line tool and you deploy code using the Git revision control
system, all running on the Heroku infrastructure.
https://www.heroku.com/about
You may also want to take a look at AWS Elastic Beanstalk - it has a closer equivalence to GAE functionality, in that it is designed to be PaaS, rather than an IaaS (i.e. EC2)
If you're interested in the cloud, and maybe want to create your own for production and/or testing you have to look at Eucalyptus. It's allegedly code compatible with EC2 but open source.
I'd be more interested in seeing how App Engine can be easily coupled with another server used for CPU intensive requests.
TyphoonAE is trying to do this. I haven't tested it, but while it is still in beta, it looks like it's atleast in active development.
The shift to cloud computing is happening so rapidly that you have no time to waste for testing different platforms.
I suggest you trying out Jelastic if you are interested in Java as well.
One of the greatest things about Jelastic is that you do not need to make any changes in the code of your application, except the changes for your application functionality, but not for the reason the chosen platform demands this. With reference to this you do not actually waste your time.The deployment process is just flawless, and you can deploy your .war file anywhere further.Using GAE requires you to modify the app around their system needs. In case if you happen to get working with Java and start looking for a more flexible platform, Jelastic is a compatible alternative.
You can also use Red Hat's Cape Dwarf project, to run GAE apps on top of the Wildfly appserver (previously JBoss) without modification.
You can check it out here:
http://capedwarf.org/