There are many cases in which we run into a need to update only static content of a production GAE app.
Examples:
CSS changes, and other related resources, such as images
New HTML pages, API of the app stays the same (RESTful web services)
Client side flow changes (only JS)
Each of the above changes requires a full deployment.
I read the documentation (Java) and searched this forum (and others), and I'm pretty sure there's no such method exists.
Partial updates of the app in GAE is possible, like updating indexes, update cron jobs, backend management (deploy, start, stop, etc.), but nothing about update static content.
Does anyone know a way for just deploying / updating the static part of a GAE app?
Is this something that folks using GAE would need / use?
Cant be done. File a feature requets!
Related
I am very, very new to Google App Engine, but I need to administer a WebApp2 (so, Python) -based website that uses it. Specifically I need to filter a queryset and delete certain model instances.
Building the site locally is fine. Locally I can get at the models by hitting
http://localhost:8000/console
in my browser. This pulls up the "interactive console", and in there I can put some code like
from application.models.user import User
and it will work. From there I can create / delete User objects in the Datastore, so this local interactive console seems to have ORM-like functionality. (I'm coming from a Django background, so that's what it reminds me of).
However I don't seem to have this option on the live website. I was hoping to find it at
https://console.cloud.google.com/home/dashboard?project=<my project>
And I thought I had found it when I found the control to "activate google cloud shell". This brings up a command line on the web server, but the website's codebase doesn't seem to exist here, so launching the Python cli and trying to import the applications models like I can do locally doesn't work. And even if I could, I doubt it would be a Django-like ORM the way it is on my local build.
Am I thinking about this in the wrong way? How do you create / filter / delete / etc. Datastore objects via the backend in a GAE / WebApp2 website? Is the "Google cloud shell" even part of the answer?
To get access to your app code in the cloud shell one option would be to create in the shell a local copy of your app repository. For an example (addressing a different question, true) you can peek at Google Cloud: How to deploy mirrored Repository. But I'm not 100% certain if that will automatically give you access to your app's datastore. Worth a try IMHO.
Another option is to hook the desired operations as handler actions inside your app itself and execute them from there. Eventually hidden/protected/restricted via authentication, for example for admin users only. This is what I use for one-time datastore migrations that I need from time to time when I make changes to my entity models.
Finally, but not really a programatic access - you could use the Datastore page in the developer console to manually find, read and modify your entities, see Managing Datastore from the Console.
Use the remote_api_shell. You run this locally where your application code lives. Then you can import your models, and perform the same queries etc, and add modify/delete entities. I use this frequently for a range of tasks. Updating more than 100K records can become slow using this method.
https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi
We have Java servlets up and running on GAE, using blobstore, datastore and other cloud services.
Currently, we're starting a migration process to cloud Endpoints and we've hit an issue: if we use a different GAE project, we would not be able to query regarding current datastore entities (to the best of my knowledge, Google doesn't want you to do this - see
this question
and the GAE terms of service - section 3.3d), so we need to use the same project for both.
I looked up whether it's possible to have one GAE instance running Java servlets and one instance running Endpoints, but I found no conclusive answer anywhere.
If we try to implement and something goes wrong, we're looking at a potentially major issue for our users, so we need to be sure beforehand.
Has anyone tried something similar, and can assure us that this works?
You have 2 options to run the old and the new code inside the same app (thus with no issues sharing access to the datastore) but as separate engine instances, so they can be developed/deployed/managed independently:
as different versions of the same app/module(s):
the old version remains the default, the new one can be accessed at a different URL during development (possibly via URL routing)
you can use traffic splitting to do live A/B testing on the new code and for gradual final migration until you make the new version the default
as different modules of the same app:
both can run (fully functional) side by side indefinitely, but you need to be more careful during development
traffic is routed to the modules in several possible ways
final migration is done by publishing the new URLs, eventually re-directing the old URLs and finally bringing down the old module code
The 2 approaches can even be combined, if needed, as the final solution described by the OP's in this somehow similar question (for the python environment, but java equivalents exist): Google App Engine upgrading part by part
I'd like to build a new single page app using Rails 4 and Angular and Capistrano for deployment process.
I want all the front end to be a static app on Amazon S3, but I'm openminded for other suggestions.
What's important to me is a fast developing process with the ability to scale up easily.
I was wondering what is the best structure I should use:
keep all assets in app/assets and set Bower path to vender directory.
that way i can use rails precompile methods and enjoying Rails html tags for index.html, but i'm sure it will be easy to upload it to S3 and keep it separated.
keep all assets including Bower components in public/app directory, which will keep it as a complete separate application, but then i need to use Grunt or any other service for precompiling assets.
any other idea?
From my experience, I found this approach to work really well:
API app (Rails/Sinatra/Grape/Node/whatever) serves only JSON APIs. Deploys to a server, say api.yourapp.com. Serves Access-Control headers.
Static web app: started by generating with yeoman an AngularJS, Gulp, Bower app. Deploys using gulp aws deploy module to S3.
There's no real reason to have both views and apis in the same app or built with the same technology (as in Rails).
Now there are issues:
S3 doesn't support nicely Angular's HTML5 mode URLs. So pure S3 website isn't an option.
Facebook doesn't read OpenGraph tags that are not in the source of the page.
Couldn't figure out the state of Google/SEO and Angular apps. I didn't see the content in the search results.
So as a solution I introduced another web server app. Can be based on anything - pure rack, node etc. I chose rack.
Solutions to the problems:
Web server app was hosted on www.yourapp.com and proxied (and cached) requests to S3. It supported all URLs (html5Mode) - just proxied to index.html.
OpenGraph meta tags - the API had an endpoint that gets a URL or ID of an object and returns meta tags information. Web server issues a request to API once per URL (caches the response) and injects it in the served index.html.
SEO - as a middleware, used prerender for rack that rendered pages on the server.
As a bonus -
Most apps today have a landing page/marketing site and the actual app. Sometimes it's better to maintain these separately. The web sever knows according to a cookie which app to present on www.yourapp.com - actual app or marketing site. On sign in - set a cookie on client side and voila.
First, I think there's a bit of a confusion here, let me try to clear it up.
There are a couple of ways for achieving this
Pure client -> API
When you have a static application, there's no need to go through the Rails asset pipeline, there are far better ways to manage assets when you are using the tooling for client side applications.
For example, your client application will be an Angular application and you will manage assets with a combination of bower (dependencies) and grunt (build and distribution).
There's no point of deploying to S3 with Capistrano, if it's a pure static application, you can use aws CLI in order to just upload your content.
I'd go through a CDN as well. Something like Fastly works really well over Amazon S3.
I have a Rake task that uploads to S3 and then clears the cache on Fastly (if I need to).
As for your Rails application, it would act as an API, it should not have any assets
Combined
If you have a combined application, some of the actions are served by the server (Rails) and just invokes some client side code (Angular).
If this is the case, I would go through Rails asset pipeline and just keep everything as Rails best practice with compilation pre-deploy etc...
It's one of those questions where "it depends" is the answer really, it all depends on what you want to achieve.
When I have a client application, I try to have a pure client and have the server only as an API, with no assets at all, this way, I separate the concerns.
EDIT 9/9/15
I'd have to say that as long as you can, I'd keep the apps separate.
It's not always possible, especially with more complex apps.
Most apps I have seen in the recent months have kept the client side and the server side code separate, I have seen less use of rails and more use of rails-api because of that (some even ditched rails completely for thinner solutions).
I am working on Restful Web application. I am maintaining different project for web client code and Google app engine server code.
When ever i made changes in the client code, i rebuilt the client code and places inside the war folder of server project through build scripts.
Here i dont want to place all files directly to war folder and i wanted to put them under folder called 'Publish' for better maintainence. How can do it?
Is there any better way to maintaining client code and Google app engine server code?
The structure also works well for Mobile application in future.
I am still new to this too, but there is versioning. If you change the version number in your project manifest file, it does not become the default (i.e., visible to your original public URL). It is public and accessible for you to test. When you are ready to "publish" just switch the new version to be the default. Use the Manage section of the Dashboard and set the Version to be the default when you are ready.
To test any of the earlier versions, you access through the Manage and click on the specific version. I don't know if the persistent storage is versioned with this same mechanism -- I can image problems if you have a huge DB.
I've been thinking lately about the pros and cons of using AppEngine.
My concern would be, when we create application for GAE, the front-end code (the UI stuff) is served from the same application instance in the GAE cloud as with the Datastore codes.
The question would be when my applications grows:
For GAE:
Do I need to create multiple instance of my application?
If so, what do I need to manually update all instances?
For Appscale:
Do I also need to create multiple instance of my application?
If so, what do I need to manually update all instances?
GAE starts new frontend instances automatically, you even can't create or update frontend instances. You just need to configure min/max latency, min/max idle instances in Application Settings. See docs for performance settings
Btw, there are also Backend Instances that can be Resident and started manually from Admin Console. But it's useful only when you need something very specific
You seem to have missed the whole point of AppEngine, which is that Google takes care of scaling your app for you automatically. You seem to be confusing 'instance' with 'version' - you have control over which version of your app is serving, but Google dynamically creates and kills instances of that app depending on load. That's the main benefit of using AppEngine in the first place.