Can I download source code of already deployed app if I've lost it's source code on my hard drive? I've looked through application dashboard but couldn't find any option to check it out.
No, you can't, and there are no plans to offer such functionality. Sorry - consider investing in a VCS and backups next time.
SDK 1.4.0 has this sort of feature (Python only):
The developer who uploaded an app version can download that version's
code using the appcfg.py
download_app command. This feature can
be disabled on a per application
basis in the admin console, under the
'Permissions' tab. Once disabled,
code download for the application
CANNOT be re-enabled.
Related
So it seems from a few SO questions I've seen that this is a problem among other users. Recently one of our head dev's left and I inherited a lot of his projects. One of which, is a website that what seems like lives on an app engine from google cloud platforms. From the App Engine documentation, to download source code you use the appcfg.py download_app command. Which I did, however the only results I get back from that call is:
Fetching file list...
Fetching files...
And then it just ends. No error message or any kind of message at all, and of course, it did not download the source code into the output dir I specified.
Scratching my head and looking at various SO posts, someone mentioned something about going into the google cloud vm directly and doing the same command, and to my surprise finding the same exact behavior that I did in my local terminal.
This made me realize it must be something else at play. I took a look at my versions tab in the App Engine dashboard on GCP. I see my instance running, it correctly says Serving and if I click the link it brings me to the website which loads fine. However, under Size it says 0 B which made me think perhaps this is why the download_app isn't downloading anything, because the version is 0 B?
What I'm trying to figure out is why it says 0 B for the version, when clearly the site runs fine and how I can get the source code for this. Here's a screenshot for reference
And screenshot of my terminal (local). Obviously I omitted the -A and -V flags, but they are correctly set and if I purposely make them incorrect I do indeed get an error message.
EDIT
Just so everyone is aware, I also made sure my user had the correct permissions. Owner, App Engine Owner... and some others. I don't think that's the problem.
When you deploy an App Engine Flexible application, the source code is uploaded to Cloud Storage on your project in a bucket named staging.<project-id>.appspot.com. You can navigate in this bucket and download the source code for a specific version as a .tar file.
Alternatively, you can find the exact Cloud Storage URL for your source code by going to Dev Console > Container Registry > Build History and select the build for your version. You'll find the link to your source code under Build Information.
One thing to note however is that the staging... bucket is created by default with a Lifecycle rule that deletes files older than 15 days automatically. You can delete this rule if you want so that all versions' source code is kept indefinitely.
In your case I believe that may not have helped since files may have been deleted already but it's worth knowing you can get the source code from there (source code isn't pushed to Source Repository by default, your developer had to configure it manually).
Posting this since none of the listed methods on the web didn't take me to the code (by June 2021)
Note: appcfg.py is deprecated by Google
You could try accessing your source code through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download code automatically but you can copy-paste the code.
Advice: Push your code to a Git repository to avoid this hassle next time.
Hope you will find this helpful.
In the developer console you can select the respective project and check:
on the Services page - which services, AKA modules - as they used to be (and still are) called in various places, you app has deployed
on the Versions page - which versions for each of the services are deployed
This information is what appcfg.py download_app expects. See also:
the various appcfg.py options using its --help flag
How do I download a specific service's source code off of AppEngine?
You can also access the deployed source code live (if everything else fails it could still be a last resort method to get the code, but tedious), see my answer to Google Cloud DataStore automatic indexing
Update:
I just now noticed in your screenshot that it's a flexible environment app. The appcfg.py docs are in the standard environment section, I suspect it's not applicable to the flexible environment, for which what's deployed is actually a docker image built during the deployment operation. From Deploying your application:
Deploy your app to App Engine using the gcloud app deploy
command. This command automatically builds a container image by using
the Container Builder service and then deploys that image to the
App Engine flexible environment. The container will include any local
modifications that you've made to the runtime image.
It might be possible to access the code on the actual GCE instance running the app, by connecting to the running instance and starting a shell in your app container, see Connecting to the instance
I have a website that is currently running under GAE... unfortunately, I, nor anyone on the team, does not have access the local environment that it was created from.... Is it possible to create a local environment or at least get a copy of the application files and database from an existing GAE installation?
What you need is the application source code, not the "local environment".
Ideally this source code would be on a version control system (ie GIT,SVN), Google cloud platform provides free GIT repositories for your projects so you might try looking there first. There's also a tool for both Java and python that allow you to download the source of a deployed version, provided you are authenticated as either the dev who uploaded it or a project owner. EDIT: as stated by Dan Cornilescu this feature can be disabled.
As for the database info there's plenty of tools available to "export" your GAE datastore info, just consider for your project that it might be easier to do the queries manually than actually implementing this tools.
Thanks for help... But unfortunately, this code is not in GIT. Furthermore,
being new to Google hosting, I wasn't clear on my setup... My web instance is actually running within Compute Engine not Application Engine. Be that as it may, with some additional search, I was first able to find out how to browse my filesystem by accessing the VM Instances menu option under the Compute Engine section of the Google Cloud Platform interface. On the VM Instances page, it will show your instance and an option to the left side of the instance to connect with a drop down box that will allow you to open a browser window that shows the instance's file system. In addition to this, I found this link https://www.youtube.com/watch?v=9ssfE6ODpak that shows how to configure Filezila FTP client to access your server instance - very helpful. From there, I was able to download all of my site files from the var/www directory. Now, onto extracting my data... Thanks again!
Does anyone know how to request all logs of a Google App Engine application? By all logs, I mean all app versions logs.
We increment the version of our app with every deployment. I have a script that pulls the log files every 30 minutes for some offline analytics.
This is kinda what my script is calling right now.
appcfg.py --append --num_days=0 --include_all request_logs /opt/gae/appname/ /var/log/gae/appname.log
This does not get all app versions logs files, but only the version that's specified in the /opt/gae/appname/app.yaml file. The problem is that whatever is in that app.yaml file might not be live, but rather still in development.
It would be nice if we had --version=0 flag that pulled the logs for all the versions, like it does for --num_days=0.
You can fetch the logs using the LogService Api, see parameter version_ids.
I'm sure you could modify appcfg.py, to download all version logs.
You could add an issue asking for this feature at http://code.google.com/p/googleappengine/issues/list.
In my Google App Engine project, I'm currently using Twig for accessing the Datastore (but I suspect my question would be relevant for other 3rd party libs like SimpleDS, Objectify, etc.).
I was thinking that I should be able to delete some or all of the following JARs from my WEB-INF/lib folder:
datanucleus-appengine-[...].jar
datanucleus-core-[...].jar
datanucleus-jpa-[...].jar
geronimo-jpa_[...].jar
geronimo-jta_[...].jar
jdo2-api-[...].jar
But when I do, I get errors from Eclipse complaining that "The App Engine SDK JAR is missing in the WEB-INF/lib directory".
Is it really necessary to retain all these (unused) JARs?
If you're using the Google Plugin for Eclipse you can delete these JARs and then open the Properties for your Google Web Application project and open the Google > App Engine and then uncheck the Use datanucleus JDO/JPA to access the datastore. In fact, unchecking this box actually deletes these JARs for you.
I don't think you can delete them from the project (as you have said, eclipse starts to complain), but I've heard you don't have to upload them to your app, as long as you upload from the command line. This has the advantage that at startup your app won't have to load them up.
There are some details of uploading to app engine via the command line here and confirmation that you can do this here
If you manage to do this I would appreciate it it if you post links to the instructions you followed or post what you needed to do, as I have this on my todo list as well, but only got as far as seeing if it was possible. Thanks!
Sorry in advance for the long post but the problem I am facing here is quite crucial for me, so here we go...
I have a Eclipse GWT (2.0) Web Application using the the GAE and making transactions with its datastore.
On the other hand I would like to make sure that I can also deploy this web application on another infrastructure than the Google App Engine. Therefore I wanted to debug my web app using another servlet container (Tomcat 6) and another datastore (mySQL or MSSQL or any other, it doesn't really matter for now.)
In order to be able to debug an Eclipse web app with Tomcat it has to carry the Dynamic Web Project facet. If it doesn't then the new server that I add to Eclipse within Servers refuses to pick my GWT module in its list of supported apps. And not only GWT Web Apps don't carry it, but they don't even allow to alter the project's facets at all!
However, I found that adding the few relevant tags to the .project file can make it eligible within Eclipse to allow new project facets additions. Here are the tags I used:
<nature>org.eclipse.jdt.core.javanature</nature>
<nature>com.google.gwt.eclipse.core.gwtNature</nature>
<nature>org.eclipse.wst.common.project.facet.core.nature</nature>
<nature>org.eclipse.wst.common.modulecore.ModuleCoreNature</nature>
<nature>org.eclipse.jem.workbench.JavaEMFNature</nature>
<nature>org.eclipse.wst.jsdt.core.jsNature</nature>
<nature>com.google.appengine.eclipse.core.gaeNature</nature>
So I did that, I could "turn" the project into an Eclipse's Dynamic Web Project and by doing so made it compliant to be added to the list of modules handled by - let's say - Tomcat 6 within Servers.
So that is exactly what I wanted, I can now switch between two debug configurations at will in Eclipse in order to debug my GWT web app either on the App Engine or on Tomcat.
The problem is that adding the Dynamic Web Project facet disturbs the DataNucleus enhancer. For some reason, once the project has become a Dynamic Web Project, it seems that the enhancer is never called anymore and I get this error message saying that some classes haven't been enhanced. Playing with the Google... App Engine... ORM classes or checking/unchecking the Enhancer in Builders wouldn't change anything. Please note that the enhancer issue affects both debug configurations: Tomcat as well as the App Engine.
So I was thinking of two solutions.
(1) Disable the Google plugin's DataNucleus enhancer and perform the enhancements myself. So I installed the DataNucleus plugin to configure project specific enhancements by following their guide about the Eclipse plugin.
They say that by right-clicking on the project one can activate DataNucleus support for the given project's files. Unfortunately, after installing the latest version of their plugin for Eclipse, no such right-click menu appears! I have therefore no way to tell their plugin that I want to activate the enhancements on a given project! How frustrating is that?? (I uninstalled/reinstalled the plugin, let perform plenty of pending Eclipse updates... but still no right-click menu.)
Does anyone know of another way to activate DataNucleus enhancement? For now I would be happy even with a pretty manual trick as it is very critical for me to be able to perform this cross-servlet container debugging within Eclipse.
(2) Use this tip from the official GWT website http://code.google.com/webtoolkit/doc/latest/FAQ_DebuggingAndCompiling.html#How_do_I_use_my_own_server_in_hosted_mode_instead_of_GWT%27s
Ok that works, and for now let's say that it saves my life BUT... it requires hosted mode and therefore doesn't allow the use of GWT 2.0.
So regarding this workaround I would like to know if there is a way to do something similar under GWT 2.0?
Turning the GWT Web App into a Eclipse Dynamic Web Project would be the ideal solution for pursuing the development of my application. As I said above, that way, it is very convenient to switch from the App Engine to Tomcat and vice versa. So I favour workaround (1) over workaround (2). But anyway, some help or piece of advice regarding any of the two points will be very very welcome.
Thank you for reading this very long post!
I always use Google Plugin (GPE) for GWT development even when I deploy to Tomcat, where I simply disable GAE option.
Perhaps, I don't understand the question - but I have had no problems debugging my app running on jetty and then transferring the app to Tomcat. Are there any issues that I need to learn?
In fact, even when my app is plain jsp or servlets, without any GWT, I still used GPE. But GPE would refuse to run jetty without either GWT or GAE enabled. So I simply create a dummy GWT app. In all my years, I had never had to worry about differences between jetty and tomcat. And if I had to write an app where I had to exploit the differences in tomcat over jetty, I would consider myself a lousy programmer.
The only reason I see to push JEE server into an external server, is for profiling the app as it runs on tomcat.
JEE is JEE, whether Tomcat or Jetty. I have heard that some people have a bias against using Jetty, like preferring starbucks over neighbourhood cafe. Coffee is coffee.
I don't understand. May be you could write another thesis to explain why you cannot debug on jetty and then deploy on tomcat.
I ran into the exact same problem, I could edit the properties of my project and see the datanucleus menu in the context so the plugin was definitely working. My issue was I was using the "Java EE" perspective in eclipse. (Juno btw). I switched to the "Java" context and my datanucleus menu was back and I was able to enable support and enable automatic enhancement.
Window -> Show perspective -> Other -> Java