I've accidentally added a new filter to my GAE application. The status of the index is 'serving' now - however I don't need that index at all and I'd like to remove. How can I do that?
It is documented here:
Deleting Unused Indexes
When you change or remove an index
from index.yaml, the original index is
not deleted from App Engine
automatically. This gives you the
opportunity to leave an older version
of the app running while new indexes
are being built, or to revert to the
older version immediately if a problem
is discovered with a newer version.
When you are sure that old indexes are
no longer needed, you can delete them
from App Engine using the following
command:
appcfg.py vacuum_indexes myapp/
This command deletes all indexes for the app that are not mentioned in the local version of index.yaml.
For GAE / Java, the documentation includes this information:
Deleting Unused Indexes
...
When you are sure that old indexes are no longer needed, you can
delete them from App Engine using the vacuum_indexes action:
./appengine-java-sdk/bin/appcfg.sh vacuum_indexes myapp/war
This command deletes all indexes for the app that are not mentioned in
the local versions of datastore-indexes.xml and
generated/datastore-indexes-auto.xml.
As of Feb 2019 it's now:
gcloud datastore indexes cleanup index.yaml
In Windows Google AppEngine Java, we have to use appcfg.cmd command to delete unused indexes of deployed application.
Syntax :
appengine-java-sdk-path\bin\appcfg.cmd vacuum_indexes project-root-path\poject-name\war\
For gae-java, as JochenJung mentioned, the "vacuum_indexes" tool will work, but you'll have to emulate a python project in the following way:
Note that the vacuum tool seems only to work when pointed at *.appspot.com, not the local dev. environment.
create app.yaml for your app and put this in your /myapp/ root directory, minimally:
application: myproj
version: 4
runtime: python
api_version: 1
where "version" is your app's version, "myproj" the GAE name of your project.
create an index.yaml and put it in the same root dir. Instead of laboriously putting into that file the index information for indices you want to keep, it turns out that the tool is going to give you a yes/no confirmation for each and every index it deletes, so it is simpler just to indicate that ALL indices should be dropped, and use the confirmation to preserve the ones you want to keep.
indexes:
# AUTOGENERATED
Then run the command as shown above,
/appcfg.py vacuum_indexes /path/to/myproj/
If you're using maven mvn appengine:vacuum_indexes. No need to mvn appengine:update, the command updates the remote server.
A full list of maven commands here.
On Windows using Java, this command worked for me:
appcfg.cmd vacuum_indexes C:\Users\Name\AndroidStudioProjects\Project\backend\src\main\webapp\
Note: Make sure you have a datastore-indexes.xml in the webapp folder (these indexes will be spared).
With the current version of gcloud, you can simply do:
gcloud datastore cleanup-indexes index.yaml
which is more intuitive than calling appcfg.cmd [...].
gcloud datastore cleanup-indexes /path/to/file/index.yaml
this command no longer works.
gcloud datastore indexes cleanup /path/to/index.yaml
this is the new command.
you should run them in google cloud console. normally you can upload the index.yaml file using file upload feature in google cloud console. your file goes to a directly called _admin you can cd to there and call,
gcloud datastore indexes cleanup index.yaml
Tip
if you are using datastore in a java project, you have datastore-indexes.xml instead of index.yaml. You might have some trouble finding the index.yaml file if you don't know where to look.
you can simply find the path of the index.yaml file by looking at the deploy console in your IDE.
Related
Recently tried to update my Gaelyk project (yes, it's old, but it works well and I still use it), but Google App Engine will no longer accept the update. The error message returned is "Deployments using appcfg are no longer supported. See https://cloud.google.com/appengine/docs/deprecations". The thing is, I never used appcfg to deploy my application; I used Gaelyk and Gradle. But obviously Gaelyk must have used appcfg under the covers.
I did download the replacement Google Cloud SDK, but this new tool is not similar at all to how Gaelyk and Gradle worked. Is there anything I can do to get Gaelyk to work anymore? Or is Gaelyk just dead and I need to rewrite my application (like in Node.js or something instead of Groovy).
This will be hard, however I will try to help you as possible. I think you may try to migrate it somehow to app.yaml configuration of GAE.
I am not sure what plugins are used in the project. From Gaelyk temple project I can see that it's using appengine-geb which, according to the documentation, behind the scenes, is using gradle-appengine-plugin (there is wrong link on this doc, but proper is bellow).
On the github of gradle-appengine-plugin I have found following.
There is a note:
NOTE: All App Engine users are encouraged to transition to the new
gradle plugin for their projects.
And in FAQ part there is following information:
How do I deploy with gcloud?
If you're using gcloud to deploy your application, the newest version of app deploy > doesn't support war
directories, you will need to provide it with an app.yaml OR you can
use the appengineStage task to create a directory that is deployable
in /build/staged-app
$ ./gradlew appengineStage
$ gcloud app deploy build/staged-app/app.yaml --project [app id]
--version [some version]
NOTES:
You must explicitly define all config files your want to upload
(cron.yaml, etc)
This does not work with EAR formatted projects.
I think the best option will be to migrate to new appenine plugin or if not possible try to implement is with gcloud app deploy command crating the config files manually (at least app.yaml). And for this migration I can provide you this document.
I hope you will manage somehow...
I can confirm that Serge's answer on the Gaelyk Groups site works; the same procedure that he figured out also worked for me. To summarize:
Run gradlew appengineRun as run previously with Gaelyk.
Copy all jar files inside the build\exploded-app\WEB-INF\lib folder into a \src\main\webapp\web-inf\lib folder (for me the new lib folder did not exist previously).
To deploy, use the new required gcloud tool, and instead of running gradlew appengineUpdate (which fails now), instead run
gcloud app deploy appengine-web.xml where that XML file can be found in your webapp/WEB-INF directory. I navigated to that directory to run the gcloud command, but you can use a relative path there if your working directory is elsewhere. (There are a number of optional flags associated with the gcloud app deploy command, but I didn't need any of them.)
Serge needed to use these instructions to convert datastore-indexes.xml to index.yaml and run gcloud app deploy index.yaml, however, I didn't need to do this because I had no datastores.
I just stumbled on the following issue in App Engine Standard with a Python 2.7 enviroment
So I deployed to my test environment yesterday and today I had the idea of updating one of my applications. I do my normal "gcloud deploy ... " and it says updating 3 files ... While I actually changed a bunch of files. Basically my deploy command says the files are not changed.
After some searching around I found that files are being uploaded to a staging area and checked with a hash. Is it safe to actually clear this staging area, or does the gcloud command have some secret force option to actually force the files to be renewed.
The gcloud command has not given any errors what so ever, nor was it aborted at some point of deployment or something. So I have no errors, but my files aren't uploaded at all. I also tried modifying alot of files, and nothing changed
I never use the promote option for these rare cases that a deploy might fail
So anyone encountered this before, or has a solution to this issue ?
I also was encountering this and the only solution I could find was to deploy to a new bucket. To do this:
go to https://console.cloud.google.com/storage/browser and create a new bucket
redeploy using gcloud app deploy --bucket gs://your-new-bucket. (Change your-new-bucket to the actual bucket name)
This uploaded all the files again and created a new version in App Engine.
You can Go to https://console.cloud.google.com/storage/browser and delete your application bucket, on the next deploy it will be recreated. Additionally you can use the parameter --verbosity=info to check which files are being uploaded.
I have a Google App Engine app (Go lang, if that matters) that I would like to deploy more than once, with slightly different setup. Think production vs. QA.
env_variables in app.yaml seemed promising, but it seems I can only have one such file. For example, I don't see a way to call "goapp deploy" with app-qa.yaml.
How can I tweak deployment configuration? Is it possible to have more than one app.yaml, without custom script that copies files to a directory and manipulates app.yaml? Any other way to configure this?
My preference is to have the delta between the staging/QA and production reflected in (and controlled via) the VCS (git in my case):
my main development happens on the main branch, where the staging environment config is reflected in the .yaml files' contents
my production branch is pulled off the main branch and contains the production environment config - the .yaml files are modified accordingly
When I perform a deployment I do it from a workspace based on the appropriate branch depending on which config I need to deploy.
The delta between the production branch and the main branch is pretty much just deployment config changes. Whenever I'm happy with the staging results and I'm ready to deploy in production I just sync the production branch to the main refpoint corresponding to the OK'd staging deployment and deploy.
Another possible approach is to directly use the SDK's appcfg.py tool, which is what goapp deploy ultimately invokes anyways:
goapp deploy wraps the appcfg.py python tool provided in the SDK. You
can also invoke this tool directly if you need greater control over
the deployment:
The appcfg.py tool allows you to actually have and use alternate .yaml files residing in the same app/module directory (you'd probably have to use that anyways if you go with multiple or non-standard module configurations, since auto-detection from the app's dir won't work anymore):
appcfg.py update app-qa.yaml
App Engine Modules could be used for such purposes.
I cannot get my server code to update. I'm running a PHP instance on GAE and no matter what I do, the files won't update. In the source code view, I can see the files have updated, but when I attempt to access the updated file, I'm still viewing the old version. I've also attempted disconnecting my Bitbucket repo and using the appcfg.py update project-name command, but the files aren't refreshing when I attempt to access them. I'm not sure what to do to force the changes to take place.
My app.yaml contains the following code
- url: /(.+\.php)$
script: \1
secure: always
So the files should be getting read, right?
I was able to figure out what went wrong. I downloaded my code using appcfg.py download_app -A <your_app_id> -V <your_app_version> <output-dir> and noticed that I was downloading the old versions of the files (and wasn't downloading the new files). Turns out using source control within GAE will upload new code, but won't deploy it. I attempted to use appcfg.py update project-name one more time, but it didn't work. Turns out I didn't disconnect my Bitbucket account (could have sworn that I did...). Once disconnected, I was able to update the project using appcfg.py update project-name. While I was figuring this out, I reached out to Google support and received this message:
To use the feature of push to deploy you need to spin-up the Jenkins
Instance on GCE (Google Compute Engine) and then it will take the
updated code and execute it in the environment. Go through [1] for how
to enable the Jenkins instance and its configuration according to
different run time.
In your issue, you just mirrored the code from Bit Bucket to Cloud
Repository, as it is just doing the version control for the
application not executing the application. So basically you have have
the option of using Jenkins instance as I described above to test the
different version of the code or using the appcfg.py update command
from your local repository.
I haven't attempted to install and use Jenkins since I fixed it after disconnecting my Bitbucket account), but it may help others who have run into this problem.
The following command will upload the whole project to Google App Engine:
appcfg.py -R update C:/Users/user/Desktop/myproject/
However, I just made corrections locally to a single file of the project called index.php and would like to update the server version without uploading the whole project, which is large.
I have tried:
appcfg.py update C:/Users/user/Desktop/myproject/index.php
(notice I removed -R and also added the file name at the end) But this prints:
Usage: appcfg.py [options] update appcfg.py: error:
Directory does not contain an index.yaml configuration file.
any idea?
My inexperience with google app engine made me post this question, but I got the answer on my own after a little while.
Hahahaha I just discovered something I did not know before. After making changes to the index.php file and finally decided to redeploy the whole thing. This time it took less than a minute to redeploy it to the server (the first time it took over 10).
It seems that the local appgine scans for local changes and submits just that instead of the whole project!
Therefore re-running:
appcfg.py -R update C:/Users/user/Desktop/myproject/
will update the files that were changed only.
i believe you can't. its all or nothing.