I am newbie and I was playing around couple of months back with Google Cloud and by mistake I somehow have added a Version/hello world project that looks like below and it is doing some logging which is taking 100 GB of space and Google is charging me for that space. I just want to get rid of it but I am not able to do so. Please help me.
Screenshot for reference.
EDIT
Issue: Why am I not able to completely delete it. I am beginner so please guide me how to get rid of it? I am getting charged every month. Where can I click and delete it.
P.S: I know I have not asked this question properly but I don't have the proper terminology knowledge to ask my query properly.
Just click on the square in the left side of the version row in the table pictured in your 1st screenshot to select it and the Delete button should activate.
Update:
Hm, ok. Looks like the only remaining version can't be deleted. You could also:
disable your app if this is the only service in it from the Settings Menu.
delete the app if you no longer need that app ID (you won't be able to re-use that ID for a new app)
deploy a new version which doesn't do anything, switch traffic to it and delete the old version. I'd make the new version a standard environment one, which has a free quota, instead of a flexible environment one, eventually with just a static dir handler pointing to an empty dir.
You can use this with your CI/CD script. This works with Linux.
gcloud app versions delete `gcloud app versions list | sed 's/ */:/g' | cut -f 2 -d : | tail -n +2 | head -n -1` --quiet
gcloud app versions delete [NAME]
For quick and nice way of deleting App Version through CLI :
gcloud app services delete {services list} --version=XXXX
Example: gcloud app services delete night-worker default day-worker --version=version2019
That way you will save yourself time for deleting App Engine versions.
Related
I have a Google App Engine app, which connects to Google Cloud Storage.
I noticed that the amount of data stored was unreasonably high (4.01 GB, when it should be 100MB or so).
So, I looked at how much each bucket was storing, and I found that there was an automatically created bucket called us.artificats. that was taking up most of the space.
I looked inside, and all it has is one folder: containers/images/.
From what I've Googled, it seems like these images come from Google Cloud Build.
My question is, can I delete them without compromising my entire application?
I have solved this problem by applying a deletion rule. Here's how to do it:
Open the project in Google Cloud console
Open the storage management (search for "Storage" for example).
In the Browser tab, select the container us.artifacts....
Now, open the Lifecycle section. You should see something like:
Click on Add a rule and provide the following conditions:
In the action, select Delete object
In the conditions, select Age and enter for example 3 days
Click on create to confirm the creation
Now all objects older than 3 days will be automatically deleted. It might take a few minutes for this new rule to be applied by Google Cloud.
For those of you seeing this later on, I ended up deleting the folder, and everything was fine.
When I ran Google Cloud Build again, it added items back into the bucket, which I had to delete later on.
As #HarshitG mentioned, this can be set up to happen automatically via deletion rules in cloud storage. As for myself, I added a deletion step to my deployment GitHub action.
Here is the reference to the documentation: link
Built container images are stored in the app-engine folder in Container Registry. You can download these images to keep or run elsewhere. Once deployment is complete, App Engine no longer needs the container images. Note that they are not automatically deleted, so to avoid reaching your storage quota, you can safely delete any images you don't need. For more information about managing images in Container Registry, see the Container Registry documentation.
This can be automated by adding a Lifecycle rules like #HarshitG mentioned.
You can add a trigger to your lifecycle rules on console.cloud.google.com.
Access the bucket with artifacts (default is "us.artifacts.yourAppName.appspot.com")
Go to "Life cycle".
Click on "Add a rule".
Check "Object delete" and press "Continue".
Check the filter to delete the bucket, I chose "age" and selected three days as the number of auto delete old elements (after element has 3 days of life it's auto deleted).
Click on "Create" and the rule is working now, then you do not need to visit every day to clean the bucket.
Same issue. Thanks for the update, Caleb.
I'm having the same issue, but I don't have an App running; I just have:
Firebase Auth
Firestore
Firebase Functions
Cloud Storage
Not sure why I have 4GB stored in those containers, and I'm not sure if I should delete them or if that would break my functions.
UPDATE:
I deleted the container folder and all still works. Not sure if those are backups or whatnot, but I can't find anything online or in the docs. I will post here if something happens. As soon as a cloud function ran, the folder had 33 files again.
I recommend against setting up a lifecycle rules on your Storage buckets. It will likely lead to breaking subsequent updates to the function (as described in Cloud Function build error - failed to get OS from config file for image)
If you are interested in cleaning up container images, you should instead delete container images stored in Google Cloud Registry https://console.cloud.google.com/gcr. Deleting container images on GCR repos will automatically cleanup objects stored in your Cloud Storage.
https://issuetracker.google.com/issues/186832976 has relevant information from a Google Cloud Functions engineer.
I want to automatically start/stop our app engine services by running a bash script.
I know it's easy to run gcloud app versions start/stop, but I don't want to manually check the version number. I want to dynamically pass the version that is serving 100% traffic to gcloud and tell it to stop.
On the flip side, I also want to tell gcloud to start the most recently deployed version.
What's the recommended way to do this?
Thanks!
One way to do that is to use gcloud's keys and flags: projections, --format, --filters. To read more directly from the terminal use gcloud topic, for example:
gcloud topic projections
In order to see what fields/properties are available use --format=flattened, like:
gcloud app services list --format=flattened
For the sake of simplicity I will leave outside everything but gcloud.
for SERVICE in $(gcloud app services list --format='table[no-heading](id)'); do
echo "for service $SERVICE :"
RECENT=$(gcloud app versions list --format='table[no-heading](id)' --filter="service=$SERVICE" | tail -n1)
echo 'y' | gcloud app versions start $RECENT
VERSIONS=$(gcloud app versions list --format='table[no-heading](id)' --filter="service=$SERVICE AND version.servingStatus=SERVING AND NOT id=$RECENT" | tr '\n' ' ')
echo 'y' | gcloud app versions stop $VERSIONS
done
'table[no-heading](service)' outputs a table without heading, which is set in brackets, and a single column with service IDs, which is set in parentheses.
--filter="service=$SERVICE AND version.servingStatus=SERVING AND NOT id=$RECENT" will only show versions from indicated service that are serving, except the one indicated by RECENT.
Additionally, if you would want to use dates for filtering:
gcloud app versions list --format='table(id, version.servingStatus, version.createTime.date(format="%s"))' --filter="service=default" --sort-by="~version.createTime"
version.createTime.date(format="%s") is a function date converting version.createTime.date into the number of seconds since the Epoch.
%s comes from strftime(3) and returns dates in Epoch format which is easier to understand and compare.
--sort-by="~version.createTime"sorts by creation date and because of ~ in descending order.
One approach is to use the --stop-previous-version and/or --promote options when deploying with gcloud app deploy (they should be the default if I interpret the docs correctly, unless you use --no-stop-previous-version and/or --no-promote):
--promote
Promote the deployed version to receive all traffic. Overrides the
default app/promote_by_default property value for this command
invocation. Use --no-promote to disable.
--stop-previous-version
Stop the previously running version when deploying a new version that
receives all traffic. Overrides the default
app/stop_previous_version property value for this command
invocation. Use --no-stop-previous-version to disable.
But, if you're using the standard environment and dynamic scaling, you should be aware that if the previous version handles a lot of traffic there may be service degradation/interruptions during the switch (it may take a while for the GAE autoscaler to determine how many new version instances it needs to spin up to handle that traffic, see Use traffic migration or splitting when switching to a new default version. You can perform these programmatically, see Not applicable to the flex environment, which doesn't support traffic splitting.
Also potentially of interest: GAE shutdown or restart all the active instances of a service/app
You can only control at which deployed version(s) is the traffic routed to by default, you can't really stop all traffic to a deployed version, it can always be reached via targeted routing.
BTW, the gcloud app versions [start|stop] commands are only applicable to manually scaled services:
It may only be used if the scaling module for your service has been
set to manual.
As far as I know, you need to manually change the default version on app engine admin panel after you deployed a new version. Is there a way to automatically change default version to the latest one?
You can appcfg.py [options] set_default_version as documented at https://cloud.google.com/appengine/docs/python/tools/uploadinganapp -- i.e, use a two-line shell script or cmd file, doing the set_default_version right after the update. However, this won't work right if you've changed index.yaml since the index needs to be rebuilt, which takes time; in this case you must not set_default_version until the new index is ready (again, that uploadinganapp URL explains this well).
I upload my_app to app engine app by:
appcfg.py update c:\my_app ...
If I already uploaded for my_app then done a minor changement in file,
Does it upload whole project to app engine and overwrite whole previous project?
Or does it upload only relevent change and overwrite relevent part?
And what is the case for the issue for this command:
bulkloader.py --restore --filename=my_kind.dump ...
Did you try it?
update uploads the whole application each time. There's no concept of a delta. Normally, when you upload a new version I would suggest changing the version setting - that way you can keep up to 10 previous versions of your app on the site, and only set the new one to be the default once you are sure it is working.
If you upload without changing the version, AppEngine actually creates a new version before deleting the old one, so you need a spare slot in your versions list.
I don't understand your question about the bulkloader. Are you asking if that does a delta? No, it can't, because it sends the data serially via the remote API - there's no way for it to know in advance which rows in your data file already exist in the datastore.
i'm trying to upload my app engine project for the very first time and i have no clue why it is not working. the error from my terminal is:
[me][~/Desktop]$ appcfg.py update ProjectDir/
Application: tacticalagentz; version: 1
Host: appengine.google.com
Starting update of app: tacticalagentz, version: 1
Scanning files on local disk.
Error 404: --- begin server output ---
This application does not exist (app_id=u'tacticalagentz').
--- end server output ---
i'm using python 2.6.5 and ubuntu 10.04.
not sure if this is relevant, but i just created a google app engine account today. and i also just created the application today (like a couple of hours ago). this is really frustrating because i just want to upload what i have so far (as a demo). in my app.yaml this is my first line:
application: tacticalagentz
Furthermore, i checked on my admin console, and i CLEARLY see the app id right there, and it matches letter for letter with the app id in my app.yaml
could someone please enlighten me and tell me what i am doing wrong? or is it something beyond my comprehension (like indexing issue with Google that they need time to index my app id) ?
thank you very much in advance
apparently adding the "--no_cookies" parameter will work
appcfg.py update --no_cookies ProjectDir/
the way i was able to find my answer was by uploading my app from my Mac OS X (thank god i have linux mac and windows). AppEngine on Mac OS X comes with a GUI interface, and it worked for uploading. so then i found the command they used in the console, which included "--no_cookies". perhaps if you run into similar issues in the future, this is one approach to getting the answer
App Engine for Java have the same problem. The problem is about account login.
If you are using Eclipse, use Sign In button.
If u are using command-line, use "-e" option, like this:
appcfg.sh -e your#email.com update yoursite/
I had the same problem. When I changed the name of the app I used in the launcher to match the one in the app engine, It worked without any problem. The way I figured out, it was the name mismatch which caused the problem. You can see the name of your registered app in the admin console of app engine.(https://appengine.google.com/)
Here's what fixed it for me:
i had an instance of dev_appserver.py myProjDirectory/ on a different terminal.
i guess the scripts are somehow linked and aren't thread safe
An alternate option that worked for me is to just "Clear Deployment Credential" from the Control option of the GUI. When the app was deployed after this, it opened a google page to allow GAE to access the user profile and then deployment was successful.
The key bit is
This application does not exist (app_id=u'tacticalagentz').
which is telling you that appspot.com doesn't know of an application by that name. The admin console (https://appengine.google.com/) shows your applications. Check there. You might have made an inadvertent typo when you registered the app.