From the tutorial, which I confirmed by creating a simple project, the index.yaml file is auto-generated when a query is run. What I further observe is that until then the admin console (http://localhost:8080/_ah/admin/datastore) does not show the data-store.
My problem is this: I have a project for which data/entities are to be added manually through the datastore admin console. The website is only used to display/retrieve data, not to add data to the data-store.
How do I get my data-store to appear on the console so I can add data?
Yes, try retrieving from the empty data-store through the browser just so I can get the index.yaml to populate, etc. But that does not work.
The easiest way is probably just to create a small python script inside your project folder and create your entities in script. Assign it to a URL handler that you'll use once, then disable.
You can even do it from the python shell. It's very useful for debugging, but you'll need to set it up once.
http://alex.cloudware.it/2012/02/your-app-engine-app-in-python-shell.html
In order to do the same on production, use the remote_api:
https://developers.google.com/appengine/articles/remote_api
This is a very strange question.
The automatic creation of index.yaml only happens locally, and is simply to help you create that file and upload it to AppEngine. There is no automatic creation or update of that file once it's on the server: and as the documentation explains, no queries can be run unless the relevant index already exists in index.yaml.
Since you need indexes to run queries, you must create that file locally - either manually, or by running the relevant queries against your development datastore - then upload it along with your app.
However, this has nothing at all to do with whether the datastore viewer appears in the admin. Online, it will always show, but only entity kinds that actually have an instance in the store will be shown. The datastore viewer knows nothing about your models, it only knows about kinds that exist in the datastore.
On your development server you can use the interactive console to create/instantiate/save an entity, which should cause the entity class to appear in the datastore interface, like so:
from google.appengine.ext import ndb
class YourEntityModel(ndb.Model):
pass
YourEntityModel().put()
Related
I have a Google App Engine app, which connects to Google Cloud Storage.
I noticed that the amount of data stored was unreasonably high (4.01 GB, when it should be 100MB or so).
So, I looked at how much each bucket was storing, and I found that there was an automatically created bucket called us.artificats. that was taking up most of the space.
I looked inside, and all it has is one folder: containers/images/.
From what I've Googled, it seems like these images come from Google Cloud Build.
My question is, can I delete them without compromising my entire application?
I have solved this problem by applying a deletion rule. Here's how to do it:
Open the project in Google Cloud console
Open the storage management (search for "Storage" for example).
In the Browser tab, select the container us.artifacts....
Now, open the Lifecycle section. You should see something like:
Click on Add a rule and provide the following conditions:
In the action, select Delete object
In the conditions, select Age and enter for example 3 days
Click on create to confirm the creation
Now all objects older than 3 days will be automatically deleted. It might take a few minutes for this new rule to be applied by Google Cloud.
For those of you seeing this later on, I ended up deleting the folder, and everything was fine.
When I ran Google Cloud Build again, it added items back into the bucket, which I had to delete later on.
As #HarshitG mentioned, this can be set up to happen automatically via deletion rules in cloud storage. As for myself, I added a deletion step to my deployment GitHub action.
Here is the reference to the documentation: link
Built container images are stored in the app-engine folder in Container Registry. You can download these images to keep or run elsewhere. Once deployment is complete, App Engine no longer needs the container images. Note that they are not automatically deleted, so to avoid reaching your storage quota, you can safely delete any images you don't need. For more information about managing images in Container Registry, see the Container Registry documentation.
This can be automated by adding a Lifecycle rules like #HarshitG mentioned.
You can add a trigger to your lifecycle rules on console.cloud.google.com.
Access the bucket with artifacts (default is "us.artifacts.yourAppName.appspot.com")
Go to "Life cycle".
Click on "Add a rule".
Check "Object delete" and press "Continue".
Check the filter to delete the bucket, I chose "age" and selected three days as the number of auto delete old elements (after element has 3 days of life it's auto deleted).
Click on "Create" and the rule is working now, then you do not need to visit every day to clean the bucket.
Same issue. Thanks for the update, Caleb.
I'm having the same issue, but I don't have an App running; I just have:
Firebase Auth
Firestore
Firebase Functions
Cloud Storage
Not sure why I have 4GB stored in those containers, and I'm not sure if I should delete them or if that would break my functions.
UPDATE:
I deleted the container folder and all still works. Not sure if those are backups or whatnot, but I can't find anything online or in the docs. I will post here if something happens. As soon as a cloud function ran, the folder had 33 files again.
I recommend against setting up a lifecycle rules on your Storage buckets. It will likely lead to breaking subsequent updates to the function (as described in Cloud Function build error - failed to get OS from config file for image)
If you are interested in cleaning up container images, you should instead delete container images stored in Google Cloud Registry https://console.cloud.google.com/gcr. Deleting container images on GCR repos will automatically cleanup objects stored in your Cloud Storage.
https://issuetracker.google.com/issues/186832976 has relevant information from a Google Cloud Functions engineer.
I would like to accesss the sitecore DB and items from console application like
Sitecore.Data.Database db = Sitecore.Context.Database
or
Sitecore.Data.Database db = Sitecore.Data.Database.GetDatabase("master")
how do I configure and setup my console application to access the DB as above?
Thanks Everyone for the suggestion, I am really interested in config changes, I used webservice, but it has very limited methods. For example, if I would like create an Item with the template and insert the item with prepopulated value, there is no such option. The reason I am looking for the console apporach is I would like to import the contents from XML or excel sheet and push those to the sitecore tree, eventually use the scheduled task to run the console app periodically. I do not want to copy the entire web.config and app_config. If anyone has already done this, could you please post your steps and necessary config changes?
You have two options I think:
1) Import the Sitecore bits of a website's web.config into your console application's app.config, so that the Sitecore API "just works"
I'm sure I read a blog post about this, but I can't find the reference right now. (I will have another look) But I think the simple but long winded approach is to copy all of the <sitecore/> element and all the separate files it references. I'm fairly sure you can whittle this down to a subset of the config required for data access with a bit of thinking.
2) Don't use the Sitecore API directly, connect to a web service that exposes access to it remotely.
There are a few of these that already exist. Sitecore itself exposes one, Sitecore Rocks has one, and Hedgehog TDS has one too. And you can always write your own (since any web service running inside the Sitecore ASP.Net app can make database calls and report values back and forth - just remember to consider security if this web service might end up exposed externally for any reason)
John West links to some relevant stuff here:
http://www.sitecore.net/Learn/Blogs/Technical-Blogs/John-West-Sitecore-Blog/Posts/2013/09/Getting-Data-Out-of-the-Sitecore-ASPNET-CMS.aspx
-- Edited to add --
I've not found the blog post I remember. But I came across this SO thread:
Accessing Sitecore API from a CLI tool
which refers to this blog post:
http://www.experimentsincode.com/?p=232
which I think gives the info you'll need for option 1.
(And it reminds me that, of course, when you copy the config stuff you have to copy the Sitecore binaries into your app's folder as well)
I would just like to expand on #JermDavis' post and note that Sitecore isn't a big fan of being accessed when not in a web application. However, if you still want to do this, you will need to make sure that you have all of the necessary configuration settings from the web.config and App_Config of your site in your console application's app.config file.
Moreover, you will never be able to call Sitecore.Context in a console application, as the Sitecore Context sits on top of the HttpContext which means that it must be an application and have a valid request for you to use it. What you are looking for is something more along the lines of Sitecore.Configuration.Factory.GetDatabase("master").
Good luck and happy coding :)
This sounds like a job for the Sitecore Item Web API. I use the Sitecore Item Web API whenever I need to access Sitecore data from the master database outside the context of the Content Management server or outside of the context of the Sitecore application. The Web API definitely does not allow you to do everything that the standard Sitecore API does but it can act as a good base and I now extend upon the Web API instead of writing my own custom web services whenever possible.
Thanks to JemDavis's advise.
After I copied the configuration and made changes to config section to get rid of conflicts. I copied almost all of Sitrecore, analytics and lucene dlls, it worked great.
Only thing you have to remember is, copy the app_config folder to the same location where your dlls are.
Thanks again JemDavis....
I'm still new to Liferay and using Liferay 6.2
what i'm doing:
I am trying to add a document manually into my database using insert statement.
I inserted into dlfileentry, dlfileversion and AssertEntry.
Also, i created a folder with the valid name and file.
The issue:
upon entering the Documents and Media portlet, i can see the document name there but when i click on checkout, it will prompt a error saying that Documents and Media is temporarily unavailable. however i am still able to download the valid document.
Am i doing something wrong? Personally, i feel that i am missing one more table for the database but i'm not sure .
Thanks!
Yes, you're doing something wrong: You should never write to Liferay's database with SQL, as there might be more data required than what's directly visible to you. Obviously, you're running into exactly such an issue.
Liferay has an API which you can use locally, from within the same application server, or remotely as JSON or SOAP service. You should exclusively use this for write access to the database.
Alternatively, you might consider WebDAV access to your document repository as the way to add more documents to the document library.
Trying to view items in memcache in GAE.
https://code.google.com/p/googleappengine/issues/detail?id=7245
http://googleappengine.blogspot.com/2012/08/app-engine-171-released.html
These links say "Namespaces in the Memcache Viewer - The Admin Console now supports retrieving memcache values that are stored in a non-default namespace."
But I still couldn't the view item trying different combinations.
tried, ns.key, ns_key, key (where ns is namespace, key is key set in the program).
I use Development SDK 1.7.6, with GO runtime. What should I enter it in the search space as seen in the picture, or what is the best way?
This is not currently implemented in the development server (but is available in the production admin console as pointer by Jesse in the comment).
You should open a new feature request in dev_appserver2 projects.
If you feel like contributing it yourself, you could easily add support by patching:
memcache_viewer.html to add a new <input> field to capture the namespace.
memcache_viewer.py to add namespace arg to memcache.get call
I am setting cache properly as my project does grab the correct result from it. I want to view it in the dev console memcache viewer (http://localhost:8080/_ah/amdin/memcache)
But when I paste in the key, it says "no such key". The memcache statement I use is:
stuff=memcache.get(userid, namespace='book')
So when I use the userid (aka key) in the memcache viewer, it fails. Am I to somehow add the namespace? I tried that and that failed. I also tried the entity key, that was a fail. any ideas?!?
To the best of my knowledge you can't retrieve a namespaced memcache key using the dev console memcache viewer.
You have more than an option though:
a. Use the interactive console with a code like this:
from google.appengine.api import memcache
print memcache.get(youruserid, namespace='book')
b. Create an admin section (Webapp+memcache api) to your web application adding this feature