Does anyone have a good example of a sharded counter entity and how to use it in Objectify? The examples I have seen so far are in the native Java datastore api.
For example, suppose you want to keep a counter of how many views a specific page Entity gets (and there can be many page entities).
Just don't want to shoot myself in the foot =P
This library is an Objectify implementation of a Sharded Counter engine
https://github.com/instacount/appengine-counter
Related
I am exploring web development with Google AppEngine (Java). My application has a very basic data storage requirement that is well suited to AppEngine's 'map' like datastore.
The basic unit is one class that will have member variables that will be written or read from the database per transaction (this is because it interacts with an Android app).
I am considering using Objectify for interfacing.
My questions are : What happens if I later change the size (number of variables) in my base class ? I know AppEngine isn't typed but will Objectify cause any problems if some variables are available for some keys and not for others ?
This is addressed extensively in the manual:
http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Migrating_Schemas
The short answer is that you can add and remove fields at will. In addition, there are facilities for more sophisticated transformations of data.
If you decide to move from the Objectify framework to the Low-level API later, you won't have a problem. App engine's datastore is typed, but not with all the Java types. I don't know whether you'd be able to get JDO or JPA to work without reading and re-writing all your data, but I think you probably would.
Objectify 4's method of storing a Map is pretty nice - storing properties as something like "fieldname-mapkey"
I want to use the Google App Engine to store my data and then query/display/ edit it using Google Spreadsheets as the user interface, with multiple concurrent users having their own view of the data. The problem I have now is that if I put everyone's data on the same Google Spreadsheet that everyone accesses, we can't each do sorting / filtering at the same time.
Is there a way to do this, and is it a good idea to build a simple system this way? I'll eventually need to query a series of Google Word Processor documents as well.
Can someone point me in the right direction on this or suggest other options?
I would ask what the advantage of doing something like this is as opposed to say hosting your application on Google App Engine and building a javascript front end with grids to help sort/filter and view data.
Anyway to answer your questions, you can build your interface over Google Spreadsheets using Google App Scripts. This will allow you to do things like authenticate your user, query, update and display data. If you want to merely display data it turns out that Google Spreadsheets has some built-in functions to do that.
Regarding consistency you should read up on GAE's Datastore as well as its features like transactions. The datastore is not an RDBMS, but is an object database which stores objects against keys. Again something to consider if you are planning to do a lot of data analysis and computation (summations, aggregations).
Overall I would recommend doing a rough design of your system without fixing on particular technologies (like GAE, and Google Spreadsheets). Once you identify what your key goals are for your application, then you can figure out which technologies and resources would make the most sense within your budget.
I am learning how to use the Google App Engine. My question is on the meaning of words used to describe a BigTable database.
I have read google's paper on big table This paper describes the BigTable data model in terms of rows, cells, column families, columns and column keys. I felt the examples given in the paper gave me a good idea of how to start designing a BigTable database.
I then looked at the Google App Engine datastore documentation and API . This uses the terms entity, properties, key, entity groups and index to describe the data model.
My question is:
What is the relationship between the terms used in the paper and the terms used in the API?
The Datastore is built upon Megastore which is built upon an implementation of BigTable.
When developing for appengine you don't really need to concern yourself with the Megastore and BigTable concepts, as it's all out of your control under the hood and the datastore builds much more on top of them anyway. It would be like trying to model your MySQL data after learning how BerkleyDB is implemented.... interesting but ultimately not that useful for your application.
Read through the megastore paper, which will probably give you mostly what you are looking for, and also checkout some of the Google IO talks that bring up the High Replication datastore as these touch a little on what goes on inside.
I am approaching to Gwt + Gae world.
My essential need is to send over the Gwt-Rpc wire my Entity classes, without duplicating them into DTOs.
Objectify promise to do that pretty well.
It claims it will hide all the "Jdo complexity".
I never worked with Jpa or Jdo technologies.
Where's all the complexity?
I mean, can you provide me some simple examples about complex tasks in JDO, made trivial by Objectify?
Maybe relationships?
I think JDO/JPA are easy to play with on "Hello World" level. But it changes as soon as you need something more real such as composite keys, multiply relationships between entities and etc. JDO GAE implementation is quite complex and hard to grasp for beginners, partly due to unsupported features, workarounds and extensions. JDO is designed to work "everywhere", which means it is highly abstracted and very general in its nature. Great for portability, but it also means that it might not be a perfect match for a specific engine like GAE with its quite unique datastore. The Datanucleus/JDO/JPA jars are quite big (~2.3 mb in total), while Objectify's jar is pretty small. JDO/JPA might perform class path scanning at startup to find and register your entities, which could add to the load time. The time spent would be proportional to the number of classes in your project.
As per example I think in terms of amount of code JDO/JPA sample will appear simpler than lots of DAO classes for Objectify, but in general, maintenance of Objectify code will be easier for an engineer because you don't need to walk through minefield thinking what you can break in JDO :)
One example of JDO's complexity is seeing how many different states an entity can be in. As an example of how this can be overwhelming at first, scroll to the bottom of this page and look at that state diagram. Objectify does not need such a state diagram.
Another tricky part of JDO is all the 'magic' that happens behind the scenes, which sometimes made debugging difficult. Of course this is not actually magic, just bytecode rewriting, but that along is tricky enough.
Finally, JDO is a generic API. It is designed to work with object stores, SQL databases, and who knows what else. The connection between a certain JDO concept and what will actually be happening in the datastore is sometimes difficult to see. Objectify's API is closely aligned with the datastore, making it easier to understand what is going on.
We're going to develop a near-realtime event-driven application (backend and bunch of mobile clients).
I think Akka (http://akka.io) is more than suitable for this. However, my collegaue wants to use Google App Engine and its async features. I'm not convinced it's the best approach to take, I wonder if we can somehow meld those two things together. I can't find any solid contemporary info via Google.
The Channel API may be useful. However, the main limitation you may face using app engine are transactional writes to the datastore, because an entity group (parent entity and its children) can only support one to ten writes per second.
It is notable that the new Go support for app engine supports actor-style programming with goroutines. When datastore or other ops block then other goroutines run. It would be nice if someone could do this for scala and one of the actor variants. The new backend system allows this style to be used in a long running way I presume.
Unrelatedly, on the issue of writing to an entity group. I write a record that might be already there (same key_name), and now I'm wondering if I should read it first to check.