Currently I am making an app where I want to have all the dependancies as orderly as possible. With my app running Elixir on Google's Compute engine, I think, for my (SQL) database, to use the PaaS solution that Google Cloud itself provides.
This being the case, I went looking for an adaptor and unfortunately found none. However, under the impression that SQL is itself a standard, I wonder how hard it might be to use an existing adapter like Ecto.Adapters.Postgres or Ecto.Adapters.SQL with the login information of my platform.
Is this something that is possible, and furthermore what type of protocol is implied by using the predominant adapters? Is it simply a RESTful api or is it something a little more proprietary to postgres?
If you're using Google Cloud SQL (it's unclear from your question) then judging by this documentation and especially this passage:
"administer your relational MySQL databases on Google Cloud Platform"
then you should be able to use the MySQL adapter for Ecto. Unless I'm misunderstanding your question and if I am, then please add some detail to your question so we can give you a better answer.
If you've already researched this option then please provide more detail about what's not working for you.
Related
I am working on a project that we are going to put on Google Cloud.
There will be a member requirement so logins and profiles to store. Members will make projects that will be linked to their accounts. Other members can join these projects etc. Its not overly complex but I need it to be fast and scalable from the off.
I have a few (simple) questions about the best setup to go for.
Do I have a PHP front end if PHP is only in beta? Do I just use Python for the front end? Is there a better framework than others to use?
Do I create an App Engine API for the front end to call using Python or Java or something else?
Which database do I use? Do I go down the Compute Engine/MongoDB approach or just go straight for Google datastore? (MySQL is disregarded here)
Do I use a shared memcache or get a dedicated one?
These sort of things. It appears using Google Cloud is 'fairly' straight forward but would appreciate some pointers from those in the know who have already get their hands dirty, in a virtual sense of course!
Many thanks in advance
You appear to have four many-faceted Qs -- and apparently you aren't taking them to Google Groups so let me do my best here.
Do I have a PHP front end if PHP is only in beta? Do I just use Python
for the front end? Is there a better framework than others to use?
For guaranteed solidity use Python or Java - PHP and Go aren't quite as mature yet. Many Python frameworks are fine, from the very-lightweight webapp2 that comes with App Engine, through intermediate-weight ones such as "flask", all the way to rich "django". I'm personally a "frameworks shd stay out of my way!" guy so webapp2 is my own favorite.
Do I create an App Engine API for the front end to call using Python
or Java or something else?
Python and Java are both fully supported and stable. I personally of course prefer Python, but, hey!, that's just me! Endpoints, if that's what you mean by "an App Engine API", is also equally well supported each way, with Python perhaps a tad ahead in integration with the datastore thanks to https://github.com/GoogleCloudPlatform/endpoints-proto-datastore/tree/master/endpoints_proto_datastore .
Which database do I use? Do I go down the Compute Engine/MongoDB
approach or just go straight for Google datastore? (MySQL is
disregarded here)
I think the GAE datastore (with add-ons as needed, e.g to shunt images and videos off to Cloud Storage, or structured data for search including geo functionality to the Search API) is going to serve you fine.
Do I use a shared memcache or get a dedicated one?
Start with the shared (free) variety, then once you have it all working design and run stress load-tests and check how they perform with that vs a dedicated (paid) version. Do data-based decisions -- let the numbers guide you: how much better are you getting by paying $X/month for dedicated cache? Decide accordingly!-)
I have an application that is using JDBC to manipulate its data.
I'm looking for a way to run the application on Appengine. Unfortunately, it seems like there aren't many options besides datastore (which I just can't get used to).
I've tried to use an embedded JavaDB, but Appengine blocks FileOutputStream (throws an exception "java.io.FileOutputStream is a restricted class" on initial driver loading). Therefore, I haven't tried Sqlite.
I've also tried to use Jiql, a new project that offers a JDBC interface to Datastore. However, I just can't seem to establish a "connection". It is somewhat unclear to me how to use jiql (like: what user/pass do you supply?), despite the few examples on their site.
I've looked at Google Cloud SQL and eventually signed up for it. It seemed to be the perfect solution to our problem, until I realized it's a paid service (it'll be paid soon, once it leaves the beta phase). I prefer not to pay (yet) as I'm still testing and evaluating the potential of GAE.
--
What would you recommend, keeping in mind would like to use JDBC for the persistence layer of the application?
Thanks in advance!
GAE supports two frameworks for persistence, JPA and JDO. There are few frameworks that plays well with GAE rules like Objectify. But to start with datastore you can also take at google tutorial GAE DATASTORE. If you need to learn the relationship with JPA and support for JDBC persistence Check this Link
I have a traditional RDBMS based PHP app that I need to convert over to GAE and would like to properly learn how BigTable works prior to doing this. However, I'd kinda like to do it through sample problems or examples that show the maximal way to think about and utilize a non RDBMS platform such as BigTable...
It seems that this would be the best route to take prior to just jumping in and screwing some things up in a one-to-one conversion that would likely happen by the both feet in first method.
Anyone able to recommend a good starting path that perhaps helped you or something of this nature that will properly initiate someone with App Engine and BigTable?
A good way is to see the sources codes of a good projects running in GAE like jaikuengine and rietveld.
For articles, Google IO 2009 and 2010 and GAE articles offer a good resource.
Also you can learn a Column-oriented Database in Wikipedia and see all other projects like cassandra...
I would recommend having a play with the App Engine Cookbook to see how things work. It has some really good examples and has helped me a lot when trying to understand the DataStore
http://appengine-cookbook.appspot.com/cat/?id=ahJhcHBlbmdpbmUtY29va2Jvb2tyFwsSCENhdGVnb3J5IglEYXRhc3RvcmUM
I am thinking of making an (initially) small Web Application, which would eventually have a potential to grow. All things considered Google App Engine seems like a very attractive option. Say, user base and complexity grows and for one or other reason I needed to leave GAE behind. How difficult would it be to migrate away?
1) Does GAE provide a way to export the database? What format would it be? Would it be difficult to put it under MySQL (or similar)?
2) In which areas (ex. database access, others?) would I have to use GAE API? I.e. which parts of implementation would have to be abstracted away / interfaced?
Edit: 3) Alternatively, is it even worth to abstract away GAE API?
For question #1: I don't know if GAE specifically supports exports of a database but you can always roll your own, worst case scenario. If you are in a position where you need to, you'll probably have the resources to do it, too.
For question #2: You can and should always encapsulate those kinds of outside dependencies anyway. It doesn't matter whether or not they provide interfaces. Coupling to those interfaces should be kept to an absolute minimum.
For question #3: This question is not really super-clear so I cannot answer it.
I'm speaking strictly from a java webapp point of view...
Google App Engine for python has a backup/restore utility:
http://code.google.com/appengine/articles/gae_backup_and_restore.html
There is a huge interested in porting this to the java flavor.
You can use the higher level standard database apis (JDO/JPS) to allow you to move your app away from google's database services. I suggest purchasing the data nucleus tools in order to smooth the transition from big tables to something like mysql or oracle.
The packaged services GAE provides are enumerated at
http://code.google.com/appengine/docs/java/javadoc/
The stock JRE should handle porting of the urlfetch, mail, and memcache api packages.
You'll have to find a substitute technology for the users, blobstore, xmpp, and taskqueue packages.
I am trying to scrape some website and republish the data as a RSS feed. How hard is this to setup with Google App Engine? Disadvantages and Advantages using GAE. Any recommendations and guidelines greatly appreciated!
Google AppEngine offers much more functionality (and complexity) than you will need if truly all you will want to do is republish some structured data as RSS.
Personally, I would use something like Yahoo pipes for a task like this.
That being said... if you want/need to get your feet wet with GAE, go for it!
Working with Google App Engine is pretty straight forward. I would recommend going through the Getting Started guide. It's short and simple and touches on essential GAE topics. There are more pros and cons than I will list here.
Pros:
In general, App Engine is designed for high traffic web applications that need to scale. Furthermore, it is designed from a programmer's perspective. Much of the scalability issues (database optimization, server administration, etc) are dealt with by Google. Having said that, I find it to be a nice platform. It is still being actively developed by Google engineers, and scheduling of tasks (a feature that has been long requested) is in the current road map.
Cons:
Perhaps the biggest downside right now is again the lack of official scheduling support and the quota limits currently set for free accounts. However you can't complain much if its free. Currently it only supports Python as a programming interface (although a new language [Java I predict] is coming soon). Furthermore, Python 2.6 (and 3.0 for that matter) are not yet supported. In addition, Django 1.0 is not officially supported in App Engine (although you can package Django 1.0 with your application).
Harder than it would be in most other technologies.
GAE can sort of do scheduled batch stuff like this now, but it's really not intended for that type of thing. Pick pretty much any other language and platform for this particular task, and you'll make your life a lot easier.
I think BeautifulSoup could run on GAE, so all your scraping needs are handled :D
Also, GAE has a geturl thingy. The only problem I think you might have is not having enough time to get the data (30 secs limitation).
I am working on a same project and I've decided that it's easier to prepare the data on another server and push them to GAE.
You might also want to look into Yahoo! Query Language (YQL)