I am writing a web application that requires a database which will have entities like user, friends etc. Since Cloud SQL service is not free so i am looking for alternatives. Amazon RDS is one option, since they have a free tier which would suit my needs in the short term but before I get into it I would like to know more about blobstores.
Is it ideal to use blobstore to store such kind of information?
There are questions like:
how will the read/write latency be compared to a traditional db ?
if i start with blobstore and later i want to move to relational db, what are the problems that i could face ?
The most important of all is, if it is ideal to use blobstore in my scenario.
After looking at the documentation on google dev site I have found that blobstores are used to store large/medium files like images and videos.
You can't and shouldn't try to use the blobstore for structured data. That's what the datastore is for. Blobstore is for unstructured data such as files.
Related
I'm creating a blog section for a website with Amazon Web Services. I'm comparing database solutions, and I came across DynamoDB. I'd like to know if it'd be a good idea to use DynamoDB for storing a blog post of more than 1500 words (6KB approximately). Should I save the article as a file onto the S3 instead, and store its link on my DynamoDB database? What is the right way of implementation?
Thanks in advance
DynamoDB is a key-value, NoSQL database that delivers single-digit millisecond performance at scale. It is a fully managed durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications. More information here.
You can certainly use DynamoDB to build a blog application. You would need to model your data and depending upon the language that you use, you can use a DynamoDB mapper. For example, if you built you application by using the Spring Framework, you can use the Enchanced Client.
Assuming you did build with Spring Framework - you could build it very similar to this tutorial and by replacing the relational database with DynamoDB. Using DynamoDB as opposed to reading a file stored in Amazon S3 in my view is the better way to proceed here.
I need to store images and I have 2 options:
store the image into GAE datastore.
store the image somewhere (maybe also on Dropbox or another website) and store its link into GAE datastore.
What's the best practice when we need to store an image into DB, in the hypotesis that each image is bijectivelly linked to a specific element of the datastore?
I think it depends heavily on the use case.
I have a small company website running on appengine and the content images are all stored in the datastore and for that application it works well (they are all relatively small images).
If you have a high traffic site you may find storing them in GCS, or some other mechanism that supports a more cost effective CDN will be more appropriate.
If the images are large (more than 1MB) then the datastore isn't a practical solution.
There will be no hard and fast rule. Understand your use cases, your cost structure, how complex the solution will be to manage, and then choose the most appropriate solution.
Neither of the above. Google's cloud platform includes a service specifically for storing files, Google Cloud Storage, which is well integrated into GAE. You should use that.
Background:
I am new to cloud computing and large scale DB design. I have to find a storage facility for a large number of images that have a lot of metadata associated with each image. I am going to use Amazon S3 to store my image files and I need a cloud based database solution to store metadata and reference to each image. I need this so I can query a DB for customer request and pull images and their metadata and insert new data as well via some web and mobile application interface I will create.
Research done:
I found the S3 is a raw data storage solution. I found many good discussions here on bucket naming conventions and I see many people use S3 as binary storage and use a DB for metadata. I've done some research on mongoDB, dynamoDB, and other database solutions.
Question:
I need a direction of where I can find an inexpensive and reliable Database that will work well with Amazon S3, that is ideal for large amount of metadata storage.
Well if you are not looking for a relational DB, why not try http://aws.amazon.com/simpledb/
and if you want RDMS how about http://aws.amazon.com/rds/
I've been reading more about Google AppEngine and learned python in the past couple of weeks, including working with MongoDB. What I need the most is a scalable database solution. Before discovering Google AppEngine, the only three DB solutions I find useful are DynamoDB, MongoDb and BigCouch.
I find out how that I really like python language, and for one coming from ASP.NET development, I've decided to switch and develop my app using python. My first choice was to develop my application using python + bottle + mongoDB. The problem is that DynamoDB is very expensive, and the lack of easy to use backup/restore options made me pass Amazon's offering.
Google AppEngine datastore is much more affordable. However, I still can't find information regarding some specific question on Google's website
Here are some of the questions I need answer to:
Does Google Datastore support backup/restore within the administration console?
If I want to backup/restore 50TB of data, how much time it takes to backup/restore the data? Where it is stored? what are the costs?
How much time it takes to backup 1TB of data for example?
Does DataStore support caching in the database layer
Any cons that I should be aware of?
Those some of the question that I need to get answers to. MongoDB is an excellent product and developing web app using Mongo + Python + bottle is fun fun fun. However, I prefer a full DB hosted solution like one offered by Google. But before I do that, I need to be sure that I'm not missing anything.
Here are some of the questions I need answer to:
Does Google Datastore support backup/restore within the administration
console?
No. Yes. You can back up and restore data from within the Administration Console by enabling datastore_admin for an application (Thanks to Idan Shechter for pointing this out!) More info can be found here: https://developers.google.com/appengine/docs/adminconsole/datastoreadmin
You can also download the data through the command line. See: https://developers.google.com/appengine/docs/python/tools/uploadingdata
If I want to backup/restore 50TB of data, how much time it takes to backup/restore the data?
It depends on where you back the data up to. Backing up to the Blobstore or Google Cloud storage will probably take much less time than backing up to your local machine. Transferring 50TBs to your local machine will take a long time and depend on many factors including network speed.
Where it is stored?
If you use the Datastore Administration, you can backup to the Blobstore or to Google Cloud Storage. If you use the command line tools, it will be stored where you choose to download the data to.
what are the costs?
The Blobstore costs $0.13/GB/Month and gives you 5GB free. Google Cloud Storage is $0.12 per GB/Month up to the first TB. You can see more pricing info for Cloud Storage here:
https://developers.google.com/storage/docs/pricingandterms
Bandwidth costs are $0.12 per GB (The first GB is free). More details on pricing can be seen here:
https://cloud.google.com/pricing/
How much time it takes to backup 1TB of data for example?
Again, it depends on where you back up to and your transfer speeds.
Does DataStore support caching in the database layer Any cons that I should be aware of?
No, it does not support database layer caching.
I'm working on a Silverlight app that would allow a user to upload a few gigs of files to a hypothetical cloud based file store, then allow the user to view some data about those files later (more functionality than a file store). Ideally I'd like to use a free, per-user store such as SkyDrive but I can't seem to find an API for that service (and read elsewhere on stack overflow that programmatic access violates their TOS). Do any services fit this bill? I've heard of Amazon S3 but I understand that'll cost some money - is anything free?
EDIT: Could Mesh be an option?
What is LiveMesh Object and its connection with Silverlight 3.0
You could look at using Azure as it offers a blob and table storage cloud infrastrucutre and will happily run silverlight applications in an azure web role. Currently there is no cost but this will change once it RTW's.
More info at http://www.azure.com/
AFAIK, nothing in this world is free when you're dealing with gigabytes of storage, plus the bandwith to put them in the cloud.
Amazon S3 is quite reasonable on its pricing.