I'm a web developer and used to using FireBase for a lot of my projects. Much of the work is already done out-of-the box there regarding realtime synchronization etc. In my next project, however, I'm not allowed to use an external provider for storing data, and as there's no way to host Firebase on a local server, it's out of the question.
For that project, currently a Redis key-value store is used. I know Redis also supports a pub-sub model, and I'm searching for a good way to use this Redis key-value store in a realtime web application. I've found a Meteor library (https://github.com/meteor/redis-livedata), which looks promising. I'm used to using AngularJS, however. And the majority of the solutions I found only use Redis for the pub-sub model, and not for the datastore (maybe for good reason). I've found an alternative which uses Django as the datastore (https://github.com/mburst/django-realtime-tutorial). I've also come across Firehose (http://firehose.io), but this doesn't use the Redis key-value store either.
Does anyone know a good way to emulate the same functionality Firebase provides, using a Redis key-value store, preferably compatible with AngularJS?
Correction - Django is a python based web framework, not a datastore .
For your requirement , tools like deployd and loopback suit alot.
They are opensource versions of services like Firebase or Parsed.
Both of them work with javascript(nodejs) and hence would support AngularJS or any other framework you would want to use.
For datastore backend, most preferred pluggable backend is mongodb as it supports fast writes, rich querying, and persistence of data.
Related
Context
I'm developing a hybrid/mobile app using Cordova. This app displays content, such as news, magazine articles and weather info that is dynamically generated and delivered via a Web App (self-hosted developed with PHP). In order to decide which mobile devices "get" which content, I need to constantly keep track of the history of displays on each device, and gather information such as : article name, display date/time, location(lat/lon), etc.
The BIG problem I have is...
Now, because the data that I'm gathering is unstructured, I first tried with Firebase (Google), which allowed me to easily gather all the data, in real-time using a key:value schema and then retrieve my data on the webapp running another JS script. The problems with this solution are:
Querying possibilities are too limited.
I'm not comfortable leaving all my data handled by Google (Nothing against Google, butI'd prefer to own my data).
The Question
What are the storage strategies that I could use ? I've been reading a lot about Redis, Parse-server, PouchDB+CouchDB, and MongoDB, but I got lost in a world of information about NoSql databases, and now I really do not know what to do :'(
I've always used relational databases for my projects (Mysql, MariaDB), but I'm not afraid in getting my hands dirty with NoSQL databases, however I'd appreciate if anyone can point me to the easiest-to-implement solution.
EDIT (17/04/18)
I think I narrowed down my question to 2 options
PouchDB + CouchDB, or
Couchbase Mobile = Couchbase Lite + Sync Gateway
Can anyone, please give me some feedback with regards to these two options? Ease of implementation, Querying, maintenance, etc ?
Thanks!
Figured, I should probably explain my earlier comment better.
Couchbase Mobile = Couchbase Lite (runs on mobile device) + Sync Gateway + Couchbase Server (hosted in backend)
W.r.t your first concern
Querying possibilities are too limited.
Couchbase Server supports N1QL - which is a superset of SQL for NoSQL. So you should be able to do the kind of extensive querying that you want w/ Couchbase Server.This link discusses how you can do geospatial queries (since you mentioned that as a use case)
W.r.t your second concern
I'm not comfortable leaving all my data handled by Google (Nothing
against Google, butI'd prefer to own my data).
With Couchbase Mobile, you have control over where the data is hosted - you have the option of persisting the data in Couchbase servers that you can host in private/public cloud (AWS, Azure, GCP).
Note on Cordova support in Couchbase Lite:
It is supported as a community project in Couchbase Lite 1.4. However, in the latest 2.0 release, you will have to develop your Cordova plugin on top of the native API .
I am developing an app, which is in android.
In my app I want to use couchdb database for storing and sync purpose.
which will work offline too.
so, how to create couchDB database first and then insert data in that database, so that my application will use the couchDB database as local storage like SQLite.
I have aleready went through many links but they are not satisfying my need.
So Please provide some guide line or an help or comment or sample code to performed that operation.
CouchDb and Couchbase Lite are very close in terms of usage. If you are looking for an easy way to sync and store data locally on your android device I would recommend looking at Couchbase Lite for Android.
https://github.com/couchbase/couchbase-lite-android
Alrighty I have a cross platform mobile app built using xamarin and the mvvmcross framework. Part of the app is grabbing some data from a remote database. This data will be updated/deleted/created occasionally and i want to keep my users up-to-date. What's a good way to achieve this, preferably using cross platform code?
I can always set some kind of timer but I would much prefer a more elegant way.
Couchbase, as mentioned by Artur, is a great option - especially if doing disconnected editing. It does have a bit of a learning curve picking it up from scratch (which is totally worth it if you press on).
Consider Akavache (https://github.com/akavache/Akavache) as an alternative. It's well used and simpler but provides the basic capabilities you are asking about.
I recommend combining either approach with using ModernHttpClient (https://components.xamarin.com/view/modernhttpclient) which is basically just a drop-in component that gets you a performance boost.
Check out this excellent article by Rob Gibbens that covers using Akavache, ModernHttpClient, and other x-platform wholesome goodness: http://arteksoftware.com/resilient-network-services-with-xamarin/.
Check out Couchbase Mobile
http://www.couchbase.com/nosql-databases/couchbase-mobile
There is a wrapper for Xamarin
https://components.xamarin.com/view/couchbase-lite-net
With Couchbase you can save/update/synchronize data between your mobile device and cloud server using NoSQL DB.
In the context of Couchbase Mobile for your use-case where you are updating/deleting and creating your data occasionally, you can replicate the data based on per method invocation. This would happen once as oppose to replication that is on a continuous basis.
You would do this by defining the logic on when your PUSH and PULL replication would occur on a particular database endpoint.
---PUSH Replication would be from your mobile app/device to Couchbase Server(Cloud component)
The library, Couchbase Lite (app component), on your app/device would flag new data or changes on existing documents(database) and will PUSH replicate to the Sync Gateway (data orchestration component).
---PULL Replication happens from the Couchbase Server(Cloud component) to mobile device. The Sync Gateway (data orchestration component) has advanced listeners that are observing for changes in the Couchbase Server and upon remote changes will pull data to the Sync Gateway and then update your mobile app/device.
I also suggest taking a look at the mini-hack quick tutorial, specifically the Xamarin one on how to get a Xamarin app working with Couchbase Mobile. And the mobile Developer Portal is a another great resource too where there are Xamarin Android & iOS guides.
I am working on a webapp for a client that has a cPanel virtual server, and it appears that I can only use MySQL, but I want to store the data using a json-like structure, so that I can more easily use Angular.js on the frontend.
I've looked into installing a NoSQL database, and I can't find anything viable (if you know of a way to do that, that would be my best solution), so I'm thinking of storing the data as json strings in a series of text files on the server that I would write to with php.
I'd like to hear some opinions, and if there are any better solutions of which I'm not thinking of.
Go look at firebase and thank me afterwards.
In short, firebase is a cloud real-time JSON data storage. Everything for the backend is done for you and all you need to do is the front-end. Their servers are CDNs which means it will be great if you're looking to serve the entire world. All you need to do is configure your data-structure and use it!
It also provides sockets, which is great for real-time data (used for games, chat and etc).
There is a free option. The only downside is that it is a little expensive if you want to scale it, nevertheless if your app really gets to that stage - I'm sure you'll have money to hire some people to develop a similar backend for yourself.
Google app engine offer a data store (some kind of DB wrapper) to hold your data.
It does not supply an editor to this data store - only a viewer.
When developing a web application with other DB - MSSQL, MySql etc. - I change the DB structure in the development process many times.
In AE data store you should edit it's structure and data by using code - Java in my case.
Do you - AE developers - have any best practice to manage this DB updates and save them in some smart way for deployment?
I don't know about "best practice", but I have a Servlet that I use during development which can upload and download all entity data as JSON.
I can then use a regular text editor to make changes or I use a hacked version of JSONpad to edit data live in the system.
Since, I use JSON through out my application this works best for me. One could also do the sample thing with XML and use any one of the many XML editors.
Also, I do use the low-level API for all my applications, so my data models tends to be fairly simple.
There are plenty of JSON/XML editors that could be adapter for your purposes, with a little bit of work.