I want to use back4app as my database but i don't know if it has a persistent cache like CloudFirestore during Read operations? I plan on working with the Parse SDK (Rest Api) or are there some extra setup I need to do?
Related
I'm implementing a Django web service, which is about to have different platform apps,
Reactjs for computers, a swift app for ios, and Kotlin for android devices. the protocol is rest API and perhaps a chat feature included then Django channels are used as well. The data format is JSON. For deployment, I intend to use docker which includes Django, celery, and ReactJS app. And the database is on another separate server which is PostgreSQL. I was thinking to collect some user activity data and some history logs to show the user itself what she/he has done so far. After hours of searching, I came up with Kafka! unfortunately, I have no idea how can I use Kafka and integrate these stuff together and how can I deploy these things. I wish there was a system schema for this specific kind of system that shows what is what and where is what?
Kafka will only integrate your database and Django, with some effort, and ideally a separate Kafka Connect service.
From React (or other clients), you'll need to query some Django API routes which will then query your database. Kafka won't help with your frontend, and isn't really what is exposing the history/activity you're interested in displaying. In other words, you could simply write that to the database, and skip Kafka entirely.
Essentially, you're following the CQRS design pattern if you properly separate Kafka writes from end user / UI reads.
shows what's what and what's where!
Unclear what this means, but data lineage and metadata tools are a whole separate thing. For example, LinkedIn DataHub collects information such as this
I am learning web dev and I am planning on developing a simple web app using React: a decision matrix tool.
I need users to be able log in and save their matrices under their profile so I prefer to not use LocalStorage and thus learn more about databases. I am thinking of using JSON as the data format and I will also need to store basics user data for login and their profile.
I wonder how to tackle such a project since so far I have only been using GitHub Pages to host my static websites. Most of what I find by googling seems confusing or irrelevant for such a small-sized project.
My questions are:
What is the simplest way to store, access and edit JSON data as well as user data for a web app?
Are there any simple databases that can be “hosted” together with the app files on a server? Not sure if the question makes sense but I don’t understand where the database is.
What article or resource would you recommend to understand the concepts for data storage?
Ideally you would want to create a backend server using any language/framework e.g nodejs, java, django, php etc and expose the required data through APIs. But, if you don't want to create a separate backend app you can use Firebase database Firestore to save and fetch all of your required data. You can even use firebase hosting to deploy your app. Moreover, firebase also have their Authentication service which you can leverage for your app's authentication.
I'm building an offline-first app but didn't research on it until now. My current setup is the app uses Angular(1.x) and communicates to my server using NodeJS on a MongoDB Database. I'm using PassportJS for my authentication at the moment.
I'd like to migrate all my date to CouchDB and use PouchDB on my app.
How do I migrate from my current setup to PouchDB to CouchDB?
How can I authenticate my users after migration?
How do I migrate my current setup to CouchDB
Moving data
To export/import data from mongo to couch, you can simply follow these steps. Basically, you just dump your jsons and push them in Couch.
Structuring data
In CouchDB, there's no collections. Usually, to split your data into "collections", you simply add a special key to identify the collection. It can be type or collection for example.
Permissions
I'm not aware of the permissions system in MongoDB but basically, you can only define permissions at database level. So if you want some people to access certain documents, you can either use an application layer to handle permissions or you can split your documents with the per-user-pattern(One database per user and one global database without all public data).
Authentification
You can still use passportjs with CouchDB(see this example).
Also, you can use CouchDB authentification system which is builtin. Therefore, it has some limitations(eg: you can't expire someone's token and there's not builtin password recovery system).
Im building a simple angular application and there is a small administrator panel for updating the content (a .json document). I'm looking for a way to edit the json document from the administrator panel.
I can manipulate the memory-loaded json but I can't save it. Is there a way to put the json file in some kind of cloud database and connect to it without setting up a server or backend for my application?
I want my application to be easily deployable on any ftp so I can't setup a nodeserver or install something like couchdb.
Any ideas are appreciated.
You could use a provider like Parse. It's free (up to a limit of requests/month), has a nice JavaScript SDK that would get you up and running quickly. https://parse.com/
Also, check out this query builder to aid in retrieving your data from Parse. It's built as an Angular service for easy integration. https://github.com/dpollot/parse-query
EDIT
Parse also offers hosting, for free.
We have an application that we're deploying on GAE. I've been tasked with coming up with options for replicating the data that we're storing the the GAE data store to a system running in Amazon's cloud.
Ideally we could do this without having to transfer the entire data store on every sync. The replication does not need to be in anything close to real time, so something like a once or twice a day sync would work just fine.
Can anyone with some experience with GAE help me out here with what the options might be? So far I've come up with:
Use the Google provided bulkloader.py to export the data to CSV and somehow transfer the CSV to Amazon and process there
Create a Java app that runs on GAE, reads the data from the data store and sends the data to another Java app running on Amazon.
Do those options work? What would be the gotchas with those? What other options are there?
You could use a logic similar to what App Engine HRD migration or backup tool are doing:
Mark modified entities with a child entity marker
Run a MapperPipeline using App Engine mapreduce library iterating on those entity using a Datastore Input Reader
In your map function fetch the parent entity and serialize it to Google Storage using a File Output Writer and remove the marker
Ping the remote host to import those entity from the Google Storage url
As an alternative to 3 and 4, you could make multiple urlfetch(POST) to send each serialized entity to the remote host directly, but it is more fragile as an single failure could compromise the integrity of your data import.
You could look at the datastore admin source code for inspiration.