I'm building an offline-first app but didn't research on it until now. My current setup is the app uses Angular(1.x) and communicates to my server using NodeJS on a MongoDB Database. I'm using PassportJS for my authentication at the moment.
I'd like to migrate all my date to CouchDB and use PouchDB on my app.
How do I migrate from my current setup to PouchDB to CouchDB?
How can I authenticate my users after migration?
How do I migrate my current setup to CouchDB
Moving data
To export/import data from mongo to couch, you can simply follow these steps. Basically, you just dump your jsons and push them in Couch.
Structuring data
In CouchDB, there's no collections. Usually, to split your data into "collections", you simply add a special key to identify the collection. It can be type or collection for example.
Permissions
I'm not aware of the permissions system in MongoDB but basically, you can only define permissions at database level. So if you want some people to access certain documents, you can either use an application layer to handle permissions or you can split your documents with the per-user-pattern(One database per user and one global database without all public data).
Authentification
You can still use passportjs with CouchDB(see this example).
Also, you can use CouchDB authentification system which is builtin. Therefore, it has some limitations(eg: you can't expire someone's token and there's not builtin password recovery system).
Related
I'm implementing a Django web service, which is about to have different platform apps,
Reactjs for computers, a swift app for ios, and Kotlin for android devices. the protocol is rest API and perhaps a chat feature included then Django channels are used as well. The data format is JSON. For deployment, I intend to use docker which includes Django, celery, and ReactJS app. And the database is on another separate server which is PostgreSQL. I was thinking to collect some user activity data and some history logs to show the user itself what she/he has done so far. After hours of searching, I came up with Kafka! unfortunately, I have no idea how can I use Kafka and integrate these stuff together and how can I deploy these things. I wish there was a system schema for this specific kind of system that shows what is what and where is what?
Kafka will only integrate your database and Django, with some effort, and ideally a separate Kafka Connect service.
From React (or other clients), you'll need to query some Django API routes which will then query your database. Kafka won't help with your frontend, and isn't really what is exposing the history/activity you're interested in displaying. In other words, you could simply write that to the database, and skip Kafka entirely.
Essentially, you're following the CQRS design pattern if you properly separate Kafka writes from end user / UI reads.
shows what's what and what's where!
Unclear what this means, but data lineage and metadata tools are a whole separate thing. For example, LinkedIn DataHub collects information such as this
Well, probably this is a stupid question, but after to surfing the web the last week...this is my last shot.
Questions:
It is possible to create an API to call a SQL database to retrieve records?..In a secure way..?
What apps/tools/methods can I use? hopefully...VS, Python, PowerShell...etc
Can I encrypt that call?and create a method to decipher the data?
I'm not the app owner, what questions should I make to the app developer?
FYI Currently I cannot use cloud vendors or third party tools to do the transformation :(. But if there is a strong tool/vendor maybe I can negotiate it.
I have consulted:
API to database: API to Database?
How to quickly create a simple REST API for SQL Server database - https://medium.com/voobans-tech-stories/how-to-quickly-create-a-simple-rest-api-for-sql-server-database-7ddb595f751a
Any ideas?
Try rolling out redash with the bitnami stack. It's free, as in speech.
Bitnami will take care of setting up https etc if you do it on AWS. You need a t2-small I think. There's how-tos for let's encrypt etc also.
You can dish out links to "widgets" with an embedded API key. The widgets can be json or csv. They will only provide access to predefined data queries, but you can get that data from anywhere, not just your SQL server.
We are a SaaS product and we would like to be able have per-user data exports that will be used with various analytical (BI) tools like Tableau or PowerBI. Instead of just managing all those exports manually, we thought of using some cloud database such as AWS Redshift (which will be part of our service). But then, it is not clear how is user will access those databases naturally, unless we do some kind of SSO integration with AWS.
So - what is the best practice for exporting data for analytics use in SaaS products?
In this case you can build your security in to your backend API layer.
First you can set up processes to load your data to Redshift, then make sure that only your backend API server/cluster has access to redshift (e.g. through a vpc with no external ip access to redshift)
Now you have your data, you can validate your user as usual through your backend service, then when a user requests a download through the backend API, the backend can create a query to extract from redshift only the correct data based upon the users security role. In order to make this possible you may need to build some kind of security column into your redshift data model.
I am assuming getting data to redshift is not a problem.
What you are looking for, if I understand correctly is a OEM solutions.
The problem is how does one mimic the security model you have in place for your SaaS offering.
That depends on how complex is your security model.
If it is as simple as just authenticate the user and he has access to all tenant data or the data can be easily filtered for user. Things are simple for you. Trusted authentication will allow you to authenticate that user and user filtering will allow you to show him all that he has access to.
But here is the kicker, if your security is really complex , then it can become really difficult to mimic it within these products.
Here for integrating tableau this link will help:-
https://tableau.github.io/embedding-playbook/#
Power BI, this product am not a fan off. I tried to embed a view in one my applications and data refresh was a big issue.
Its almost like they want you to be a azure shop for real time reporting.( I like GCP more )
If you create the api's and populate datasets then they have crazy restrictions like 1MB/sec etc.
On the other instances datasets can be refreshed only 8 times.
I gave up on them.
Very recently I got a call from Sisense and they seemed promising as well from a OEM perspective. You might was to try them.
I have an offline app, that will use PouchDB and CouchDB to sync data with an SQL server. Several users will share the same device, so I though that each user needs his own PouchDB to store their data. Once they have an internet connection, they can synchronize data with the CouchDB, that will make the link with the SQL server.
An optional thing would be to add another PouchDB, which will be a database shared by all users to avoid data redundancy.
Is having several PouchDB on the same device with a single CouchDB a problem for synchronization? Is it a good pattern? Is adding a shared PouchDB a good thing?
You can have multiple PouchDB instance on the same device. You can sync filtered subset to separated PouchDB instances. Therefore, 1-1 sync is way faster then filtered replication.
& 3. It's a good pattern to store all user content into a single database. For sharable data, it's normally stored on a central database. Application layers can be added to control document ownership.
I create a CMS from scratch and decided to use CouchDB as my database solution. For my CMS I need various accounts and of course different user roles (admin, author, unregistered user, etc.).
First I thought I would program authorization within my CMS myself, but CouchDB has stuff like this build in, so I want to ask:
What is the best practice creating a multiuser app with CouchDB?
Create only one admin for CouchDB and manage restrictions, roles and accounts by yourself?
Use build-in functionality of CouchDB for all this? (Say create a CouchDB admin user for every admin of the CMS?)
What if I want to add other 3rd-party authorization later? Say I want users to login via Twitter/Facebook/Google?
Greetings,
Pipo
The critical question is whether you want to expose CouchDB to the public or not.
If you want to build your CMS as a classical 3-tier architecture where CouchDB is exclusively accessed from a privileged scripting layer, e.g. PHP, then I would recommend you to roll your own authorization system. This will give you better control over the authorization logic. Particularly, you can realize document based read access control (not available in the CouchDB security system).
If instead you want to expose CouchDB to the public, things are different. You cannot actually write server side logic (except for separate asynchronous listeners via the changes feed) so you will have to use CouchDB's built in authentication/authorization system. That limits you to read access controlled on a database level (not document level!). Write access can be controlled with validation functions. CouchDB admins should not be equivalent to application admins as a CouchDB admin is rather comparable to a server admin in a traditional setting. A database admin in CouchDB would be a better fit (can change design documents and therefore make modifications to the CMS installation like adding plugins). All other users with write access can be realized as database members.
I would prefer the second approach, because this will give you the possibility to leverage all the nice features of CouchDB like replication and the changes feed. However, you will have to do some filtered replication between databases with different members if you need fine grained read access control.
If you want to use other authentication mechanisms than those offered by CouchDB, you will eventually have to modify the installation (which can be an issue if you want to use a hosted CouchDB). For a facebook plugin see e.g. https://github.com/ocastalabs/CouchDB-Facebook-Authentication.