I need to create a big database where I've a lot of user end every user can store a million of row in a DB, My question is, I can make (In rails) one system where every user have a own DB ? for example one general DB where i can store the user and a lot of db where the user can store the data, or isn't good for the speed of my rails app ?
You can use Postgre Schemas for each user.
http://www.postgresql.org/docs/9.0/static/ddl-schemas.html
It's common used for multi-tenancy applications where you have an app created to server a lot of "accounts" where those accounts have a lot of users.
Pretty much they'll have the same tables but the data are separated by schemas.
I'm not sure but in the future you can scale and balance the schemas in different servers.
Here's some approaches on this gem: https://github.com/influitive/apartment
Related
This is the scenario:
I am developing a clinic management sofwate in vue, nodejs and mongodb, which each account (each client) may have their own data with their own users and their own patients in their page after they login. Shall I use a separate DB for each client or store everything in one DB and query all data from all the clients in every request ? I searched a lot but couldn't find a clue about which way to go .
In my experience, yes, separated DB(s) per customer is the best solution in the long run.
In your main administrative DB you can map customers to DB-servers giving you a lot of flexibility, for example to give more resources to big customers without impacting small ones.
Also consider using storage.directoryPerDB:true config option to reclaim disk space after deleting a DB (trial expired customer)
I've built a winforms application that i'm currently rebuilding into an ASP.NET MVC application using Web API etc. Maybe an app will be added later on.
Assume that I will provide these applications to a few customers.
My applications are made for customer accounting.
So all of my customers wil manage their customers whithin the applications I provide.
That brings me to my question. Should I work with one big database for al my customers, or should I use seperate database for each of my customers? I'd like to ask the same for web app instances, api's etc.
Technicaly I think both options are possible. If it's just a mather of preference, all input is appreciated.
Some pros and cons I could think off:
One database:
Easy to setup/maintain
Install one update for all of my customers
No possibility to restore db for one customer
Not flexible in terms of resource spreading
Performance, this db can get realy large
Multiple databases:
Preformance, databases are smaller sized and can be spread by multiple servers
Easy to restore data if customer made a 'huge mistake'
The ability to provide customer specific needs (not needed atm)
Harder to setup/maintain, every instance needs to be updated seperately.
A kind of gateway/routing thing is needed to route users to the right datbase/app
I would like to know how the 'big companies' approach this.
You seem to be talking about database multi-tenancy, and you are right about the pros and cons.
The answer to this depends a lot on the kind of application you are building and the kind of customers it will have.
I would go with multi-tenant (single DB multiple tenants) database if
Your application is a multi-tenant application.
Your users do not need to store their own data backups.
Your DB schema will not change for each customer (this is implied in multi-tenant applications anyway).
Your tenants/customers will not have a huge amount of individual data.
Your customers don't have government imposed data isolation laws they need to comply with (EU data in EU, US data in US etc.).
And for individual databases pretty much the inverse of all those points.
We are developing a new version of our web application.
We have multiple clients (500+), each client has its own database with its own data: users, products...
In the new version, all clients are going to share some data, for example, users are going to be in the platform but each client will be able to access to their users only, but instead of having the users for each client we want to have all the users in a centralize table.
Other things such as products, orders...are going to belong to each client.
Each client will have a copy of the web app installed in their domain.
Our app is an ASP MVC Entity Framework Code First, using SQL Server.
Our question is:
Option A: One database per client containing their tables (products, orders...) and one common database to store the users and other common data.
Option B: One big database containing all and add a ClientId to certain tables so the clients only see their data.
PROS AND CONS:
With Option A we have several databases, we can have 100.000 orders in a table and it is easy to retrieve that data. On the other hand we have to deal with cross database queries and having 2 Data Context. This is the prolem, beacuse we need to retrieve user data for most of the queries, that means access to both databases, the client specific and the common one.
With option B we just have to deal with 1 context and the queries are much more simple. The main concern for this approach is we could have some tables with more than 10.000 records per year, per client. So in 10 years, with 500 clients, we could have a table with 50 millions records and this could affect performance.
Thanks for your advices.
EDIT
The thing here is not a question abou single vs multiple database because we have one more thing in the game, all clients need to access a common database.
EDIT 2
Let's say we have decided to go for a single database for all our clients. So we will have multiple domains, each one with our application running, but we need each of them getting only their data.
How can we do this? Adding a ClientId to each table and filtering the data with a parameter "clientId" in the web.config of each site?
My personal preference would be for option A, and the primary reason would be for security. You basically have only a couple of points of failure for leaking one client's data to another.
You could look to put a service on top of the common data and cache frequent requests for user data to handle that side of things.
Option A would be the recommended approach as that will allow all different clients to query on their own selected transactional records and without any performance issues because of the requests from other clients.
Also, the option A would allow the entities based customization (if required in future) which would prove to be a challenge with option B. Multi-tenant based architecture is the recommendation.
The below mentioned resource can help you with some more options/possibilities.
https://msdn.microsoft.com/en-us/library/aa479086.aspx
I was doing some research into the optimal DB setup for a Salesforce like application (webapp with sensitive data for multiple 10.000's of customers, each customer can have multiple USER accounts).
My idea is, is that the split database setup (different DB for each customer) is the best since its the only way to make sure data is really isolated in terms of security, and because its not just multi customer, but each customer can have multiple users that login to work on the data. However...is having a DB instance with 10.000 'small' DB's really gonna work?
I dont really feel much for the shared schema setup since you will end up with 10's of thousands of tables, which doesn't seem very manageable.
The 1 big DB setup with just using a customerID column per table seems the easiest to implement, but not really secure...and joining tables will be quite a task as I presume.
What are your thoughts on this scenario? The seperate DB's-scenario seems best to me from a scalability and security perspective...but what do I know :D
Thanks!
IMHO, you can opt to use a different database for each tenant or for a set of tenants. That will give better scalability, good data access speeds, security compliance etc.. In case you have to do a tenant data back up, it is faster and can be automated.
The one consideration would be in accessing the data from both the shared and the shared database..
Even each tenant can opt to have a separate database for maintaining the user info so that the basic tenant authentication will come from the shared db and further the separate db can be used to authenticate the users.
Please share your thoughts on this.
If I am building a CRM web application to sell as a membership service, what is the best method to design and deploy the database?
Do I have 1 database that houses 100s of records per table or deploy multiple databases for different clients?
Is it really an issue to use a single database since I believe sites like Flickr use them?
Multiple clients is called "multi-tenant". See for example this article "Multi-Tenant Data Architecture" from Microsoft.
In a situation like a CRM system, you will probably need to have separate instances of your database for each customer.
I say this because if you'd like larger clients, most companies have security policies in place regarding customer data. If you store their customer data in the same database as another customer, you're running the risk of exposing one companies confidential data to another company (a competitor, etc.).
Sites like Flickr don't have to worry about this as much since the majority of us out on the Interwebs don't have such strict policies regarding our personal data.
Long term it is easiest to maintain one database with multiple clients' data in it. Think about deployment, backup, etc. However, this doesn't keep you from having several instances of this database, each containing a subset of the full client dataset. I'd recommend to grow the number of databases after you have established the usefulness/desirability of your product. Having complex infrastructure is not necessary if you have no traffic....
So, I'd just put a client id in the relevant tables and smile when client 4 comes in and the extent of your new deployment is one insert statement.