I have a general Database Design question: When is it better to create a new Database instead of adding new Tables to an existing? The question is related to design/maintainability andalso performance issues.
Background: we are mirroring the main tables of our customer by importing the data every night into our DB called RM2. Some new (ASP.Net-)Projects need access to this data too, now i'm wondering if i should create new databases for each project or merge their tables with the RM2(current size: 37991.94 MB).
I won't necessarily answer your question, but I'll give you a bunch of other questions to consider as well:
When should I add files to a filegroup in my database? - When files get too big, where "too big" may be a matter of opinion.
When should I add a new filegroup to my database? - When you want to be able to optimize disk usage for different database operations.
When should I add a new schema to my database? - When you have a set of objects that are logically related and may require different default permissions for users.
When should I add a new database to my application? - When you don't need any referential integrity between any of the tables in the two databases. When you don't want to allow any ownership-chained permissions to cross between two sets of objects. When you want to independently backup and restore. When you want different SQL Server recovery models for two sets of data.
I guess that may have answered your question some. ;-)
This isn't a database design question. This is an organisational question. The organisational aspects of this question are much more important than technical questions.
The answer is: whatever makes life easiest for you as developers
For instance, you say:
Some new (ASP.Net-)Projects need access to this data too.
How integrated are these projects with your project? Do you actually share data (or write to the same tables)? For instance, if you make a breaking change to one of your tables, do you need to make changes to the code in the other projects at the same time? (Sometimes really hard to synchronize between two projects).
If you don't actually share data (apart from the customer data, which I assume is effectively read-only), then use separate databases (OR schemas). This makes changes a lot easier to manage.
Another trick is to have in each database a set of views onto the customer data, which lives in elsewhere, in another schema.
So, have a database per project, with views in each database onto the customer data, which lives in a single separate database.
Performance shouldn't really be an issue, unless the databases live on separate machines.
You can get other database for projects if any project want use only self database. But if you want get datacenter of this you should create for it a other shared database + self database for each project.
Related
first of all, I don't know if this is the right platform for this question. I hope it is. This is basically an architectural issue or more specifically a database design issue.
My company has asked me to create a service based website where individual subscribers can log in to their own customizable retail store. One fundamental question related to this requirement is designing the database. As I can understand there are two major approaches
Create a separate database based on a template for each subscriber / client.
Have a single database for all clients and link the tables based on primary key fields.
If any one has experience with the above scenario or can provide any useful insights, please do let me know.
Regards
Romi
separate database
You can put them easy to a other dedicated server
You have to administrate 1,000 databases for 1,000 clients
Your application need to figure out which database have to be used
one database with relationships
You have to administrate only one database
Less additional complexity in your application (tons of configurations etc.)
You can easy JOIN tables over all clients. For statistics or what ever.
We are working on an application in CakePHP that will be used by multiple companies. We want to ensure performance, scalability, code manageability and security for our application.
Our current beta version creates a new database for each customer. When a new company joins the site we run a SQL script to create a blank database.
This has the following advantages:
- Better security (companies users are separated from each other)
- We can set the database via the subdomain (IE: monkey.site.com, uses the site_monkey database)
- Single code base.
- Performance for SQL queries is generally quite good as data is split across smaller databases.
Now unfortunately this has many disadvantages
- Manageability: changes to database have to happen across all existing databases
- The SQL script method of creation is clunky and not as reliable as we would like
- We want to allow users to login from the home page (EG. www.site.com) but we cant currently do this as the subdomain determines what database to use.
- We would like a central place to keep metrics/customer usage.
So we are torn/undecided as to what is the best solution to our database structure for our application.
Currently we see three options:
- Keep multiple database design
- Merge all companies into one DB and identify each by a 'companyId'
- Some kind of split model, where certain tables are in a 'core database' and others are in a customer specific database.
Can you guys offer some of your precious advise on how you think we should best do this?
Any feedback / info would be greatly appreciated.
Many thanks,
kSeudo
Just my suggestion:
I think better you keep the customer related data in different databases and authentication related data in a common database So when a user logs in you should have an entry with domain that user belongs to and redirect to that domain and access the corresponding database and data.
Again your concern of changes to the database, You need to implement the changes in each databases separately. I think there is some advantages to this also. Some customers may ask for few changes according to their process. So this can be easily managed if you are keeping separate databases for different customers.
I'm creating a brand new database with no legacy constraints, so I'm curious as to what the schema best practices are.
The database will be called "SecurityData". It stores information about bonds.
The schema I have already identified are:
import - Views and procs that are really linked server calls to other databases
export - Views and procs meant to be used by other databases
staging - Tables used for bulk inserts so we can verify and scrub the data.
??? - The real tables containing useful data
history - Change logs for the real tables
Questions:
Am I going schema crazy or does this make sense?
Should I use dbo for my "real tables" or should I avoid that schema as it tends to become a garbage dump?
Schemas serve a dual purpose:
security containers. Grants/deny/revokes on a schema apply to all objects in the schema. Separating related security objects into a shcema allows for easy maintenance and control of access.
namespaces. Qualifying object names with schemas allows reduced conflict probability with names used by other applications and even other modules within your own application.
So my question to you is: why do you want to use schemas in the first place? I'm not saying you shouldn't, but i want to understand which advantage of the schemas are you most appealed to. If you know the answer to that, then you'll know how many schemas you need and what those schemas are. Of course, the answer can be a mixture of the two reasons I give at start, that is OK. In that case you may find that what makes sense from a namespace point of view is a disaster from security point or view or vice-versa.
I myself I used separate schemas just like you plan to, and soley for programming namespace benefits. during development it helped me to see, just from the name of an object, where to it belongs logically in the app.
I have been googling a lot and I couldn't find if this even exists or I'm asking for some magic =P
Ok, so here's the deal.
I need to have a way to create a "master-structured" database which will only contain the schemas, structures, tables, store procedures, udfs, etc, everything but real data in SQL SERVER 2005 (if this is available in 2008 let me know, I could try to convince my client to pay for it =P)
Then I want to have several "children" of that master db which implement those schemas, tables, etc but each one has different data.
So when I need to create a new stored procedure or something like that, I just create it on the master database (and of course it's available on its children).
Actually I have several different databases with the same schema and different data. But the problem is to maintain congruency between them. Everytime I create a script to create some SP or add some index or whatever, I have to execute it in every database, and sometimes I could miss one =P
So let's say you have a UNIVERSE (would be the master db) and the universe has SPACES (each one represented by a child db). So the application I'm working on needs to dynamically "clone" SPACES. To do that, we have to create a new database. Nowadays I'm creating a backup of the db being cloned, restoring it as a new one and truncate the tables.
I want to be able to create a new "child" of the "master" db, which will maintain the schemas and everything, but will start with empty data.
Hope it's clear... My english is not perfect, sorry about that =P
Thanks to all!
What you really need is to version-control your database schema.
See do-you-source-control-your-databases
If you use SQL Server, I would recommend dbGhost - not expensive and does a great job of:
synchronizing 2 databases
diff-ing 2 databases
creating a database from a set of scripts (I would recommend this version).
batch support, so that you can upgrade all your databases using a single batch
You can use this infrastructure for both:
rolling development versions to test, integration and production systems
rolling your 'updated' system to multiple production deployments (especially in a hosted environment)
I would write my changes as a sql file and use OSQL or SQLCMD via a batchfile to ensure that I repeatedly executed on all the databases without thinking about it.
As an alternative I would use the VisualStudio Database Pro tools or RedGate SQL compare tools to compare and propogate the changes.
There are kludges, but the mainstream way to handle this is still to use Source Code Control (with all its other attendant benefits.) And SQL Server is increasingly SCC friendly.
Also, for many (most robust) sites it's a per-server issue as much as a per-database issue.
You can put things in master like SPs and call them from anywhere. As far as other objects like tables, you can put them in model and new databases will get them when you create a new database.
However, in order to get new tables to simply pop up in the child databases after being added to the parent, nothing.
It would be possible to create something to look through the databases and script them from a template database, and there are also commercial tools which can help discover differences between databases. You could also have a DDL trigger in the "master" database which went out and did this when you created a new table.
If you kept a nice SPACES template, you could script it out (without data) and create the new database - so there would be no need to TRUNCATE. You can script it out from SQL or an external tool.
Little trivia here. The mssqlsystemresource database works as you describe: is defined once and 'appears' in every database as the special sys schema. Unfortunately the special 'magic' needed to get this working is not available to the user databases. You'll have to use deployment techniques to keep your schema in synk. That is, apply the changes to every database as the other answers already suggested.
In theory, you could put a trigger on your UNIVERSE.sysobjects table (assuming SQL Server), and then you could enumerate master.dbo.sysdatabases to find all the child databases. If you have a special table that indicates it's a child database, you can reference child.dbo.sysobjects to find it.
Make no mistake, it would be difficult to implement. But it's one way you could do it.
Can anyone tell me if there are RDBMSs that allow me to create a separate database for every user so that there is full separation of users' data?
Are there any?
I know I can add UID to every table but this solution has its own problems (for example per user database schema changes are impossible).
Doesnt MySQL, PostgreSQL, Oracle and so on and so on allow you to do that?. There's the grant statements to control ACLs
I would imagine most (all?) databases allow you to create a user which you could then grant database level access to? SQL server certainly does.
Another simple solution if you don't need the databases to be massive or scalable, say for teaching SQL to students or having many testers work against their own database to isolate problems is SQLite, that way the whole database is a single file (per user), and each user cannot possibly screw up or interfere with other users.
They can even mail you the databases, or install them anywhere, say at home and at work with no internet required.
MS SQLServer2005 is one which can be used for multiple users.An instance can be created
if you have any, run the previlegs and use one user per instance
Oracle lets you create a separate schema (set of tables, indexes, functions, etc) for individual users. This is good if they should have separate different tables. Creating a new user could be a very expensive operation as you would be making new tables. Updating is a nightmare as well, as you need to update the model for each user.
If you want everyone to have the same set of tables, but only able to view their own records then you could use Fine Grain Access Control or Virtual Private Database features to do this.