If I add 3-400 databases to a single SQL Server instance will I encounter scaling issues introduced by the large number of databases?
This is one of those questions best answered by: Why are you trying to do this in the first place? What is the concurrency against those databases? Are you generating databases when you could have normalized tables to do the same functionality?
That said, yes MSSQL 2005 will handle that level of database per installation. It will more or less be what you are doing with the databases which will seriously impede your performance (incoming connections, CPU usage, etc.)
According to Joel Spolsky in the SO podcast # 11 you will in any version up to 2005, however this is supposedly fixed in SQL Server 2005.
You can see the transcript from the podcast here.
I have never tried this in 2005. But a company I used to work for tried this on 7.0 and it failed miserably. With 2000 things got a lot better but querying across databases was still painfully slow and took too many system resources. I can only imagine things improved again in 2005.
Are you querying across the databases or just hosting them on the same server? If you are querying across the databases, I think you need to take another look at your data architecture and find other ways to separate the data. If it's just a hosting issue, you can always try it out and move off databases to other servers as capacity is reached.
Sorry, I don't have a definite answer here.
Related
One of our customer is on SQL express 2012 R2 for last couple of years and their database has grown to 1 GB. At any given time they have around 15 or more workstations connect to this database. They have a dedicated 2008 server for this database. Sometimes I could see some issues with the slow response but most of the time it is just ok. I cannot tell if suggesting standard SQL would improve the performance? Or it would be a waste of money? Can anybody suggest the parameters to check before I can make this decision?
In the task manager there are 2 sqlservr.exe processes and both of them are using 0% CPU but one of the process is using 2.2 GB of memory and the other is using 68 MB of memory.
Am I already pushing the envelope too far?
Please advise.
This cannot be answered without knowing how the system is developed. The vast majority of slow issues I have run across in many years of datbase work have to do with inefficient code or missing indexes. Getting a higher version of the database won't fix either fo those two issues.
Your problem could also be caused by physical equipment that is reaching it's limit or by network issues.
You are not close the data storage capacity of SQL server express, so I would investigate other things first as SQL Server Standard edition is quite a bit more expensive.
Your best bet would be to get a good book on performance tuning and read it. There are hundreds of things that can cause slowness.
We are getting ready to setup SQL Server Transactional replication. We will have hundreds of databases on each server being replicated to an alternative location (for reporting purposes). Are there any best practices, gotchas, things to look out for, etc. when having such large number of databases on a publisher? We do plan to have a dedicated "distributor" box and have separate distribution database for each of the publishing servers. We expect to have under 10 publishers, but total number of databases among them might approach a few thousand.
Thank you in advance for any pointers!
as suggested ask to dba.stackexchange.com it's better
From my experience, first of all (cause i had problem) try to know the kb/mb which will be transferred by second and following that check your connection between distributor and subscriber.
Then calculate very much the time it should cost for your publishers' server the time it will use to transfert it to the distributor.
And finaly identify very well which tables you want to replicate, try to forget nothing, it's a relational database so try to keep it relational :)
Thing about security too, if you are on different location you are probably not on same domain, the easiest way i found was to change the host's file from windows to use a logical name and not an IP (required)
After that, your problem will start.... for me replication with different locations was a lots of problem (living in europe and working for companies with bad vpn connection speed).
Good luck :)
I have 2 databases that have the same structure, one on a local machine and one on the company's server. Every determined amount of time, the data from the local DB should be synchronized to the server DB.
I have a general idea on how to do this - create a script that somehow "merges" the information that is not on the server DB, then make this script run as a scheduled job for the server. However, my problem lies in the fact that I am not very well experienced with this.
Does SQL Server Management Studio provide an easy way to do this (some kind of wizard) and generates this kind of script? Is this something I'll have to build from scratch?
I've done some basic google searches and came across the term 'Replication' but I don't fully understand it. I would rather hear some input from people who have actually done this or who are good with explaining this kind of stuff.
Thanks.
Replication sounds like a good option for this, but there would be some overhead (not technical overhead, but the knowledge need to support it).
Another SQL Server option is SSIS. SSIS provides graphical tools to design what you're trying to do. The SSIS package can also run SQL statements, if appropriate. An SSIS package can be started, and therefore scheduled, from a SQL Server job.
You should consider the complexity of the synchronization rules when choosing your solution. For example, would it be difficult to resolve conflicts, such as a duplicate key, when merging the data. A SQL script may be easy to create if the rules are simple. But, complex conflict rules may be more difficult to implement in a script (or, replication).
SQL Server Management Studio unfortunately doesn't offer much in this way.
You should have a serious look at some of the excellent commercial offerings out there:
Red Gate Software's SQL Compare and SQL Data Compare - excellent tools, highly recommended! You can even compare a live database against a backup from another database and synchronize the data - pretty nifty!
ApexSQL's SQL Diff and SQL Data Diff
They all cost money - but if you're serious about it, and you use them in your daily routine, they're paid for in no time at all - well worth every dime.
The only "free" option you have in SQL Server 2008 would be to create a link between the two servers and then use something like the MERGE statement (new in SQL Server 2008) to transfer the data. That doesn't work for structural changes, and it's limited only to having a live connection between the two servers.
You should definitely read up on transactional replication. It sounds like a good fit for the situation you've described. Here are a few links to get you started.
How Transactional Replication
Works
How do I... Configure
transactional replication between two
SQL Server 2005 systems?
Performance Tuning SQL Server
Transactional Replication
What you want is Peer-to-Peer Transactional Replication, which allows data to be updated at both databases yet keep them in sync through a contiguous merge of changes. This is the closes match to what you want, but is a fairly costly option (requires Enterprise Edition on both sites). Another option is Bidirectional Transactional Replication, but since this requires also two EE licenses, I say that peer-to-peer is easier to deploy for the same money.
A more budget friendly option is Updatable Subscriptions for Transactional Replication, but updatable subscriptions are being deprecated and you'd bet your money on a loosing horse.
Another option is to use Merge Replication. And finally, for the cases when the 'local' database is quite mobile there is Sync Framework.
Note that all these options require some configuration and cooperation from the Company's server DB.
There are some excellent third party tools out there. For me, xSQL Data Compare has always done the trick. And because the comparisons are highly modifiable it is suitable for almost every data compare or data-synchronization scenario. Hope this helps!
In my current project, the DB is SQL 2005 and the load is around 35 transactions/second. The client is expecting more business and are planning for 300 transactions/second. Currently even with good infrastructure, DB is having performance issues. A typical transaction will have at least one update/insert and a couple of selects.
Have you guys worked on any systems that handled more than 300 txn/s running on SQL 2005 or 2008, if so what kind of infrastructure did you use how complex were the transactions? Please share your experience. Someone has already suggested using Teradata and I want to know if this is really needed or not. Not my job exactly, but curious about how much SQL can handle.
Its impossible to tell without performance testing - it depends too much on your environment (the data in your tables, your hardware, the queries being run).
According to tcp.org its possible for SQL Server 2005 to get 1,379 transactions per second. Here is a link to a system that's done it. (There are SQL Server based systems on that site that have far more transactions... the one I linked was just the first I one I looked at).
Of course, as Kragen mentioned, whether you can achieve these results is impossible for anyone here to say.
Infrastructure needs for high performance SQL Servers may be very differnt than your current structure.
But if you are currently having issues, it is very possible the main part of your problem is in bad database design and bad query design. There are many way to write poorly performing queries. In a high transaction system, you can't afford any of them. No select *, no cursors, no correlated subqueries, no badly performing functions, no where clauses that aren't sargeable and on and on.
The very first thing I'd suggest is to get yourself several books on SQl Server peroformance tuning and read them. Then you will know where your system problems are likely to be and how to actually determine that.
An interesting article:
http://sqlblog.com/blogs/paul_nielsen/archive/2007/12/12/10-lessons-from-35k-tps.aspx
In one of the stackoverflow podcasts (#18 I think) Jeff and Joel were talking about multi vs single tenant databases. Joel mentioned that "FogBugz on Demand" used a database per customer architecture and I was wondering if there is a point beyond which you'll need to have multiple database servers to distribute the load?
Technically the limit of databases per instance in SQL Server is 32,767, but I doubt that you could use a SQL Server instance that has more than 2,000 databases, at that point the server would probably be not responsive.
You may be able to have close to 30,000 databases if they were all auto-closed and not being used. you can find more information about capacity limits here:
Maximum Capacity Specifications for SQL Server
Joel has talked about this at another place (sorry, no reference handy) and said that before switching to MS SQL 2005 the management console (and the backend) had problems attaching more than 1000 or 2000 databases. It seems that 2005 and probably 2008 again improved on these numbers.
As for all performance questions are always dependent on your actual hardware and workload an can only definitely answered by local benchmarking/system monitoring.
I'd think mostly it depends on the memory limitations of the machine. SQL Server likes to keep as much cached in memory as possible, and as you add databases you reduce the amount of memory available.
In addition, you might want to consider the number of connections to a SQL Server. After 500-1000, it gets very cloggy and slow. So that is a limitation as well.
I think it is more a question of the load on the databases. As was said above, if there is no load then 32,767. With a high load then it comes down, eventually to 1 or less than 1.