Based on our requirement we need to have one database per account as we cannot have a single database. So we were thinking about multiple database on the SQL Server and based on my research we can have appx 32,000 database on single database instance but the resource limit will be reached far sooner.
How many is too many databases on SQL Server?
Another point was to use no-sql database like MongoDB. As far as I know we can create database per account with different database name.
I wanted to check reference to resources used by MongoDB, do I need more ram, processor and lots of servers for the architecture.
We are looking to have about 10,000 databases max on a server. Can that work on the a single mongodb server?
If you do have similar architecture currently running, please do share.
Related
We have approx. 8 odd SQL Servers used for different purposes like inserting data in 1 server, update in another etc. (or connecting to only that database based on user’s region).
The problem is sometimes query for data needs to be done from multiple SQL Server databases. So say, I have an Id property, and based on the Id data needs to be retrieved from multiple of these 8 servers (if there is an Id match, so basically querying all database).
So basically the server which the user is logged into, will use “Linked Server” functionality and connect to other SQL Servers (with the server which the user is currently on acts as the source SQL Server), and using “UNION” functionality to club all data.
As a lot of transactions is taking place each day, this approach is not feasible, performance wise.
So any recommendations on a better approach to achieve the same above functionality. I read a concept called “Server Groups” but not sure of it.
The application is made in .Net Web Forms using Jquery/Ajax/HTML/API and ADO.NET.
If you have a .net application which is outside these 8 servers can't you establish individual connections and pass the ID from .net app to these servers ?
As far as I know "Server Group" is a concept in SSMS which helps you to group the servers and can run common scripts at same time.
We have a requirement where we will have to move data between different database instance on regular basis. (For e.g. some customers willing to pay more for the better performance). So this is not going to be one off.
The database tables has referential integrity. Is there a way in which this can be done without rewriting sql script (or some other method) every time we migrate customers data?
I came across this How to move data between multiple database's table while maintaining foreign-key relationships/referential integrity?. However it appears that we have write script every time we migrate data (please correct me if I misunderstood the answer on this thread).
Thanks
Edit:
Both servers are using SQL Server 2012 (same version). Its an Azure SQL Server database.
They are not necessarily linked (no firewall between them)
We are only transferring some data, not the whole database. This is only for certain customers who opted pay more.
The schema are exactly same in both databases.
Preyash - please see the documentation on the Split-Merge tool. The Split-Merge tool enables you do move data between databases, as you have described, based on a sharding key (e.g., customer ID). One modification that you will need for your application is to add a shard map (i.e., a database that understand the global state of which customers resides in which databases).
Have a look into Azure Data Sync. It is much more aligned with your requirements. But you may end up in having another SQL Azure DB to maintain a Hub. Azure data Sync follows hub-spoke pattern and will let you do all flexible directional syncs with a few minutes of syncing gap. It is more simple and can set it up very fast without any scripts and all as you wanted.
Does anyone have experience building database reports - doesn't matter which database - i just want design ideas - for a system that is made up of many separate, but identical databases?
I cannot "combine" all databases into one. They must be separate.
But the structure is identical across all databases...
I need to build a web interface that will allow a user to get a "global" report that will query all databases and build one combined report.
Do you have any comments on how the model would look like? or anything you think i need to beware of?
Thanks.
I don't have first hand experience with cross database reports, my experience comes from a product the company i work for sells which can create reports from multiple databases, from your description i believe you require something of the "combine" tables kind, in this case i recommend you to detect the tables used in the query, and unify them in a single temporary intermediary database, for example Access, SQL Server CE or SQLite and then run the query against this temporary database or table.
If your databases are Microsoft SQL Server, then using SQL Server Reporting Services seems like a good solution. The software for the report generation / display is bundled along with the database software.
It gives you a web interface, where you can configure 'data sources' from any number of remote databases, and combine data from these sources into reports. It is user friendly and you can do all the report design / configuration through the web interface without having to write any code.
some references :
Building report using SQL Server stored procedure
http://blog.hoegaerden.be/2009/11/10/reporting-on-data-from-stored-procedures-part-1/
I have a machine where we have SSRS and SQL Server running, Currently we use SSRS and configured it to use localhost SQL Server hosted ReportServer and ReportServertempdb.
Now we need decomission this machine and move the reports to a brand new machine. Now the brand new machine does not have SQL Server, but we have another SQL Server which has 2 pre-existing databases: ReportServer and ReportServertempdb which are already being used by another SSRS instance. Now we are planning to use them for our to-be-moved SSRS instance as well.
Is it possible for multiple SSRS instance to use same ReportServer and ReportServertempdb? If yes how do you merge these? There are certain conflicting values. For example in the ReportServer DB in the table "ConfigurationInfo", SystemReportTimeout is 1800 for my current instance, but on my proposed instance value is -1. How to tackle this during a merge? I have many conflicting records in both ReportServer databases.
You can create several databases on your machine, each DB + TempDB for one instance. But it is not possible to share DB + TempDB for two different instances.
Even if this would work in theory, you will get a lot of problems with sync. caches tables, security tokens, etc.
Edit: So imho backup your databases and restore on a different name. Then you SSRS configration manager to connect to a existing database.
If you use security tokens for those DB's, make sure your have the .snk file.
Though you can share RS databases across multiple instances, you cannot merge two different ones to the same one. Unfortunately we cannot even rename RS databases as they are considered system databases. Your best bet would be to see if you can host your databases on another SQL instance on the same server or redeploy your reports on the new server.
For additional information you can refer to the following links:
Scaleout deployment: http://msdn.microsoft.com/en-us/library/ms157293(v=SQL.90).aspx
Managing RS DBs:http://msdn.microsoft.com/en-us/library/ms156421%28v=SQL.90%29.aspx
http://msdn.microsoft.com/en-us/library/ms159093.aspx
A colleague I work with recently told me that SQL Express and MS Access were essentially the same thing; that does not seem to be an accurate statement. I know you can convert Access to a SQL DB and maybe under the covers they are similar, but I would assume that the SQL DB engine and what is used to run access are not the same. Not only that, but the SQL statement syntax, etc. I know are not the same.
I am mainly trying to understand so that I am more informed about the versions.
Um, no, not the same.
First off, I need to clear up some terminology. MS Access is a Rapid Application Development (RAD) tool that allows you to quickly build forms and reports that are bound to relational data. It comes with a file-based database engine (Jet/ACE).
Access the RAD tool can be used with many different backend databases (Jet, SQL Server, any db that supports ODBC, etc). I have to assume your colleague was specifically commenting on Jet/ACE, ie the database engine that MS Access uses.
I think the single biggest difference between the Jet/ACE database engine and MS SQL Server Express is that Jet/ACE is file-based and SQL Server Express uses a client/server model. This means that SQL Server Express requires a running service to provide access to the datastore. This can complicate deployment in some scenarios.
SQL Server Express is really just a throttled-back version of SQL Server: max database size of 4GB (10GB in 2008R2), only uses a single physical CPU, etc. These limitations are imposed to prevent large organizations from using the freely available Express edition in place of a full-blown SQL Server install. The upshot to this is that SQL Server Express offers a truly seamless upgrade path to SQL Server. It is also (generally speaking) a more robust and fully featured database management system then Jet/ACE.
Similarities
relational database management systems
written by Microsoft
Differences
MS Access
File based
free distributable runtime (2007 or later)
RAD tools (form/report designer)
uses Jet SQL
max file size 2GB
SQL Server Express
Client/Server model
free
no RAD tools
uses Transact-SQL
max database size 4GB (10GB for SSE R2), max one physical CPU
I think what your colleague had in mind was SQL Server CE, which is a super-lightweight embedded database, which is still (IMO) far superior to Access in database-management aspect. SQL Express cannot even be compared with Access without offending the former.
Here are the datasheets for both products so you can see some hard facts on the difference between the two databases.
Access:
http://office.microsoft.com/en-us/access-help/access-specifications-HP005186808.aspx
SQL (Express is listed on the far right column):
http://www.microsoft.com/sqlserver/2008/en/us/editions-compare.aspx
The comment I have always read is that Access is great for single user single access database use, the minute you scale beyond a single user look elsewhere. While that may be a "bit" of a stretch, Access really does not do well in a multi-user environment. From experience we've had a client who has ignored and ignored our requests to migrate a backend database from Access to SQL, and there have been numerous occasions where we have had to restore from backups, or take the Access database offline due to corruption.
They are two completely different technologies with two different target markets. The database engines are indeed different, as you mention T-SQL is different than Access SQL.
You can "scale up" an Access database to SQL by creating an SSIS package or other tool to do the import, but this takes the Access schema and data and migrates it to a true SQL database. It does more than just attach the Access database or the like.
Anytime you need a "real" database I'd highly recommend looking at any of the SQL versions that are available over Access.
Just remember that with MS-Access you don't have size limitations if you play your cards right. There is no reason, for example, not to have many 2 to 4 Gig tables each contained singularly in their own database. Your ODBC applications can open a connection to multiple MS-Access databases and query the single table in each. So you can have a database containing trillions of records, stored in multiple MDB files. One company I went to work for was using a single MS-Access database to run a issue tracking system done in MS-Access forms. They could only use it one person at a time because of sharing issues that would lock MS-Access up. I wrote a Win32 Perl native Windows GUI user-interface to the database that was better at field/record validation, and my ODBC code was able to manage the connection for simultaneous user access. I managed the opening and reading and writing and closing of the database for each user through my Perl program. I did not leave the database open. I did not maintain a persistent connection for each user, but instead only maintained a connection long enough to retrieve a record for edit. Then I closed the connection until it was time to write the record back to the database. Also, I wrote my own record locking program logic by maintaining a user login table that contained the record id of the record a user was currently editing, then erased that entry when no longer editing that record. When another user went to edit the same record, the program checked if that record was currently open for edit by another user. The system worked flawlessly. MS-Access never locked up via ODBC and multi-user access. I even embedded the password to the database in my compiled Perl program so that no one could get to the data in the Access database other than through my Perl program.