Where I'm at we have a software package running on a mainframe system. The mainframe makes a nightly dump into sql server, such that each of our clients has it's own database in the server. There are a few other databases in the server instance as well, plus some older client dbs with no data.
We often need to run reports or check data across all clients. I would like to be able to run queries using sp_msforeachdb or something similar, but I'm not sure how I can go about filtering unwanted dbs from the list. Any thoughts on how this could work?
We're still on SQL Server 2000, but should be moving to 2005 in a few months.
Update:
I think I did a poor job asking this question, so I'm gonna clarify my goals and then post the solution I ended up using.
What I want to accomplish here is to make it easy for programmers working on queries for use in their programs to write the query using one client database, and then pretty much instantly run (test) code designed and built on one client's db on all 50 or so client dbs, with little to no modification.
With that in mind, here's my code as it currently sits in Management Studio (partially obfuscated):
use [master]
declare #sql varchar(3900)
set #sql = 'complicated sql command added here'
-----------------------------------
declare #cmd1 varchar(100)
declare #cmd2 varchar(4000)
declare #cmd3 varchar(100)
set #cmd1 = 'if ''?'' like ''commonprefix_%'' raiserror (''Starting ?'', 0, 1) with nowait'
set #cmd3 = 'if ''?'' like ''commonprefix_%'' print ''Finished ?'''
set #cmd2 =
replace('if ''?'' like ''commonprefix_%''
begin
use [?]
{0}
end', '{0}', #sql)
exec sp_msforeachdb #command1 = #cmd1, #command2 = #cmd2, #command3 = #cmd3
The nice thing about this is all you have to do is set the #sql variable to your query text. Very easy to turn into a stored procedure. It's dynamic sql, but again: it's only used for development (famous last words ;) ). The downside is that you still need to escape single quotes used in the query and much of the time you'll end up putting an extra ''?'' As ClientDB column in the select list, but otherwise it works well enough.
Unless I get another really good idea today I want to turn this into a stored procedure and also put together a version as a table-valued function using a temp table to put all the results in one resultset (for select queries only).
Just wrap the statement you want to execute in an IF NOT IN:
EXEC sp_msforeachdb "
IF '?' NOT IN ('DBs','to','exclude') BEGIN
EXEC sp_whatever_you_want_to
END
"
Each of our database servers contains a "DBA" database that contains tables full of meta-data like this.
A "databases" table would keep a list of all databases on the server, and you could put flag columns to indicate database status (live, archive, system, etc).
Then the first thing your SCRIPT does is to go to your DBA database to get the list of all databases it should be running against.
We even have a nightly maintenance script that makes sure all databases physically on the server are also entered into our "DBA.databases" table, and alerts us if they are not. (Because adding a row to this table should be a manual process)
How about taking the definition of sp_msforeachdb, and tweaking it to fit your purpose? To get the definition you can run this (hit ctrl-T first to put the results pane into Text mode):
sp_helptext sp_msforeachdb
Obviously you would want to create your own version of this sproc rather than overwriting the original ;o)
Doing this type of thing is pretty simple in 2005 SSIS packages. Perhaps you could get an instance set up on a server somewhere.
We have multiple servers set up, so we have a table that denotes what servers will be surveyed. We then pull back, among other things, a list of all databases. This is used for backup scripts.
You could maintain this list of databases and add a few fields for your own purposes. You could have another package or step, depending on how you decide which databases to report on and if it could be done programmatically.
You can get code here for free: http://www.sqlmag.com/Articles/ArticleID/97840/97840.html?Ad=1
We based our system on this code.
Related
I'm currently working on a new project where I was hoping to automate the execution of all stored procedures within a specific schema in a database dynamically. I'd like to be able to simply add the stored procedures to the specific schema (e.g. Build), and have a process that runs on a set schedule which simply iterates through all the stored procedures in the schema, and runs them in parallel.
We have an existing custom ETL system that we've built, that will let us setup a bunch of jobs, this currently relies on using multiple agent jobs, that pick up the stored procedures, and executes them. I'm hoping for our new project to use something better, and was thinking the Service Broker would be the answer.
I've investigated this product: http://partitiondb.com/go-parallel-sql/ which seems to provide some pre-built code that will allow me to queue up the procedures, however I can't extract the database they provide, apparently it has a corrupted header :-(
I'm new to the world of service brokers, and am having a little difficulty in working out how I could automatically get something to queue up a bunch of stored procedures that get executed in parallel.
Any help would be much appreciated.
Cheers
To paraphrase one of my favorite poets - "mo' tech, mo' problems". Service Broker is a great solution to asynchronous processing, but it doesn't seem like a good fit here. Specifically, if all you're looking to do is run a (possibly unknown) set of stored procedures on a schedule, dynamic SQL seems like a better fit to me. Something like (untested)
create or alter procedure dbo.gottaRunEmAll
as
begin
declare p cursor fast_forward, local for
select name
from sys.objects
where schema_id = schema_id('Build');
open p;
declare #name sysname, #sql nvarchar(max);
while(1=1)
begin
fetch next from p into #name;
if (##rowcount <> 0)
break;
set #sql = concat('exec [Build].', quotename(name), ';');
exec(#sql)
end
close p
deallocate p
end
Better (imo) would be to have the above procedure maintained explicitly to call the procedures that you want and how you want them.
I'm creating some views with a lot of references to tables in another database.
At some point the other database needs to change.
I want to make it easy for the next developer to change the scripts to use another database.
This obviously work like it should:
CREATE VIEW ViewName
AS
SELECT *
FROM AnotherDatabase.SchemaName.TableName;
But when I do:
DECLARE #DB CHAR(100)
SET #DB = 'AnotherDatabase'
GO
CREATE VIEW ViewName
AS
SELECT *
FROM #DB.SchemaName.TableName;
I get the error:
Msg 137, Level 15, State 2, Procedure ViewName, Line 3
Must declare the scalar variable "#DB".
I could do something like:
DECLARE #SQL ...
SET #SQL = ' ... FROM ' + #DB + ' ... '
EXEC (#SQL)
But that goes against the purpose of making it easier for the next developer - because this dynamic SQL approach removed the formatting in SSMS.
So my question is: how do I make it easy for the next developer to maintain T-SQL code where he needs to swap out the database reference?
Notes:
I'm using SQL Server 2008 R2
The other database is on the same server.
Consider using SQLCMD variables. This will allow you to specify the actual database name at deployment time. SQL Server tools (SSMS, SQLCMD, SSDT) will replace the SQLCMD variable names with the assigned string values when the script is run. SQLCMD mode can be turned on for the current query windows from the menu option Query-->SQLCMD mode option.
:SETVAR OtherDatabaseName "AnotherDatabaseName"
CREATE VIEW ViewName AS
SELECT *
FROM $(OtherDatabaseName).SchemaName.TableName;
GO
This approach works best when SQL objects are kept under source control.
When you declare variables, they only live during the execution of the statement. You can not have a variable as part of your DDL. You could create a bunch of synonyms, but I consider that over doing it a bit.
The idea that your database names are going to change over time seems a bit out of the ordinary and conceivably one-time events. However, if you do still require to have the ability to quickly change over to point to a new database, you could consider creating a light utility directly in SQL to automatically generate the views to point to the new database.
An implementation may look something like this.
Assumptions
Assuming we have the below databases.
Assuming that you prefer to have the utility in SQL instead of building an application to manage it.
Code:
create database This;
create database That;
go
Configuration
Here I'm setting up some configuration tables. They will do two simple things:
Allow you to indicate the target database name for a particular configuration.
Allow you to define the DDL of the view. The idea is similar to Dan Guzman's idea, where the DDL is dynamically resolved using variables. However, this approach does not use the native SQLCMD mode and instead relies on dynamic SQL.
Here are the configuration tables.
use This;
create table dbo.SomeToolConfig (
ConfigId int identity(1, 1) primary key clustered,
TargetDatabaseName varchar(128) not null);
create table dbo.SomeToolConfigView (
ConfigId int not null
references SomeToolConfig(ConfigId),
ViewName varchar(128) not null,
Sql varchar(max) not null,
unique(ConfigId, ViewName));
Setting the Configuration
Next you set the configuration. In this case I'm setting the TargetDatabaseName to be That. The SQL that is being inserted into SomeToolConfigView is the DDL for the view. I'm using two variables, one {{ViewName}} and {{TargetDatabaseName}}. These variables are replaced with the configuration values.
insert SomeToolConfig (TargetDatabaseName)
values ('That');
insert SomeToolConfigView (ConfigId, ViewName, Sql)
values
(scope_identity(), 'dbo.my_objects', '
create view {{ViewName}}
as
select *
from {{TargetDatabaseName}}.sys.objects;'),
(scope_identity(), 'dbo.my_columns', '
create view {{ViewName}}
as
select *
from {{TargetDatabaseName}}.sys.columns;');
go
The tool
The tool is a stored procedure that takes a configuration identifier. Then based on that identifier if drops and recreates the views in the configuration.
The signature for the stored procedure may look something like this:
exec SomeTool #ConfigId;
Sorry -- I left out the implementation, because I have to scoot, but figured I would respond sooner than later.
Hope this helps.
I have three websites which uses an abstract database structure with tables like: Items, Places, Categories, etc... and stored procedures like GetItemsByCategory, GetRelatedItems, etc... Actually im using exactly the same database structure for these 3 different websites.
From a code perspective im using the same code for all websites (except the HTML which is specific foreach one), and all the common code is in few projects used by all websites, so everytime that i detect a bug (which is in all websites) i just fix it on one place (the common part used by all) and automatically all websites get the fix.
Actually im using Asp.net MVC3 and Sql server.
Everytime i want to extend some funcionality, and i need a new table, stored procedure or something related with database, i have to do the modification in each database.
Do you know any approach that i could use to be able to have the same flexibility and do database modifications only one time for all websites?
Do you think I'm using a good approach or i should use something different in your opinion?
If the databases are on a single server, you could generate the script for the procedure from Management Studio, and make sure to use the option to "check for object existence" (Tools > Options > SQL Server Object Explorer > Scripting). This will yield something like this (most importantly it produces your stored procedure code as something you can execute using dynamic SQL):
USE DBName;
GO
SET ANSI_NULLS ON;
GO
SET QUOTED_IDENTIFIER ON;
GO
IF NOT EXISTS (...)
BEGIN
EXEC dbo.sp_executesql #statement = N'CREATE PROCEDURE dbo.whatever ...
'
END
GO
Now that you have this script, you can modify it to work across multiple databases - you just need to swipe the #statement = portion and re-use it. First you need to stuff the databases where you want this to work into a #table variable (or you can put this in a permanent table, if you want). Then you can build a command to execute in each database, e.g.
DECLARE #dbs TABLE (name SYSNAME);
INSERT #dbs(name) SELECT N'db1';
INSERT #dbs(name) SELECT N'db2';
INSERT #dbs(name) SELECT N'db3';
-- now here is where we re-use the create / alter procedure command from above:
DECLARE #statement NVARCHAR(MAX) = N'CREATE PROCEDURE dbo.whatever ...
';
-- now let's build some dynamic SQL and run it!
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'';
SELECT #sql = #sql + '
EXEC ' + QUOTENAME(name) + '.dbo.sp_executesql N''' + #statement + ''';'
FROM #dbs;
EXEC sys.sp_executesql #sql;
Alternatively, you could create a custom version of my sp_msforeachdb or sp_ineachdb replacements:
Making a more reliable and flexible sp_MSforeachdb
Execute a Command in the Context of Each Database in SQL Server
I used to use a tool called SQLFarms Combine for this, but the tool doesn't seem to exist anymore, or perhaps it has been swallowed up / re-branded by another company. Red Gate has since produced SQL Multi Script that has similar functionality.
If you added a column to all your tables called websiteId you could just have one database. Store the unique websiteId in each site's web.config and just pass it with each request for data. Obviously each site's data is stored with their websiteId so data can be queried per website.
It means a bit of refactoring in your db and any calls to your your db, but once done, you only have one database to maintain.
Of course this is assuming your databases are on the same server...
How can I perform this query on whatever way:
delete from sys.tables where is_ms_shipped = 0
What happened is, I executed a very large query and I forgot to put USE directive on top of it, now I got a zillion tables on my master db, and don't want to delete them one by one.
UPDATE: It's a brand new database, so I don't have to care about any previous data, the final result I want to achieve is to reset the master db to factory shipping.
If this is a one-time issue, use SQL Server Management Studio to delete the tables.
If you must run a script very, very carefully use this:
EXEC sp_msforeachtable 'DROP TABLE ?'
One method I've used in the past which is pretty simple and relatively foolproof is to query the system tables / info schema (depending on exact requirements) and have it output the list of commands I want to execute as the results set. Review that, copy & paste, run - quick & easy for a one-time job and because you're still manually hitting the button on the destructive bit, it's (IMHO) harder to trash stuff by mistake.
For example:
select 'drop table ' + name + ';', * from sys.tables where is_ms_shipped = 0
No backups? :-)
One approach may be to create a Database Project in Visual Studio with an initial Database Import. Then delete the tables and synchronize the project back to the database. You can do the deletes en masse with this approach while being "buffered" with a commit phase and UI.
I am fairly certain the above approach can be used to take care of the table relationships as well (although I have not tried in the "master" space). I would also recommend using a VS DB project (or other database management tool that allows schema comparing and synchronization) to make life easier in the future as well as allowing version-able (e.g. with SCM) schema change-tracking.
Oh, and whatever is done, please create a backup first. If nothing else, it is good training :-)
Simplest and shortest way I did was this:
How to Rebuild System Databases in SQL Server 2008
The problem with all other answers here is that it doesn't work, since there are related tables and it refuses to execute.
This one, not only it works but actually is what I am looking for: "Reset to factory defaults" as stated in the question.
Also this one will delete everything, not only tables.
This code could be better but I was trying to be cautious as I wrote it. I think it is easy to follow an easy to tweak for testing before you commit to deleting your tables.
DECLARE
#Prefix VARCHAR(50),
#TableName NVARCHAR(255),
#SQLToFire NVARCHAR(350)
SET #Prefix = 'upgrade_%'
WHILE EXISTS(
SELECT
name
FROM
sys.tables
WHERE
name like #Prefix
)
BEGIN
SELECT
TOP 1 --This query only iterates if you are dropping tables
#TableName = name
FROM
sys.tables
WHERE
name like #Prefix
SET #SQLToFire = 'DROP TABLE ' + #TableName
EXEC sp_executesql #SQLToFire;
END
I did something really similar, and what I wound up doing was using the Tasks--> script database to only script drops for all the database objects of the originally intended database. Meaning the database I was supposed to run the giant script on, which I did run it on. Be sure to include IF Exists in the advanced options, then run that script against the master and BAM, deletes everything that exists in the original target database that also exists in the master, leaving the differences, which should be the original master items.
Not very elegant but as this is a one time task.
WHILE EXISTS(SELECT * FROM sys.tables where is_ms_shipped = 0)
EXEC sp_MSforeachtable 'DROP TABLE ?'
Works fine on this simple test (clearing a on the second loop after failing on the first attempt and proceeding onwards to delete b)
create table a
(
a int primary key
)
go
create table b
(
a int references a (a)
)
insert into a values (1)
insert into b values (1)
I found strange rules in MS SQL CREATE VIEW syntax.
It must be on the first line of query batch processing and it must be created in the current database.
I should make VIEWs that have dynamic name described by string variables
(type: VARCHAR or NVARCHAR). And those VIEWs should be created in other databases.
Because of the rule, CREATE VIEW statement must be on the first line of query batch processing, it cannot be after USE statement.
So, I tried to change databases with USE & GO statement. But GO statement seemed to make clear all the variables. Therefore they are not available that describe VIEW name after GO statement.
Do you have any opinon for me?
And if you know the reasons of CREATE VIEW syntax rules, please tell me.
Oh~, Sorry. I missed one thing. The names of databases are also dynamic.
And VIEWs, I want to make, not only should access tables of other databases
but also shoule be created in other databases.
Though I don't know OLAP well, I think this situation is involved OLAP.
You are able to dynamically create a sql-string and execute it.
DECLARE #ViewName VARCHAR(100)
SET #ViewName = 'MyView'
USE MyDB;
EXEC ( 'CREATE VIEW dbo.' + #ViewName + ' '
+ 'AS SELECT * FROM dbo.MyTable')
CREATE SYNONYM Resource1 FOR LinkedServer.Database.Schema.Table
GO
CREATE VIEW Resource1View
AS
SELECT *
FROM Resource1
GO
Now you can alter the synonym as much as you like and all your views referencing it will refer to the correct thing. If this doesn't solve the problem, then I would suggest that the way you're designing your system is not best. Please describe more what you are doing and why so we can advise you better.
As for "GO", is it actually not a SQL statement. It is never submitted to the server. The client sees the line with GO on it, and separates the submitted query into separate batches. A trace will prove this, as will EXEC 'SELECT 1' + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10) + 'SELECT 2'.
If you're using OLAP as in Analysis Services, then I'm not experienced enough with that to help you, but I would think there'd be ways to choose the database to connect to just like in SSRS, and that queries don't have to live in the database but could live in the SSAS application.
I found it. It's the nested EXEC.