I have a single windows shell command I'd like to run (via EXEC master..xp_cmdshell) once for each row in a table. I'm using information from various fields to build the command output.
I'm relativity new to writing T-SQL programs (as opposed to individual queries) and can't quite get my head around the syntax for this, or if it's even possible/recommended.
I tried creating a single column table variable, and then populating each row with the command I want to run. I'm stifled at how to iterate over this table variable and actually run the commands. Googling around has proven unhelpful.
Thanks in advance for any help!
You could always use a cursor:
USE Northwind
DECLARE #name VARCHAR(32)
DECLARE #command VARCHAR(100)
DECLARE shell_cursor CURSOR FOR
SELECT LastName FROM Employees
OPEN shell_cursor
FETCH NEXT FROM shell_cursor INTO #name
WHILE ##FETCH_STATUS = 0
BEGIN
SET #command = 'echo ' + #name
EXEC master.dbo.xp_cmdshell #command
FETCH NEXT FROM shell_cursor INTO #name
END
CLOSE shell_cursor
DEALLOCATE shell_cursor
GO
Is this a once-off job? If so, you might be better off coming from the reverse direction. That is to say, instead of writing a stored procedure to call XP_CMDSHELL to run some program against table data, you should consider writing a program to work against the table data directly. If one scripting product comes to mind, it's PowerShell. It has integrated support for any database that the windows platform supports and you'll find a ton of scripts on www.poshcode.org to do that kind of thing.
On the other hand, if this is something that is to be scheduled, I guess there's nothing hugely wrong with your idea, apart from the fact that XP_CMDSHELL is disabled out of the box with SQL Server these days. Re-enabling it is opening your server to a whole new world of exploits, especially if that table data is sourced from a web page form or some other questionable source.
-Oisin
Related
I have the following example of script. It works fine for this example as it is small and basic. I have a far bigger script with around 50 steps that works fine on its own. As in, hit F5 and does everything, checks if table exists, deletes, writes to tables, drops temps and all results are written to the necessary places. I can't seem to place this large script into this small example:
alter procedure james_tester
#tablename nvarchar(200)
as
BEGIN
declare #sql nvarchar(max)
set #sql =
---->
'select * from'
---->
+ #tablename
EXECUTE sp_executesql #sql
END
---When executing:
execute james_tester 'dbo.calendar_delete'
In my case the section in between the arrows is large and will have numerous variables. I just want to know if there is a function or possibly another way to place a large piece of script at a time in that single quote part. I hope I am describing this sufficiently. What affects the entire script currently from just putting a single quote before and after, is that there are already many comments, and single quotes used in the script that seem to stop the entire script from being highlighted red as text and working fine.
James
It is in fact only the presence of single quotes that will cause problems. Take the following illustration:
declare #bigText varchar(max);
SET #bigText = '"The time has come," the Walrus said,
"To talk of many things:
Of shoes and ships and sealing-wax --SQL Comment has no impact
Of cabbages and kings /* c/c++ comment has no impact */
And why the sea is boiling hot //c# comment has no impact
And whether pigs have wings."';
SELECT #bigText;
Single quotes can be doubled using regex or some other string replace function, so that should not be too hard either.
BUT (and there always is a but of course) whether the remaining text is a legal SQL string, which you can execute is an entirely different question. The presence of extraneous comments etc. will almost certainly bite you.
In case anyone comes to this post and needs to know how this ended. I have no real security risks here of SQL injections etc on the database. The procedure takes a few minutes to run now and just takes this execute in the first two rows to run, with the variables being the periods of SAP backup I need to extract:
execute jc_tester '01' , '02' , '03'
go
alter procedure jc_tester
#Period1 varchar (3) ,#Period2 varchar (3) , #Period3 varchar (3)
as
begin
declare #sql nvarchar(max)
set #sql =
replace('...... | | | | ','|','''') + ......... #period1 +
replace('.....','|','''')
execute sp_executesql #sql
end
Each section of text contained ' that I could not seem to avoid and caused havoc when handling as a string. These had to be replace with pipes, then included in a replace statement.
Some other things I learnt along the way that threw my progress out. Don't use Go's in the script to be used in #sql. I did not know this. Rather use ;. Also remove any comments that use the leading '---'. Instead wrap all comments in the script with /...../ . I could swear this made a difference as so many of the comments in the script in #sql were all trailing ---'s.
This process has saved time and now that it is complete I can explore other options other than dynamic sql in a stored procedure and learn how to do it in probably a more appropriate manner. But I'm glad this ran eventually.... Thanks for the guidance.
Before I start, I need to point out that I am a SQL noob. I can write basic statements, but anything past JOIN statements is probably fairly new to me.
That said, I have cobbled together a script that deletes records from tables. The script itself does what it needs to do; however, when I run this script, I change the "USE" line to whatever database is next, stepping through databases manually. I use a command which populates a temporary table with a list of database names as reference.
How can I run my script against each database name in the temporary table directly, preferably all from a single stored procedure?
Well, one option is that you can use a cursor to grab all the database names (exclude databases you don't want to execute on), and use dynamic sql to execute for each database. It unfortunately has to be "dynamic" since you can't just do a while loop, USE #dbname /*your magic code*/ Fetch Next from MyCursor into #dbname... It errors out when trying to do USE #dbname, So you actually do have to use exec #variable in it.
DECLARE #dbname varchar(max)
DECLARE #executeme nvarchar(max)
DECLARE DBCursor CURSOR FOR
SELECT Name
FROM sys.databases
WHERE name not in ('master', 'tempdb', 'model', 'msdb'); --add additional exclusions here
OPEN DBCursor;
FETCH NEXT FROM DBCursor INTO #dbname;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #executeme =
N'use '+#dbname+'
--YOUR SCRIPT HERE
'
EXEC sp_executesql #executeme
FETCH NEXT FROM DBCursor into #dbname;
END;
CLOSE DBCursor;
DEALLOCATE DBCursor;
GO
This is going to loop across all databases, execute your script in each, until there is none left in the list of databases. You can exclude certain databases in the select statement for the cursor (like the master, etc.) and add in any additional logic as you see fit for excluding.
Additionally, you can implement this in a stored procedure, so all you have to do is run the stored procedure, sit back and drink your favorite drink while it does the heavy lifting for you ;)
I have three websites which uses an abstract database structure with tables like: Items, Places, Categories, etc... and stored procedures like GetItemsByCategory, GetRelatedItems, etc... Actually im using exactly the same database structure for these 3 different websites.
From a code perspective im using the same code for all websites (except the HTML which is specific foreach one), and all the common code is in few projects used by all websites, so everytime that i detect a bug (which is in all websites) i just fix it on one place (the common part used by all) and automatically all websites get the fix.
Actually im using Asp.net MVC3 and Sql server.
Everytime i want to extend some funcionality, and i need a new table, stored procedure or something related with database, i have to do the modification in each database.
Do you know any approach that i could use to be able to have the same flexibility and do database modifications only one time for all websites?
Do you think I'm using a good approach or i should use something different in your opinion?
If the databases are on a single server, you could generate the script for the procedure from Management Studio, and make sure to use the option to "check for object existence" (Tools > Options > SQL Server Object Explorer > Scripting). This will yield something like this (most importantly it produces your stored procedure code as something you can execute using dynamic SQL):
USE DBName;
GO
SET ANSI_NULLS ON;
GO
SET QUOTED_IDENTIFIER ON;
GO
IF NOT EXISTS (...)
BEGIN
EXEC dbo.sp_executesql #statement = N'CREATE PROCEDURE dbo.whatever ...
'
END
GO
Now that you have this script, you can modify it to work across multiple databases - you just need to swipe the #statement = portion and re-use it. First you need to stuff the databases where you want this to work into a #table variable (or you can put this in a permanent table, if you want). Then you can build a command to execute in each database, e.g.
DECLARE #dbs TABLE (name SYSNAME);
INSERT #dbs(name) SELECT N'db1';
INSERT #dbs(name) SELECT N'db2';
INSERT #dbs(name) SELECT N'db3';
-- now here is where we re-use the create / alter procedure command from above:
DECLARE #statement NVARCHAR(MAX) = N'CREATE PROCEDURE dbo.whatever ...
';
-- now let's build some dynamic SQL and run it!
DECLARE #sql NVARCHAR(MAX);
SET #sql = N'';
SELECT #sql = #sql + '
EXEC ' + QUOTENAME(name) + '.dbo.sp_executesql N''' + #statement + ''';'
FROM #dbs;
EXEC sys.sp_executesql #sql;
Alternatively, you could create a custom version of my sp_msforeachdb or sp_ineachdb replacements:
Making a more reliable and flexible sp_MSforeachdb
Execute a Command in the Context of Each Database in SQL Server
I used to use a tool called SQLFarms Combine for this, but the tool doesn't seem to exist anymore, or perhaps it has been swallowed up / re-branded by another company. Red Gate has since produced SQL Multi Script that has similar functionality.
If you added a column to all your tables called websiteId you could just have one database. Store the unique websiteId in each site's web.config and just pass it with each request for data. Obviously each site's data is stored with their websiteId so data can be queried per website.
It means a bit of refactoring in your db and any calls to your your db, but once done, you only have one database to maintain.
Of course this is assuming your databases are on the same server...
Where I'm at we have a software package running on a mainframe system. The mainframe makes a nightly dump into sql server, such that each of our clients has it's own database in the server. There are a few other databases in the server instance as well, plus some older client dbs with no data.
We often need to run reports or check data across all clients. I would like to be able to run queries using sp_msforeachdb or something similar, but I'm not sure how I can go about filtering unwanted dbs from the list. Any thoughts on how this could work?
We're still on SQL Server 2000, but should be moving to 2005 in a few months.
Update:
I think I did a poor job asking this question, so I'm gonna clarify my goals and then post the solution I ended up using.
What I want to accomplish here is to make it easy for programmers working on queries for use in their programs to write the query using one client database, and then pretty much instantly run (test) code designed and built on one client's db on all 50 or so client dbs, with little to no modification.
With that in mind, here's my code as it currently sits in Management Studio (partially obfuscated):
use [master]
declare #sql varchar(3900)
set #sql = 'complicated sql command added here'
-----------------------------------
declare #cmd1 varchar(100)
declare #cmd2 varchar(4000)
declare #cmd3 varchar(100)
set #cmd1 = 'if ''?'' like ''commonprefix_%'' raiserror (''Starting ?'', 0, 1) with nowait'
set #cmd3 = 'if ''?'' like ''commonprefix_%'' print ''Finished ?'''
set #cmd2 =
replace('if ''?'' like ''commonprefix_%''
begin
use [?]
{0}
end', '{0}', #sql)
exec sp_msforeachdb #command1 = #cmd1, #command2 = #cmd2, #command3 = #cmd3
The nice thing about this is all you have to do is set the #sql variable to your query text. Very easy to turn into a stored procedure. It's dynamic sql, but again: it's only used for development (famous last words ;) ). The downside is that you still need to escape single quotes used in the query and much of the time you'll end up putting an extra ''?'' As ClientDB column in the select list, but otherwise it works well enough.
Unless I get another really good idea today I want to turn this into a stored procedure and also put together a version as a table-valued function using a temp table to put all the results in one resultset (for select queries only).
Just wrap the statement you want to execute in an IF NOT IN:
EXEC sp_msforeachdb "
IF '?' NOT IN ('DBs','to','exclude') BEGIN
EXEC sp_whatever_you_want_to
END
"
Each of our database servers contains a "DBA" database that contains tables full of meta-data like this.
A "databases" table would keep a list of all databases on the server, and you could put flag columns to indicate database status (live, archive, system, etc).
Then the first thing your SCRIPT does is to go to your DBA database to get the list of all databases it should be running against.
We even have a nightly maintenance script that makes sure all databases physically on the server are also entered into our "DBA.databases" table, and alerts us if they are not. (Because adding a row to this table should be a manual process)
How about taking the definition of sp_msforeachdb, and tweaking it to fit your purpose? To get the definition you can run this (hit ctrl-T first to put the results pane into Text mode):
sp_helptext sp_msforeachdb
Obviously you would want to create your own version of this sproc rather than overwriting the original ;o)
Doing this type of thing is pretty simple in 2005 SSIS packages. Perhaps you could get an instance set up on a server somewhere.
We have multiple servers set up, so we have a table that denotes what servers will be surveyed. We then pull back, among other things, a list of all databases. This is used for backup scripts.
You could maintain this list of databases and add a few fields for your own purposes. You could have another package or step, depending on how you decide which databases to report on and if it could be done programmatically.
You can get code here for free: http://www.sqlmag.com/Articles/ArticleID/97840/97840.html?Ad=1
We based our system on this code.
I would like to write a 'generic' script to operate on a number of databases. The 'use databasename' commands will be distributed in the body of the script. I would rather show the database name information at the top of the script. So ideally:
declare #db varchar(100)
set #db = 'data_in'
use #db
This fails. So perhaps
declare #db varchar(100)
set #db = 'data_in'
exec ('use '+#db)
This executes, but presumably changes the database only in the context of the query in the string.
So is there any other way without placing the whole script into a string and executing it that way ie. a cure worse than the disease?
Check out Scripting Variables in SQLCMD.
This enables you to put variables into your scripts in the form:
USE $(db1_name)
...some code...
USE $(db2_name)
...some code...
And have the variable values interpolated from environment variables, parameters provided at runtime, or hard coded value assignments.
Ed already mentioned SQLCMD, a very good choice for scripting.
If that doesn't do it for you, and you have sa rights on the server, and you don't mind the risk of using undocumented features and modifying the master database, you might look into user-defined system stored procedures.
A user-defined system stored procedure (UDSSP) is created in the master database, prefixed with "sp_", and marked as a system object with the undocumented system proc sp_MS_marksystemobject (SQL2005).
It takes its database context from the current connection, or a three-part name if so called.
Sample invocation:
declare #db sysname
declare #sql nvarchar(max)
set #db = 'yourdatabase'
set #sql = 'exec '+quotename(#db)+'..sp_#yourproc'
exec (#sql)
Notes:
If you go this route, I strongly suggest using a unique prefix that sorts toward the top, like sp_#yourproc, rather than sp_yourproc, so you can find them again later, and other people know that they are something special.
Once the procedure is marked as system, it can't be updated. To make changes, you must drop, recreate and remark as system.
Don't do this unless you know what you are doing and have done a little more research. Don't do this if you are risk-averse. Don't do this if you don't have a development instance to test on first.
Backup the UDSSPs to file or CSV. A server upgrade can wipe them out.
You could NOT put a use statement in your script, and supply the database with the osql command (or whatever you're using) like this:
osql -S servername -d databasename -U username -P password -i script.sql
I think your presumption’s right; I had the same problem yesterday. I solved it by putting the commands into a single string. Why don’t you like that solution?