I have this SQL file with the CREATE statements for 697 stored procedures in it, but unfortunately it won't execute due to a syntax problem.
The script requires a GO statement between every CREATE PROCEDURE call.
The question is: how do I add GO before every CREATE PROCEDURE statement?
I am looking to achieve this through the use of PowerShell code.
The Select-String might guide us in the right direction as it is able to find the 697 stored procedures in the SQL file. The below returned a count of 697. But not sure how to use this to add text in front of every finding.
(Select-String -Path $sqlFile -Pattern "CREATE PROCEDURE" -AllMatches).Matches.Count
I also tried to replace text in the file with the below command
(Get-Content $sqlFile).replace('CREATE PROCEDURE', ' GO CREATE PROCEDURE') | Set-Content $sqlFile
This however resulted in an error when executing the SQL script:
A fatal scripting error occurred. Incorrect syntax was encountered while parsing GO
Thank you for the help.
The solution is found. Use the `ncharacter.
(Get-Content $sqlFile).replace('CREATE PROCEDURE', " GO `n CREATE PROCEDURE") | Set-Content $sqlFile
Related
MSSQL lets you do a multi-row UPDATE and INSERT from some source, like another table, or JSON & XML data, etc. Psuedo-example:
INSERT INTO TBL SELECT * FROM SOURCE
Is it possible to use a PowerShell hash table as the source?
#psuedo code
$carHashTable = [pscustomobject]#{
color = 'red'
make = 'geo'
model = 'metro'
year = 1996
insured = 0
}
#here-string
$sql = #"
insert into
SomeDumbTable
select
*
from $carHashTable
"#
Invoke-Sqlcmd -Query $sql
All the examples I've seen online use a foreach loop to do the actual insert or update. Bossmang says loops (in SQL proper) are bad.
The only other option I can think of is using PowerShell to create a temp table in SQL, then use the temp table as the source to do the the multi-insert.
Attempting to answer my own question. If I make an incorrect statement, please correct me.
1) YES. You can convert a hash table using this module created by James Brundage into an object that the SqlServer module can deal with.
#import module that converts PSObjects into a DataTable that SQL can handle (external author)
import-module -name C:\PowerShell\ConvertTo-DataTable.ps1 -Verbose
#import the csv, and convert it to a DataTable
$Jobs = import-csv -Path $Repository\$File -Delimiter '|' | ConvertTo-DataTable
2) For the larger question, can you UPDATE without a foreach loop?, I'm going to say NO, at least not with a SQL #temp table, which was my original intention. Here's what I've observed:
Write-SqlTableData does not appear to support #temp tables
Write-SqlTableData DOES allow me to use the converted PS $object
to write data; however, it only appends data to tables; I don't know
of a way to UPDATE on some key
The only way I was able to achieve this was by gleaning a lot on
StackOverflow, and bringing in the SQL .NET client libraries
System.Data.SqlClient.
Those libraries allowed me to open and maintain a session with SQL so that I could create a #temp table, bulk copy content into it, then finally UPDATEing the real target table.
The SqlServer Modules for PowerShell seems to immediately open and close the session, so any #temp tables created there are only avilable to that session
The session created by the .NET client libraries can be opened and maintained to run multiple queries, but it's separate from sessions created by the SqlServer modules -- PS can't gain access to #temp tables created in .Net sessions, and vice versa.
I'm new to Powershell. I have a bunch of stored procedures that are exactly the same except they reside in different schemas and target different names. Very simply, they look like this (real procs are much longer)
CREATE PROC [a].[ProcessTable] AS UPDATE a.Table_a
CREATE PROC [b].[ProcessTable] AS UPDATE b.Table_b
CREATE PROC [c].[ProcessTable] as UPDATE c.Table_c
I wanted to create just one file (call it 'FileWithStoredProc.sql') with the base stored procedure and then use an outer powershell script to execute it multiple times:
loop through an array of {a, b, c, ... n}
use Invoke-SQLcmd -inputfile "FileWithBasicStoredProc.sql" -Variable $args
Then in my target file, have something like
CREATE PROC [$(arg1)].[ProcessTable] AS UPDATE $(arg1).Table_$(arg1)
I can now see that it's fine as long as I want to use $(arg1) as a string, say
print 'Creating stored proc ' + $(arg1) + '.ProcessTable'
But I want the target file to intepret my variable as part of a sql object name.
How do I do this? I cannot redesign/rename my SQL objects, and since the procedure is quite long, with a lot of these schema swaps, I don't want to use the -Query parameter in my powershell wrapper. Is what I want possible, without changing everything into a giant 'exec sql' string? I'd like to keep these target sql files relatively easy to read and modify.
I have to automate one process to get rid of daily effort reduction in our organization. We need to execute multiple scripts on different SQL Server instances and each script contains database name as well.
So initially our client put all the scripts on a particular location and I need to execute each of the scripts and then move the script file to different folder.
After a script got an error, it logged the error and one file has been generated.
After successful execution I need to generate one log file where all the successful results script wise also getting generated. Like when we execute one script in SSMS, after executing the script it generates a message like "1 row affected".
How can I do that?
invoke-sqlcmd -inputfile "E:\test.sql" -serverinstance ".\Your_Instance_Name" -database "user" | out-File -filepath "E:\result.txt"
The "1 row(s) affected" message is generated automatically, unless set nocount is specified.
A client application, such as Invoke-SqlCmd, SSMS and sqlcmd, can do whatever with the row count number. SSMS and sqlcmd print it per default, Invoke-SqlCmd doesn't seem to do that. This is not a bug, though it's certainly a bit surprising.
The simple approach is to issue an explicit select for ##ROWCOUNT. Like so,
insert mydb.schema.table(column1, column2....) values(...); select ##rowcount;
Or, use sqlcmd.exe instead.
Invoke-Sqlcmd -ServerInstance '.' -Database 'MyDB'
-Query 'EXEC SprocA #param1= "value";EXEC SprocB #param1= "value";'
Basically I have my Invoke-SqlCmd running a query that invokes two stored procedures. Both the stored procedures output a bunch of rows.
However if sprocA does not output any results (empty select results or no rows), then the invoke command does not seem to print the output of the second sprocB even if it has data.
If I change the order of the stored procedures in my Invoke-SqlCmd commands query parameter, then this works perfectly and returns the output of the first stored procedure.
If I had three stored procedure calls where the first returns data and the second does not and the third does, it prints output of the first result and third result.
Basically it does not print any output only if the first stored procedure has no output. Seems weird.
Anything I can do to get around this SQL wise ? Could be a PowerShell thing?
I was also able to repro this with two Select statements where one returns data and the other does not.
This is the documented behavior of invoke-sqlcmd
When this cmdlet is run, the first result set that the script returns
is displayed as a formatted table. If subsequent result sets contain
different column lists than the first, those result sets are not
displayed. If subsequent result sets after the first set have the same
column list, their rows are appended to the formatted table that
contains the rows that were returned by the first result set.
It looks like both result sets are actually returned, but not output by defaut.
EG
PS C:\> Invoke-SqlCmd "select 1 a; select 2 b, 3 c;" | % { $_ | Out-Default }
outputs
a
-
1
b c
- -
2 3
Actually, it is hard to believe MS made such a mistake or maybe it's not a mistake. Whatever, when you run Invoke-SqlCmd and with a query like the following,
select * from table1 where id = 1111 -- non-exists id, this select returns nothing
select * from table2 where id = 2222 -- exists id, this select returns something
On SSMS you can see 2 result sets, the first one is empty.
However, the Invoke-SqlCmd doesn't return anything when the first result set is empty, even other result sets are not. My face was like ?_?
Another approach is to write you own invoke SQL function like following to return whatever result sets, even the empty ones.
$sql_Conn = New-Object System.Data.SqlClient.SQLConnection
$sql_Conn.ConnectionString = $sqlConnectionString
$sql_Conn.Open()
$sql_cmd = New-Object system.Data.SqlClient.SqlCommand($Query, $sql_Conn)
$sql_ds = New-Object system.Data.DataSet
$sql_da = New-Object system.Data.SqlClient.SqlDataAdapter($sql_cmd)
[void]$sql_da.fill($sql_ds)
$sql_Conn.Close()
return $sql_ds
Everything is fine until a new problem comes about the keyword GO in your script, you must know it if you use SSMS. The thing is, this GO is not a SQL command. it is just a separator used by SSMS. MS developed codes can handle the GO in a good manner, e.g. SSMS, Invoke-SqlCmd and sqlcmd.exe. If you use your own SQL invoke function you will get syntax issue
Incorrect syntax near 'GO'
While people most likely to ask you to update the SQL script to remove all GO lines, however, things are not always under controlled, normally need to work with different people and teams.
At last, I have to trim the GO in my scripts like the following
$Query = $Query -ireplace "(^|\r|\n)[ \t]*\bGO\b[ \t]*(\r|\n|$)", '$1$2'
https://github.com/LarrysGIT/Invoke-Sql
Of course, the story is not over, the more I am trying to automate SQL related tasks. The more issue found. There are multiple ways to automatically execute SQL script. None of them are perfect so far.
Invoke-Sql (My own script)
* Is able to handle the key separator 'GO'
* Is able to handle duplicate columns
* Fully support multiple result sets, even the first result set is empty
* Unable to handle `Create or alter` keywords if there are contents ahead
* Unable to handle special characters like '194 160' (non-breaking space) in SQL script (edited by some document edit tool, MS word for example)
Invoke-SqlCmd
* Is able to handle the key separator 'GO'
* Is able to handle special characters like 'non-breaking space'
* Unable to handle duplicate columns
* Unable to fully handle multiple result sets (when the first table is empty)
sqlcmd.exe
* Is able to handle all things (briefly tested)
* The returned result sets are plain text, hard to parse
Microsoft.SqlServer.SMO snapin
* The API of SQL server management studio
* Theoretically should be able to handle all cases
* Need to dig more
I experienced a very strange issue and can be repeated.
Basically, I use invoke-sqlcmd to call a script file by using -inputfile, but if the script file has some execution error (like insert into a table where a column should not be null), the script file will be executed twice. I can see the two executions from profiler as well.
Here is the way to reproduce the issue (My environment: Win 8.1 + SQL2014 + PS 5.0)
create two tables in a database
Use TestDB
create table dbo.s (id int identity primary key, b varchar(50));
create table dbo.t (id int primary key, s_id int, b varchar(50));
alter table dbo.t add constraint fk_t foreign key (s_id) references dbo.s(id)
Now create a sql file (let's call it, c:\temp\t.sql) with the following two lines
insert into dbo.s ( b) select 'hello world'
insert into dbo.t (s_id, b) -- purposely missing id to cause an error
select 1, 'good morning'
Run the following PS cmdlet
invoke-sqlcmd -Server "<my_local_server>" -database TestDB -inputfile "c:\temp\t.sql"
Your PS will return an error, now if you open an SSMS query window and do the following
select * from TestDB.dbo.s
You will see two records there instead of one.
On the other hand, if I run sqlcmd.exe, there is NO such issue, i.e. just one record in dbo.s.
Is there some configuration in SQLPS I missed?
I see you asked this same question on the MSDN Database Engine forum: https://social.msdn.microsoft.com/Forums/en-US/d4167226-2da7-49ec-a5c2-60e964785c2c/powershell-invokesqlcmd-calls-stored-procedure-second-time-after-query-timeout-is-expired. Below is the SMO workaround from that thread.
$SqlServerName = "YourServer";
$DatabaseName = "YourDatabase";
$ScriptFileName = "C:\Scripts\YourSqlScriptFile.sql";
Add-Type -Path "C:\Program Files\Microsoft SQL Server\110\SDK\Assemblies\Microsoft.SqlServer.Smo.dll";
$sr = New-Object System.IO.StreamReader($ScriptFileName);
$script = $sr.ReadToEnd();
$sr.Close();
$Server = New-Object Microsoft.SqlServer.Management.Smo.Server($SqlServerName);
$db = $Server.Databases[$DatabaseName];
$db.ExecuteNonQuery($script);
Updating this tread with the fix from Microsoft:
Cumulative Update 2 for SQL Server 2016 SP1
Cumulative Update 4 for SQL Server 2014 Service Pack 2
FIX: "Invoke-sqlcmd" cmdlet executes a query statement multiple times if an error occurs in SQL Server 2014 or 2016
Even i had a similar issue.
Fixed it by adding -QueryTimeout parameter to Invoke-Sqlcmd cmdlet.
Basically it seems that the query somehow times out and due to a bug in Invoke-Sqlcmd cmdlet it tries to insert it again. Strange but try this. Keep the query timeout big enough for the query to execute.
I know this is a old thread, but maybe it can help someone.
I found out if you move your line "select 1, 'good morning'" in your example before the insert statement, which has the exception, it works like intended.
I had a similar Issue with try catch, that the first return value will be ignored when it's an exception, so I had to make sure that the first line was Select 1. Strange bug.