I have a query that compiles some data based on a supplied Id number. I want to run it once for a long list of Ids that I have, and save the result set to a CSV. I know I can do it manually, but I'm looking for an automated way.
I already tried running my query without specifying the Id in the where clause. This ended up giving my a file a little smaller than 1Gb which is too big for needs.
Here is the basic idea of what I am trying to accomplish:
Declare #num int
set #num = 0
While #num < 100
Begin
--I'm trying to figure out a way to store the
--result set generateed by this procedure is saved to 'DataExtract' + #num + '.csv'
Exec LongRunningProcedure #num
Set #num = #num + 1
End
If you have sufficient permissions you could use powershell to accomplish this. I can't really be sure that this will run exactly right for you because I don't know what your output looks like, but hopefully this will get you where you're going.
$SQLAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SQLCommand = New-Object system.Data.SqlClient.SqlCommand
$DataSet = New-Object System.Data.DataSet
[STRING]$SQLServer = 'YourServer';
[STRING]$SQLDatabase = 'YourDatabase';
[STRING]$SQLConnectString = "Data Source=$SQLServer; Initial Catalog=$SQLDatabase; Integrated Security=True";
$SQLConnection = New-Object System.Data.SqlClient.SqlConnection($SQLConnectString)
$extractFile = "C:\test.csv"
for($x = 0; $x -le 100; $x++) {
$file = "$($extractFile.Split('.')[0])$($x).csv"
$SQLQuery = "Exec LongRunningProcedure #num = $x"
$SQLConnection.Open()
$SQLCommand.CommandText = $SQLQuery
$SQLCommand.Connection = $SQLConnection
$SQLAdapter.SelectCommand = $SQLCommand
$SqlAdapter.Fill($DataSet) | Out-Null
$SQLConnection.Close()
$DataSet.Tables[0] | Export-Csv $file
}
Related
In PowerShell I'm attempting to write an if statement on whether a COUNT result from SQL Server is greater than 0. Below is what I have:
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Our Connection String"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = "SELECT COUNT(*) FROM dbo.Source WHERE SourceId IS NULL"
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$Table = New-Object System.Data.DataTable
$SqlAdapter.Fill($Table) | Out-Null
if ($Table > 0) {
Write-Output "The Count is greater than zero, please investigate"
}
I know the if statement at the bottom is bad syntax, and I've tried many other ways of expressing it to no avail. I run the script, I don't get any errors, but it just completes and nothing happens. I have verified that if I remove the Out-Null from the $SqlAdapter.Fill() line, PowerShell will return a count number, so I know everything up until that point is working.
I just need to get it to write output if the count is greater than 1, which in this SQL query, it definitely is. How would I write this if statement?
This should do what you want.
if ($Table.Column1 -gt 0) {
Write-Output "The Count is greater than zero, please investigate"
}
Since you are not providing any schema for your data table, the count will go into column name Column1.
Simply capture the output of $SqlAdapter.Fill($Table) in a variable instead of suppressing it via Out-Null and then check that variable.
$cnt = $SqlAdapter.Fill($Table)
if ($cnt -gt 0) {
# do stuff
}
If I want to insert the values in the array in one connection, how can i modify the code below? So far i get the error "There is already an open DataReader associated with this Command which must be closed first" unless I put the $Connection.Open() and $Connection.Close() inside the for loop, which will cost me the speed of a single connection.
$list = 'aaa','bbb','cccc','ddddd','eeeee','ffff'
$Connection = New-Object System.Data.SQLClient.SQLConnection
$Connection.ConnectionString = "server='$Server';database='$Database';trusted_connection=true;"
$Connection.Open()
$Command = New-Object System.Data.SQLClient.SQLCommand
$Command.Connection = $Connection
foreach($i in $list) {
$sql ="if not exists (select 1 from [bfs] where [key] = '$i' )
begin
insert bfs
select '$i'
end
"
$Command.CommandText = $sql
$Command.ExecuteReader()
}
$Connection.Close()
Try using ExecuteNonQuery() instead of ExecuteReader(). ExecuteNonQuery() does not build a DataReader.
...
$Command.CommandText = $sql
$Command.ExecuteNonQuery()
...
In the code below, I'm trying to query a DB with multiple select statements using variables brought in from a csv and load a data-table using a reader.
The code runs without error but does not retrieve any data.
$csv = Import-Csv $filepath
$database = "DBNAME"
$connectionString = "Server=$dataSource;uid=$user; pwd=$pwd;Database=$database;Integrated Security=True;"
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
$connection.Open()
$sqlCommand = $connection.CreateCommand()
$Datatable = New-Object System.Data.DataTable
ForEach ($row in $csv){
$query = "Select Emailaddress,Column2 from Users Where [Emailaddress] = '$row.Email'"
$sqlCommand.CommandText = $query
$DataReader = $sqlCommand.ExecuteReader()
$DataTable.Load($DataReader)
}
$DataTable | export-csv "c:\Output\Seereader.csv" -NoTypeInformation
$connection.Close()
This:
$query = "Select Emailaddress,Column2 from Users Where [Emailaddress] = '$row.Email'"
Should probably be this:
$query = "Select Emailaddress,Column2 from Users Where [Emailaddress] = '$($row.Email)'"
Whilst the previous answer works, it is vulnerable to SQL injection.
Obligatory xkcd
If you're not sure what "SQL Injection" is; it's only a very worthy Google away...
i.e. you really need to go and find out!
The correct way...
Parameterise your queries!
# Your query; with a #param
$Query = "SELECT Emailaddress, Column2 FROM [Users] WHERE [Emailaddress] = #emailAddress";
# Set up your basic command
$command = $connection.CreateCommand()
$command.CommandText = $Query
# Fill in the parameters!
$command.Parameters.AddWithValue("#emailAddress", $row.Email)
# Run boy, run!
$results = $command.ExecuteReader()
# Resultification (that's definitely not a made up word)
$table = New-Object System.Data.DataTable
$table.Load($results)
Safe and sound :-)
I have a Powershell Function that is being used to run multiple queries in SQL and export as CSVs. Each of these queries relies on a date variable. Is there a way to pass this date variable from Powershell into these SQL Scripts (not stored procedures) using my current setup? Any help is much appreciated!
Function Run-Query
{
param([string[]]$queries,[string[]]$sheetnames)
Begin
{
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; User ID = $uid; Password = $pwd;"
Write-host "Connection to database successful."
}#End Begin
Process
{
# Loop through each query
For($i = 0; $i -lt $queries.count; $i++)
{
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
# Use the current index ($i) to get the query
$SqlCmd.CommandText = $queries[$i]
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
# Use the current index ($i) to get the sheetname for the CSV
$DataSet.Tables[0] #| Export-Csv -NoTypeInformation -Path "C:\Users\mbaron\Downloads\$($sheetnames[$i]).csv"
}
}#End Process
End
{
$SqlConnection.Close()
}
}#End function run-query.
You could add a marker in your queries where the data is being used, then do a replace with the relevant date, e.g.:
cls
$date = '1/1/2016'
$query = 'some $$marker$$ script'
$query = $query.replace('$$marker$$', $date )
$query
I'm trying to call a SQL Server stored procedure from PowerShell but I always get errors on parameters.
Stored procedure has 8 parameters, all with default values
#simchain nvarchar
#idSimulation int
#idCompany varchar
#modelName nvarchar
#simDate datetime
#mySim int
#statusFloor int
#statusCap int
From Management Studio I can call this procedure even without any parameter, so just executing EXEC [dbo].[E_simulations] works.
From PowerShell I create a connection and a command but I always get an error on missing parameters, for example
Procedure or function 'E_simulations' expects parameter '#simchain', which was not supplied.
Here is my test code (just to test proper execution)
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection;
$SqlConnection.ConnectionString = $ConnectionString;
$SqlCommand = $SqlConnection.CreateCommand();
$SqlCommand.CommandText = "EXEC [dbo].[E_simulations]";
$SqlConnection.Open();
$returnedValue = $SqlCommand.ExecuteNonQuery();
Am I missing something?
I have made quick test. I hope this will help
Made test proc:
CREATE PROC doTest (
#param1 INT = 1,
#param2 VARCHAR(10) = 'xxx'
)
AS
BEGIN
PRINT 'THIS ONE'
SELECT 1 As Data
END
Find PowerShell code, which executes proc:
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=localhost;Database=ForTests;Integrated Security=True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = "doTest"
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
Execute result gave this
Data
----
1
I done everything same what you wrote and I have result without assigning params