Powershell Job Step in SQL Error - sql-server

I'm new to PowerShell and still early in SQL Server, but I'm trying to write a PowerShell step for a SQL Agent job that looks at a CSV file that contains names of .sql files.
It should then look at a different directory and if the names from the CSV file exist in that directory it should open the .sql file and execute the function inside.
I'm getting an error:
Unable to cast object of type 'System.String to type System.Type
Any help would be greatly appreciated.
$excelFile = "C:/ExcelTest/Test.csv"
$functionDirectory = "some directory"
$excel_Array = (Get-Content $excelFile)[0].split(",")
foreach ($sqlName in $excel_Array)
{
if($sqlName::exists($functionDirectory + "/" + $sqlName) -ne $true)
{
invoke-sqlcmd -inputfile $functionDirectory + "/" + $sqlName -serverinstance "serverinstance" -database "database"
}
}

If understand the question correctly, you need to use Test-Path not ::exists
$excelFile = "C:/ExcelTest/Test.csv"
$functionDirectory = "some directory"
Import-Csv $excelFile |
Foreach-Object {
$filename = $functionDirectory + '\' + $_[0]
if (Test-Path $filename) {
invoke-sqlcmd -inputfile $filename -serverinstance "serverinstance" -database "database"
}
}

I would adjust a few things in your script to properly handle a CSV and then utilize built-in cmdlet for testing the path of a given file.
[cmdletbinding()]
param()
Import-Module SQLPS -DisableNameChecking
$functionDirectory = "C:\temp\PowerShell_Testing2"
$excelFile = Import-Csv "C:\temp\PowerShell_Testing\SQLFileList.csv"
foreach ($e in $excelFile) {
$fileonly = Split-Path $e.SQLFile -Leaf
$fdFile = $functionDirectory + "\" + $fileonly
if (Test-Path $fdFile) {
Write-Host "Found File $fdFile"
Invoke-Sqlcmd -ServerInstance "MANATARMS\SQL12" -InputFile $fdFile -Database master
}
}
In my setup there is only one file that just runs:
SELECT TOP 1 name FROM sys.databases
If your CSV contains say a ServerName, and then SQLFile you can adjust your script to also pull the ServerInstance value like this:
[cmdletbinding()]
param()
Import-Module SQLPS -DisableNameChecking
$functionDirectory = "C:\temp\PowerShell_Testing2"
$excelFile = Import-Csv "C:\temp\PowerShell_Testing\SQLFileList.csv"
foreach ($e in $excelFile) {
$fileonly = Split-Path $e.SQLFile -Leaf
$fdFile = $functionDirectory + "\" + $fileonly
if (Test-Path $fdFile) {
Write-Host "Found File $fdFile"
Invoke-Sqlcmd -ServerInstance $e.ServerName -InputFile $fdFile -Database master
}
}

Related

how to execute multiple Invoke-Sqlcmd in one transaction?

I would like to perform a bunch of invoke-sqlcmd in one sql transaction. Here's what I'm doing:
try{
$scope = New-Object -TypeName System.Transactions.TransactionScope
GetFiles $SqlFilesDirectory
$scope.Complete()
}
catch{
$_.exception.message
}
finally{
$scope.Dispose()
}
Here's how GetFiles is defined:
#
# Get SQL Files recursively
#
function GetFiles($path = $pwd)
{
$subFolders = Get-ChildItem -Path $path -Directory | Select-Object FullName,Name | Sort-Object -Property Name
$sqlFiles = Get-ChildItem -Path $path -Filter *.sql | Select-Object FullName,Name | Sort-Object -Property Name
foreach ($file in $sqlFiles)
{
Write-Host "file: " $file.Name
Invoke-Sqlcmd -ServerInstance $ServerInstance -Database $DBName -Username $SvcAdminAccount -Password $SvcAdminPassword -InputFile $file.FullName -QueryTimeout 65535
}
foreach ($folder in $subFolders)
{
Write-Host "`nGetting files for subfolder: " $folder.Name
GetFiles $folder.FullName
}
}
How do we perform a series of invoke-sqlcmd in one transaction?
Here's the output:
The behavior that I want is that ALL
changes are rolled back if a single sql script fails.

Export return of SQL script to Excel document using powershell

At the moment I have the following code which grabs the return table and outputs it into a CSV file.
Push-Location; Import-Module SQLPS -DisableNameChecking; Pop-Location
$SQLServer = "localhost"
$today = (get-date).ToString("dd-MM-yyyy")
$DBName = "ZoomBI"
$ExportFile = "\\Shared_Documents\FC Folder\Despatch\Brexit Files\DHL\DHL "+$today+".csv"
$Counter = 0
$Storedprocedure = "EXEC [dbo].[DHLDeliveries]"
while ( $true )
{
# Remove the export file
if (Test-Path -Path $ExportFile -PathType Leaf) {
Remove-Item $ExportFile -Force
}
# Clear the buffer cache to make sure each test is done the same
$ClearCacheSQL = "DBCC DROPCLEANBUFFERS"
Invoke-Sqlcmd -ServerInstance $SQLServer -Query $ClearCacheSQL
# Export the table through the pipeline and capture the run time. Only the export is included in the run time.
$sw = [Diagnostics.Stopwatch]::StartNew()
Invoke-Sqlcmd -ServerInstance $SQLServer -Database $DBName -Query $Storedprocedure | Export-CSV -Path $ExportFile -NoTypeInformation
$sw.Stop()
$sw.Elapsed
$Milliseconds = $sw.ElapsedMilliseconds
$Counter++
Exit
}
However, instead of that I need to be able to output the results to an Excel document with two sheets
and put the results into each sheet.
# Create a Excel Workspace
$excel = New-Object -ComObject Excel.Application
# make excel visible
$excel.visible = $true
# add a new blank worksheet
$workbook = $excel.Workbooks.add()
# Adding Sheets
foreach($input in (gc c:\temp\input.txt)){
$s4 = $workbook.Sheets.add()
$s4.name = $input
}
# The default workbook has three sheets, remove them
($s1 = $workbook.sheets | where {$_.name -eq "Sheet1"}).delete()
#Saving File
"`n"
write-Host -for Yellow "Saving file in $env:userprofile\desktop"
$workbook.SaveAs("$env:userprofile\desktop\ExcelSheet_$Today.xlsx")
Can anyone help ?
I would take a look at the ImportExcel module. It took me 2 lines of code to create an excel document with two sheets.
https://www.powershellgallery.com/packages/ImportExcel/5.4.2
https://www.youtube.com/watch?v=fvKKdIzJCws&list=PL5uoqS92stXioZw-u-ze_NtvSo0k0K0kq

How do you script out SQL Server agent jobs to single or individual files

I've tried various Powershell scripts but they fail with:
The following exception occurred while trying to enumerate the collection: "An exception occurred while executing a Transact-SQL statement or batch.".
At H:\Create_SQLAgentJobSripts2.ps1:89 char:22
foreach ($job in $s.JobServer.Jobs)
~~~~~~~~~~~~~~~~~
CategoryInfo : NotSpecified: (:) [], ExtendedTypeSystemException
FullyQualifiedErrorId : ExceptionInGetEnumerator
What has gone wrong or how can I get better debugging on this error?
I executed this script:
.\Create_SQLAgentJobSripts2.ps1 .\ServerNameList.txt
Here's the script
param([String]$ServerListPath)
#write-host "Parameter: $ServerListPath"
#Load the input file into an Object array
$ServerNameList = get-content -path $ServerListPath
#Load the SQL Server SMO Assemly
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | Out-Null
#Create a new SqlConnection object
$objSQLConnection = New-Object System.Data.SqlClient.SqlConnection
#For each server in the array do the following.
foreach($ServerName in $ServerNameList)
{
Write-Host "Beginning with Server: $ServerName"
Try
{
$objSQLConnection.ConnectionString = "Server=$ServerName;Initial Catalog=CED_NCT_RESOURCE_TRACK;Persist Security Info=True;User ID=CEDNCTAdmin;Password=CEDNCTAdmin;"
Write-Host "Trying to connect to SQL Server instance on $ServerName..." -NoNewline
$objSQLConnection.Open() | Out-Null
Write-Host "Success."
$objSQLConnection.Close()
}
Catch
{
Write-Host -BackgroundColor Red -ForegroundColor White "Fail"
$errText = $Error[0].ToString()
if ($errText.Contains("network-related"))
{Write-Host "Connection Error. Check server name, port, firewall."}
Write-Host $errText
continue
}
# Won't be using this object again
Remove-Variable -Name objSQLConnection
#If the output folder does not exist then create it
$OutputFolder = ".\$ServerName"
if (!(Test-Path $OutputFolder))
{
write-host ("Creating directory: " + $OutputFolder)
New-Item -ItemType directory -Path $OutputFolder
}
else
{
write-host ("Directory already exists: " + $OutputFolder)
}
write-host "File: $(".\$OutputFolder\" + $($_.Name -replace '\\', '') + ".job.sql")"
# Connect to the instance using SMO
$s = new-object ('Microsoft.SqlServer.Management.Smo.Server') $ServerName
write-host ("SQL Server Edition: " + $s.Edition)
write-host ("SQL Agent ErrorLogFile: " + $s.JobServer.ErrorLogFile)
# Instantiate the Scripter object and set the base properties
$scrp = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') ($ServerName)
write-host ("SCRP ToString():" + $scrp.ToString())
write-host ("Test scrp - Server: " + $scrp.Server)
#The next step is to set the properties for the script files:
$scrp.Options.ScriptDrops = $False
$scrp.Options.WithDependencies = $False
$scrp.Options.IncludeHeaders = $True
$scrp.Options.AppendToFile = $False
$scrp.Options.ToFileOnly = $True
$scrp.Options.ClusteredIndexes = $True
$scrp.Options.DriAll = $True
$scrp.Options.Indexes = $False
$scrp.Options.Triggers = $False
$scrp.Options.IncludeIfNotExists = $True
#Now, we can cycle through the jobs and create scripts for each job on the server.
# Create the script file for each job
foreach ($job in $s.JobServer.Jobs)
{
$jobname = $job.Name
write-host ("Job: " + $jobname)
$jobfilename = ($OutputFolder + "\" + $jobname + ".job.sql")
$scrp.Options.FileName = $jobfilename
write-host "Filename: $jobfilename"
#This line blows up
$scrp.Script($job)
}
}
Possibly you're not instantiating the Server object correctly. Try the following instead...
# Alternative 1: With servername and port, using Trusted Connection...
$ServerName = 'YourServerName,1433'
$ServerConnection = New-Object Microsoft.SqlServer.Management.Common.ServerConnection -ArgumentList #( $ServerName )
# Alternative 2: With an SqlConnection object
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=$ServerName;Initial Catalog=CED_NCT_RESOURCE_TRACK;Persist Security Info=True;User ID=CEDNCTAdmin;Password=CEDNCTAdmin;"
$SqlConnection.Open() | Out-Null
$ServerConnection = New-Object Microsoft.SqlServer.Management.Common.ServerConnection -ArgumentList #( $SqlConnection )
# Then...
$Server = New-Object Microsoft.SqlServer.Management.Smo.Server -ArgumentList #( $ServerConnection )
$Server.JobServer.Jobs | ForEach-Object {
Write-Host "Job: $($_.Name)"
}

how to write powershell script to list the available databases from multiple servers?

Requirement : My requirement is I have to list the available databases from 150 servers. Each server has minimum 1 and maximum 15 instances.
Below script is working only for instances listed in sqlserver.txt but I need to fetch multiple instances across multiple servers.
Help is highly appriciated.
ForEach ($instance in Get-Content "C:\PowerSQL\SQL_Servers.txt")
{
Import-Module SQLPS -DisableNameChecking
Invoke-SQLcmd -Server $instance -Database master 'select ##servername as InstanceName,name as DatabaseName,state_desc as DBStatus from sys.databases' | Format-Table
}
You can use this script to find all reachable instances on your network and running your query there:
Import-Module SQLPS -DisableNameChecking
$servers = [System.Data.Sql.SqlDataSourceEnumerator]::Instance.GetDataSources()
ForEach ($i in $servers) {
$instance = $i.ServerName+"\"+$i.InstanceName
Invoke-SQLcmd -Server $instance -Database master 'select ##servername as InstanceName,name as DatabaseName,state_desc as DBStatus from sys.databases' | Format-Table
}
If you need only server name to pass then use $instance = $i.ServerName. Part of code was taken from here long time ago.
EDIT
With writing in CSV file and error catching:
Import-Module SQLPS -DisableNameChecking
$servers = [System.Data.Sql.SqlDataSourceEnumerator]::Instance.GetDataSources()
$results = #()
ForEach ($i in $servers) {
$instance = $i.ServerName+"\"+$i.InstanceName
try {
$sqlres = Invoke-SQLcmd -Server $instance -Database master 'select ##servername as InstanceName,name as DatabaseName,state_desc as DBStatus from sys.databases'
ForEach($st in $sqlres) {
$instanceinfo = #{
InstanceName = $st.InstanceName
DatabaseName = $st.DatabaseName
DBStatus = $st.DBStatus
}
$results += New-Object PSObject -Property $instanceinfo
}
} catch {
"error when running Invoke-SQLcmd "+$instance
Write-Host($error)
}
}
$results | export-csv -Path D:\sql_instances_info.csv -NoTypeInformation
Im not sure what is the problem here. You can put all servers/instances in txt file and iterate:
#array of addresses, this can be fetched from file
$list = "localhost\SQL2014",".\SQL2014","(local)\SQL2014" #MyServer\MyInstance
$list | `
% { Invoke-Sqlcmd -Server $_ -Database master 'select ##servername as InstanceName,name as DatabaseName,state_desc as DBStatus from sys.databases' } | `
Format-Table -AutoSize
If those are remote servers without integrated security you would need to pass -UserName and -Password arguments.

How to create only specific delete statements using Scripter

I am using the Scripter class to give me a script for the data out of an existing database. I want to script a dataset that can be inserted into a production database. We are doing this to test if an installation of our Software is correct.
Unfortunately the dataset has to be removed later without any entries left behind so that it does not interfere with the data of our customers. So what I need are INSERT and DELTE statements. These are maintained manually at the moment which is too much of a burden.
Very well so I just went and executed the Scripter twice (once for INSERT, once for DELETE)
Problem is that when setting ScriptDrops to true then the output is in the form
DELETE FROM [dbo].[TableName]
What I would like is something of the form:
DELETE FROM [dbo].[TableName] WHERE ID = 'GUID'
Technically this would be possible since there are Primary Keys on all the tables.
The Scripter class must also in some form know of that things since it also gets the order of the DELETE-statements (dependencies) correct via foreign keys.
Any help on this would be appreciated.
Following are the 2 PowerShell-scripts I am using to export the data:
ScriptRepositoryData.ps1
$scriptPath = $MyInvocation.MyCommand.Path
$scriptDirectory = Split-Path $scriptPath -Parent
. $scriptDirectory\DatabaseScripting.ps1
$filepath='c:\data.sql'
$database='ECMS_Repository'
$tablesToExclude = #(
"SomeUnwantedTable"
)
$tablesListFromDatabase = GetTableList $database
$tablesArray = #()
$tablesListFromDatabase |% {
if (-not $tablesToExclude.Contains($_.Name.ToString()))
{
$tablesArray += $_.Name
}
}
ScriptInsert $database $tablesArray $filepath
DatabaseScripting.ps1
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMOExtended") | out-null
Function GetTableList ($database)
{
Invoke-SqlCmd -Database $database -query "SELECT * FROM sys.tables"
}
Function ScriptInsert ($database, $tables, $destination)
{
try {
$serverMO = new-object ("Microsoft.SqlServer.Management.Smo.Server") "localhost"
if ($serverMO.Version -eq $null) {Throw "Can't find the instance localhost"}
$urnsToScript = New-Object Microsoft.SqlServer.Management.Smo.UrnCollection
$databaseMO = $serverMO.Databases.Item("ECMS_Repository")
if ($databaseMO.Name -ne $database) {Throw "Can't find the database $database"}
$tables |% {
$tableListMO = $databaseMO.Tables.Item($_, "dbo")
$tableListMO |% {
$urnsToScript.Add($_.Urn)
}
}
$scripter = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') $serverMO
$scripter.Options.ScriptSchema = $False;
$scripter.Options.ScriptData = $true;
$scripter.Options.ScriptDrops = $true;
$scripter.Options.ScriptAlter = $true;
$scripter.Options.NoCommandTerminator = $true;
$scripter.Options.Filename = $destination;
$scripter.Options.ToFileOnly = $true
$scripter.Options.Encoding = [System.Text.Encoding]::UTF8
$scripter.EnumScript($urnsToScript)
Write-Host -ForegroundColor Green "Done"
}
catch {
Write-Host
Write-Host -ForegroundColor Red "Error occured"
Write-Host
Write-Host $_.Exception.ToString()
Write-Host
}
}
Unfortunately I did not find a way to do this using the Sql Management Objects.
Anyhow I now use the output of the Scripter and select the IDs of each table. I then use the IDs to change every line that looks like
DELETE FROM [dbo].[tableName]
to this:
DELETE FROM [dbo].[tableName] WHERE ID IN ('guid1', 'guid2')
Here is how I did it:
$content = Get-Content $destination
Clear-Content $destination
$content |% {
$line = $_
$table = $line.Replace("DELETE FROM [dbo].[","").Replace("]","")
$query = "SELECT ID, ClassID FROM" + $_
$idsAsQueryResult = Invoke-SqlCmd -Database $database -query $query
$ids = $idsAsQueryResult | Select-Object -Expand ID
if ($ids -ne $null) {
$joinedIDs = [string]::Join("','",$ids)
$newLine = $line + " WHERE ID IN ('" + $joinedIDs + "')"
Add-Content $destination $newLine
}
}
Where $destination is the script that has been generated with the Scripter class and $database is a string containing the database name.
I had to select a second column (ClassID which is there on all tables due to our OR mapper re-store) because of some weird error in Select-Object which I do not fully understand.
This of course only works because all tables have primary keys and all primary keys are named ID and are not combined primary keys or something.
You could of course achieve the same thing for other more complicated database schemas by extracting primary key information via SQL management objects.

Resources