How to create only specific delete statements using Scripter - sql-server

I am using the Scripter class to give me a script for the data out of an existing database. I want to script a dataset that can be inserted into a production database. We are doing this to test if an installation of our Software is correct.
Unfortunately the dataset has to be removed later without any entries left behind so that it does not interfere with the data of our customers. So what I need are INSERT and DELTE statements. These are maintained manually at the moment which is too much of a burden.
Very well so I just went and executed the Scripter twice (once for INSERT, once for DELETE)
Problem is that when setting ScriptDrops to true then the output is in the form
DELETE FROM [dbo].[TableName]
What I would like is something of the form:
DELETE FROM [dbo].[TableName] WHERE ID = 'GUID'
Technically this would be possible since there are Primary Keys on all the tables.
The Scripter class must also in some form know of that things since it also gets the order of the DELETE-statements (dependencies) correct via foreign keys.
Any help on this would be appreciated.
Following are the 2 PowerShell-scripts I am using to export the data:
ScriptRepositoryData.ps1
$scriptPath = $MyInvocation.MyCommand.Path
$scriptDirectory = Split-Path $scriptPath -Parent
. $scriptDirectory\DatabaseScripting.ps1
$filepath='c:\data.sql'
$database='ECMS_Repository'
$tablesToExclude = #(
"SomeUnwantedTable"
)
$tablesListFromDatabase = GetTableList $database
$tablesArray = #()
$tablesListFromDatabase |% {
if (-not $tablesToExclude.Contains($_.Name.ToString()))
{
$tablesArray += $_.Name
}
}
ScriptInsert $database $tablesArray $filepath
DatabaseScripting.ps1
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | out-null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMOExtended") | out-null
Function GetTableList ($database)
{
Invoke-SqlCmd -Database $database -query "SELECT * FROM sys.tables"
}
Function ScriptInsert ($database, $tables, $destination)
{
try {
$serverMO = new-object ("Microsoft.SqlServer.Management.Smo.Server") "localhost"
if ($serverMO.Version -eq $null) {Throw "Can't find the instance localhost"}
$urnsToScript = New-Object Microsoft.SqlServer.Management.Smo.UrnCollection
$databaseMO = $serverMO.Databases.Item("ECMS_Repository")
if ($databaseMO.Name -ne $database) {Throw "Can't find the database $database"}
$tables |% {
$tableListMO = $databaseMO.Tables.Item($_, "dbo")
$tableListMO |% {
$urnsToScript.Add($_.Urn)
}
}
$scripter = new-object ('Microsoft.SqlServer.Management.Smo.Scripter') $serverMO
$scripter.Options.ScriptSchema = $False;
$scripter.Options.ScriptData = $true;
$scripter.Options.ScriptDrops = $true;
$scripter.Options.ScriptAlter = $true;
$scripter.Options.NoCommandTerminator = $true;
$scripter.Options.Filename = $destination;
$scripter.Options.ToFileOnly = $true
$scripter.Options.Encoding = [System.Text.Encoding]::UTF8
$scripter.EnumScript($urnsToScript)
Write-Host -ForegroundColor Green "Done"
}
catch {
Write-Host
Write-Host -ForegroundColor Red "Error occured"
Write-Host
Write-Host $_.Exception.ToString()
Write-Host
}
}

Unfortunately I did not find a way to do this using the Sql Management Objects.
Anyhow I now use the output of the Scripter and select the IDs of each table. I then use the IDs to change every line that looks like
DELETE FROM [dbo].[tableName]
to this:
DELETE FROM [dbo].[tableName] WHERE ID IN ('guid1', 'guid2')
Here is how I did it:
$content = Get-Content $destination
Clear-Content $destination
$content |% {
$line = $_
$table = $line.Replace("DELETE FROM [dbo].[","").Replace("]","")
$query = "SELECT ID, ClassID FROM" + $_
$idsAsQueryResult = Invoke-SqlCmd -Database $database -query $query
$ids = $idsAsQueryResult | Select-Object -Expand ID
if ($ids -ne $null) {
$joinedIDs = [string]::Join("','",$ids)
$newLine = $line + " WHERE ID IN ('" + $joinedIDs + "')"
Add-Content $destination $newLine
}
}
Where $destination is the script that has been generated with the Scripter class and $database is a string containing the database name.
I had to select a second column (ClassID which is there on all tables due to our OR mapper re-store) because of some weird error in Select-Object which I do not fully understand.
This of course only works because all tables have primary keys and all primary keys are named ID and are not combined primary keys or something.
You could of course achieve the same thing for other more complicated database schemas by extracting primary key information via SQL management objects.

Related

Convert JSON array to single JSON object in Powershell

I am trying to convert JSON array into single JSON Object.
Below is the output screenshot
I want the output as
I am using the below powershell script
$getdb_Conn=Invoke-Sqlcmd -ServerInstance $ServerName -Database 'master' -Username $UserName -Password $Password -Query $getdb
$deadlockDB=$getdb_Conn.Database_Name
$deadlockSP=$getdb_Conn.SP_Name
$deadlockTable_Name=$getdb_Conn.Table_Name
$deadlockTIMESTAMP=$getdb_Conn.TIMESTAMP
$Obj = [PSCustomObject]#{
Database = $deadlockDB
SP = $deadlockSP
Table = $deadlockTable_Name
TIMESTAMP = $deadlockTIMESTAMP
}
Write-Output ( $obj | ConvertTo-Json)
Please someone help me, on how to the get required output. I do not want the JSON in an array.
The issue is you have one single non-array object that contains properties, which all contain arrays of values. To achieve the desired results, you need an array of objects that each contain one set of property values. You can choose to do this at the source when you are building $Obj or build a new set of objects to be sent to ConvertTo-Json.
For building a new set of objects using your current $Obj:
for ($i = 0; $i -lt $obj.TIMESTAMP.Count; $i++) {
[pscustomobject]#{
Database = $obj.Database[$i]
SP = $obj.SP[$i]
Table = $obj.Table[$i]
TIMESTAMP = $obj.TIMESTAMP[$i]
} | ConvertTo-Json
}
Building the objects from the source data:
$getdb_Conn=Invoke-Sqlcmd -ServerInstance $ServerName -Database 'master' -Username $UserName -Password $Password -Query $getdb
foreach ($conn in $getdb_Conn) {
[PSCustomObject]#{
Database = $conn.Database_Name
SP = $conn.SP_Name
Table = $conn.Table_Name
TIMESTAMP = $conn.TIMESTAMP
} | ConvertTo-Json
}

SMO to script just the database and it settings (not the objects in it)

I have put together a nice PowerShell script to script out the objects (tables, functions, sprocs etc) from a database, limiting it to the ones in a list.
But I am stuck trying to find a way to script the database itself. Each time I do that, it seems to try to script out the whole database (it is way to large for that to go well).
Assuming I have a $db variable that is a reference to my database, how can I use SMO to script out that database, creating it with the same Properties and DatabaseScopedConfigurations, but none of the actual objects in it?
Update:
For reference here is my current script. It takes a server and database name and will script out all the objects found in a file called DbObjectsList.txt (assuming they are in the database). But this does not actually make the database. The database I am running this on is a legacy one, and it has a bunch of odd options set. I would like to preserve those.
$serverName = "MyServerName"
$databaseName = "MyDbName"
$date_ = (date -f yyyyMMdd)
$path = ".\"+"$date_"
# Load the Sql Server Management Objects (SMO) and output to null so we don't show the dll details.
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') > $null
# Setup the scripting options
$scriptOptions = new-object ('Microsoft.SqlServer.Management.Smo.ScriptingOptions')
$scriptOptions.ExtendedProperties = $true
$scriptOptions.AnsiPadding = $true
$scriptOptions.ClusteredIndexes = $true
# Dri = Declarative Referential Integrity
$scriptOptions.DriAll = $true
$scriptOptions.Triggers = $true
$scriptOptions.NoCollation = $false
$scriptOptions.SchemaQualify = $true
$scriptOptions.ScriptSchema = $true
$scriptOptions.EnforceScriptingOptions = $true
$scriptOptions.SchemaQualifyForeignKeysReferences = $true
$scriptOptions.NonClusteredIndexes = $true
$scriptOptions.Statistics = $true
$scriptOptions.Permissions = $true
$scriptOptions.OptimizerData = $true
# get a reference to the database we are going to be scripting from
$serverInstance = New-Object ('Microsoft.SqlServer.Management.Smo.Server') $serverName
$db=$serverInstance.Databases | Where-Object {$_.Name -eq $databaseName}
$dbname = "$db".replace("[","").replace("]","")
$dbpath = "$path"+ "\"+"$dbname" + "\"
if ( !(Test-Path $dbpath))
{
$null=new-item -type directory -name "$dbname"-path "$path"
}
# Load the list of db objects we want to script.
$listPath = ".\DbObjectList.txt"
if ((Test-Path $listPath))
{
$dbListItems = Get-Content -Path $listPath
}
else
{
throw "Could not find DbObjectst.txt file (it should have a list of what to script)."
}
# Setup the output file, removing any existing one
$outFile = "$dbpath" + "FullScript.sql"
if ((Test-Path $outFile)){Remove-Item $outFile }
$typeDelimiter = "=========="
foreach ($dbListItem in $dbListItems)
{
# Let the caller know which one we are working on.
echo $dbListItem
if ($dbListItem.StartsWith($typeDelimiter))
{
# Pull the type out of the header
$startIndex = $typeDelimiter.Length;
$stopIndex = $dbListItem.LastIndexOf($typeDelimiter)
$type = $dbListItem.Substring($startIndex, $stopIndex - $startIndex).Trim()
continue;
}
if ($type -eq $null)
{
throw "Types not included DbObjectsList.txt. Add types before groups of objects, surrounded by " + $typeDelimiter
}
foreach ($dbObjectToScript in $db.$type)
{
$objName = "$dbObjectToScript".replace("[","").replace("]","")
$compareDbListItem = "$dbListItem".replace("[","").replace("]","")
if ($compareDbListItem -eq $objName)
{
"-- " + $dbListItem | out-File -Append $outFile
$dbObjectToScript.Script($scriptOptions)+"GO" | out-File -Append $outFile
}
}
}

Powershell - proper way to execute SQL query with multiple select statements and result tables

I'm trying to execute an SQL query with few select statements, that returns multiple tables as a result. The problem is that I can't find a way to read and use the tables separately.
Expected results:
Actual results: (it is printed row by row)
Purpose: I've made a script that creates an empty excel file with multiple sheets and each of the sheets will be used to contain each resultset of the query.
The only thing left is to put the needed text into the sheets. Here is my code for that part only:
$ConnectionString = "Data Source=...;Initial Catalog=...;User Id=...;Password=..."
$DBServerName = $ConnectionString.split('=')[1].split(';')[0]
$DBName = $ConnectionString.split('=')[2].split(';')[0]
$DBUser = $ConnectionString.split('=')[3].split(';')[0]
$DBPassword = $ConnectionString.split('=')[4].split(';')[0]
$CurrentFilePath = "C:\SQLqueryWithManyResultsets.sql"
$query = Get-Content -literalPath $CurrentFilePath | Out-String #getting the query string from file
$resultTables = Invoke-Sqlcmd -Query $query -ServerInstance $DBServerName -Database $DBName -DisableVariables -Password $DBPassword -Username $DBUser -ErrorAction Stop
foreach ($result in $resultTables) {
$result | Format-Table #where the magic happens
}
I've made a lot of research, but I cannot find a proper way to store and read the tables the way i need.
Try this:
Clear-Host;
$objConnection = New-Object System.Data.SqlClient.SqlConnection;
$objConnection.ConnectionString = "...";
$ObjCmd = New-Object System.Data.SqlClient.SqlCommand;
$ObjCmd.CommandText = "...";
$ObjCmd.Connection = $objConnection;
$ObjCmd.CommandTimeout = 0;
$objAdapter = New-Object System.Data.SqlClient.SqlDataAdapter;
$objAdapter.SelectCommand = $ObjCmd;
$objDataSet = New-Object System.Data.DataSet;
$objAdapter.Fill($objDataSet) | Out-Null;
for ($i=0; $i -lt $objDataSet.Tables.Count; $i++) {
Write-Host ($objDataSet.Tables[$i] | Format-Table | Out-String);
}
$query = $null;
$objDataSet = $null;
$objConnection.Close();
$objConnection = $null;

Powershell / SQL Server / Import-CSV

Working on another script for work and I'm attempting to read from a CSV containing only one column of data. And then for each item to find the corresponding ID when querying the SQL database. Then to put the result(ID1, CSVID1) in to an excel file(I have this part working fine).
Now I have run in to an issue as to how to populate the dataset within a foreach loop.
$excelAssets = Import-Csv .\test.csv -Header assetId | Foreach-Object {
$assetId = $_.assetId
# SQL Query Variables
$query = "SELECT AssetId AS AssetID, BrandId AS BrandID FROM [AssetLibrary_BrandAsset] WHERE AssetId = $assetId"
$connection = New-SqlConnection -Server $dataSource -Database $dataBase
#Execute the SQL commands and place the results in dataset
if ($connection.State -eq 'Open')
{
$swLap = Start-Elapsed $sw "Executing SQL Query"
Write-Verbose "$query";
$dataSet += Invoke-SQLQuery -connection $connection -query $query -ExecutionTimeout '0'
$i++
$connection.Close();
End-Elapsed $sw $swLap
} ELSE {
Write-Error "$($(Format-Elapsed $swLap)) SQL Connection Not Open - Exiting...";
exit;
}
}
Now $dataSet += doesn't work and I have googled numerous times to try and find the answer to this problem. Any help is appreciated.
Using the $dataSet
$dataTable = new-object "System.Data.DataTable" "Results"
$dataTable = $dataSet.Tables[0]
$rowDT = $dataTable.Rows.Count;
$colDT = $dataTable.Columns.Count;
Write-Host -NoNewLine "$(Format-Elapsed $sw.Elapsed) Rows: ";
Write-Host -NoNewLine "$($rowDT+1)" -ForegroundColor "Green";
Write-Host -NoNewLine " Columns: "
Write-Host -NoNewLine "$($colDT+1)" -ForegroundColor "Green";
Write-Host -NoNewLine " Cells: "
Write-Host "$( ($colDT+1)*($rowDT+1) )" -ForegroundColor "Green";
#Create a 2D Array of the DataTable
# http://stackoverflow.com/questions/13184191/fastest-way-to-drop-a-dataset-into-a-worksheet
$tableArray = New-Object 'object[,]' $rowDT, $colDT;
$swLap = Start-Elapsed $sw "DataTable transformation"
# i = row and j = column
for ($i=0;$i -lt $rowDT; $i++)
{
#Write-Progress -Activity "Transforming DataTable" -status "Row $i" -percentComplete ($i / $rowDT*100)
for ($j=0;$j -lt $colDT; $j++)
{
$tableArray[$i,$j] = $dataTable.Rows[$i].Item($j).ToString();
}
}
End-Elapsed $sw $swLap
$rowOffset = 1; $colOffset = 1;# 1,1 = "A1"
# Write out the header column names
for ($j=0;$j -lt $colDT; $j++)
{
$ActiveWorksheet.cells.item($rowOffset, $j+1) = $dataTable.Columns[$j].ColumnName;
}
$headerRange = $ActiveWorksheet.Range($ActiveWorksheet.cells.item($rowOffset, $colOffset), $ActiveWorksheet.cells.item($rowOffset, $colDT+$colOffset-1));
$headerRange.Font.Bold = $false
$headerRange.Interior.Color = $headingColour
$headerRange.Font.Name = $headingFont
$headerRange.Font.Color = $headingFontColour
$rowOffset++;
# Extract the data to Excel
$tableRange = $ActiveWorksheet.Range($ActiveWorksheet.cells.item($rowOffset, $colOffset), $ActiveWorksheet.cells.item($rowDT+$rowOffset-1, $colDT+$colOffset-1));
$tableRange.Cells.Value2 = $tableArray;
# Resize the columns in Excel
$swLap = Start-Elapsed $sw "Resize Excel Worksheet"
$wholeRange = $ActiveWorksheet.UsedRange
$wholeRange.EntireColumn.AutoFit() | Out-Null
End-Elapsed $sw $swLap
# Save Excel workbook
$ActiveWorkbook.SaveAs("$OutputFile")
$ActiveWorkbook.Close()
After assigning to $dataSet the first time, it's type is probably not array, meaning that the += operator doesn't behave exactly as you expect.
You can either initialize $dataSet as an empty array before you start assigning to it:
Import-Csv .\test.csv -Header assetId | Foreach-Object -Begin {$dataSet = #()} -Process {
# rest of script in here
} -End {return $dataSet}
or you can cast it during assigning:
[array]$dataSet += Invoke-SQLQuery -connection $connection -query $query -ExecutionTimeout '0'
finally, an alternative solution would be to ensure that the output from Invoke-SQLQuery is treated as an array before you assign it to $dataSet:
$dataSet += #(Invoke-SQLQuery -connection $connection -query $query -ExecutionTimeout '0')
Whatever suits your style of coding.

Fine-tuning Powershell SQL Script

My company has a program that tracks our Employee workouts. When we had this program made, we did not think about adding the ability to Add or Remove an employee to the program.
I wrote a script in PowerShell that allows us to do this easier than in SSMS. I would like to see if anyone can help me clean it up a bit and fine tune it.
My biggest headache is this 1 or -1 that gets returned anytime we execute a function. I would also like this to ask if they are finished, then loop back or exit. Right now it just exits as soon as they are done.
<#Writes the invoker to log#>
$trandate = Get-Date
$tranuser = $env:UserName
<# Variables to open the connection to the SQL server #>
$sqlcn = New-Object System.Data.SqlClient.SqlConnection
$sqlcn.ConnectionString = "server=10.10.1.19\VTSWORKOUT;Integrated
Security=true;Database=VTSWORKOUT;"
<# Read what the user wants to do #>
$input = Read-Host "Do you want to [A]dd a New Employee, [R]emove an Employee or [E]xit?"
switch($input){
<# Stuff for adding an employee to the database #>
A{
$eid = Read-Host "What is the Employees ID number?"
$fname = Read-Host "What is the Employees first name?"
$lname = Read-Host "What is the Employees last name?"
$dept = Read-Host "What department is the Employee in?"
$pay = Read-Host "Is the Employee Salaried? [0]Yes or [1]No"
$hire = Read-Host "When was the Employee hired? Input as MM-DD-YYYY"
Out-File -FilePath "L:\Personnel\WorkoutApp\workouts.log" -Append -InputObject "On $trandate, $tranuser added Employee# $eid, $fname $lname"
$sqlcn.Open()
$sqlcmd = $sqlcn.CreateCommand()
$query = "INSERT INTO employees values (#eid,#lname,#fname,#dept,#pay,#hire)"
$sqlcmd.CommandText = $query
$sqlcmd.Parameters.AddWithValue("#eid", $eid) | Out-Null
$sqlcmd.Parameters.AddWithValue("#fname", $fname) | Out-Null
$sqlcmd.Parameters.AddWithValue("#lname", $lname) | Out-Null
$sqlcmd.Parameters.AddWithValue("#dept", $dept) | Out-Null
$sqlcmd.Parameters.AddWithValue("#pay", $pay) | Out-Null
$sqlcmd.Parameters.AddWithValue("#hire", $hire) | Out-Null
$sqlcmd.ExecuteNonQuery()
$sqlcn.Close()
}
<# Stuff for removing an employee from the database#>
R{
<#Collect reason for removal#>
$reason = Read-Host -Prompt "Why are you deleting this employee?"
$eid = Read-Host "What is the ID number of the Employee you want to remove?"
$sqlcn.Open()
$sqlcmd = $sqlcn.CreateCommand()
$query = "SELECT EmployeeID,FirstName, LastName from Employees WHERE EmployeeID = #eid"
$sqlcmd.CommandText = $query
$sqlcmd.Parameters.AddWithValue("#eid", $eid) | Out-Null
$sqlcmd.ExecuteNonQuery()
$Reader = $sqlcmd.ExecuteReader()
$arry = #()
while ($Reader.Read()) {
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
#convert hashtable into an array of PSObjects
$arry+= new-object psobject -property $row
}
$sqlcn.Close()
write-host $arry
$empResult = Read-Host "Is that the correct employee? [Y]es or [N]o"
<#If the correct employee was found, continue below.
If the wrong employee was returned, Kill Program #>
switch($empResult) {
Y{
Out-File -FilePath "L:\Personnel\WorkoutApp\workouts.log" -Append -InputObject "On $trandate, $tranuser deleted Employee $eid for the following reason: $reason"
$sqlcn.Open()
$sqlcmd = $sqlcn.CreateCommand()
$query = "DELETE FROM Employees WHERE EmployeeID = #eid"
$sqlcmd.CommandText = $query
$sqlcmd.Parameters.AddWithValue("#eid", $eid)
$sqlcmd.ExecuteNonQuery()
$adp = New-Object System.Data.SqlClient.SqlDataAdapter $sqlcmd
$data = New-Object System.Data.DataSet
$adp.fill($data) | Out-Null
$sqlcn.Close()
}
N{
Out-File -FilePath "L:\Personnel\WorkoutApp\workouts.log" -Append -InputObject "On $trandate, $tranuser tried to deleted Employee $eid. But exited the program before doing so."
Write-Host "Please restart the program. If the issue persists, please contact the IT department."
Read-Host -Prompt "Press Enter to exit"
}
}
}
<# Line to exit the program #>
E{
exit
}
}
Any thoughts on cleaning this up would be greatly appreciated.
This is off-topic, but I'll give you an answer.
Generally, you don't want to use Parameters.AddWithValue() at all, because that sends every parameter as an NVARCHAR. It's not deprecated, but it's not a good idea to use it. If you've got datetimes or other non-string parameters, you can end up with problems. It's usually preferable to use Parameters.Add():
$sqlcmd.Parameters.Add("#eid", [System.Data.SqlDbType]::Int).Value = $eid
Obviously, the datatype you use from [System.Data.SqlDbType] should match the datatype of the actual column in the database. This also has the benefit that there won't be any return value that you need to send to Out-Null or cast as [void].
This is also a mess:
$sqlcmd.ExecuteNonQuery()
$Reader = $sqlcmd.ExecuteReader()
$arry = #()
while ($Reader.Read()) {
$row = #{}
for ($i = 0; $i -lt $reader.FieldCount; $i++)
{
$row[$reader.GetName($i)] = $reader.GetValue($i)
}
#convert hashtable into an array of PSObjects
$arry+= new-object psobject -property $row
}
First, you're executing the query twice. Both ExecuteNonQuery() and ExecuteReader() will execute the query! You do that multiple times in your script.
Second, you can just do this:
$DataTable = New-Object System.Data.DataTable
$DataTable.Load($sqlcmd.ExecuteReader())
Then, if you really don't want to work with a DataTable -- they're more complex than a custom object but really not that bad -- you can do this to convert it to a generic object pretty easily:
$Data = $DataTable | ConvertTo-Csv -NoTypeInformation | ConvertFrom-Csv
This will make everything a string, though, so be sure that's what you want. You might also try this:
$Data = $DataTable | Select-Object -Property <list>
You don't want to use Select-Object * because you'll get extra properties you probably don't want.
This is also executing the query twice:
$sqlcn.Open()
$sqlcmd = $sqlcn.CreateCommand()
$query = "DELETE FROM Employees WHERE EmployeeID = #eid"
$sqlcmd.CommandText = $query
$sqlcmd.Parameters.AddWithValue("#eid", $eid)
$sqlcmd.ExecuteNonQuery()
$adp = New-Object System.Data.SqlClient.SqlDataAdapter $sqlcmd
$data = New-Object System.Data.DataSet
$adp.fill($data) | Out-Null
$sqlcn.Close()
Both $sqlcmd.ExecuteNonQuery() and $adp.fill($data) execute the query! Additionally, ExecuteNonQuery() returns the number of records affected. You could do this:
$sqlcmd.ExecuteNonQuery() | Out-Null
Or this:
[void]$sqlcmd.ExecuteNonQuery()
But what you really should do is verify that the result is what you expect. You shouldn't be getting -1 for INSERT or DELETE statements.
Learn to look up the documentation for the methods you're calling and understand what the possible return values are and why. All the .Net methods are thoroughly documented on MSDN. You can almost always find them by Googling "C# ". You'll find C# examples that can easily be converted to PowerShell, too.

Resources