Duplicate SQL results from Powershell CSV export - sql-server

I am trying to output SQL results to a .csv file using Powershell separated by its respective column.
The script I wrote works, but it will duplicate the same result three times in the csv. Even if I have only 1 result from the Select statement from the table, it will output it three times in the .csv file.
I tried using pscustomobject as well. But it throws me an error and does not output anything.
Clear-Variable Results
Clear-Variable Report
[string] $query = "Select Name, Value From options with(nolock) where Name IN('ExportFolder','ImportFolder','GlobalExportFolder'); Select ##ROWCOUNT AS AffectedRows"
[string[]] $servers = #('sqlinstance=mytestdb')
foreach($server in $servers)
{
$instance = ($server -split '=')[0]
$db = ($server -split '=')[1]
Try{
$Results = Invoke-Sqlcmd -ServerInstance $instance -Database $db -Query $query
$ExportFolder = ($Results.ItemArray[1])
$GlobalExportFolder = ($Results.ItemArray[3])
$ImportFolder = ($Results.ItemArray[5])
$Array = '$ExportFolder','$GlobalExportFolder','ImportFolder'
$mail = $Array | Select-Object #{n="SQLServer";e={$instance}},#{n="DBName";e={$db}}, #{n="ExportFolder";e={$ExportFolder}}, #{n="GlobalExportFolder";e={$GlobalExportFolder}}, #{n="ImportFolder";e={$ImportFolder}}
$mail | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\HostNameCheck.csv" -NoTypeInformation -Append -Verbose
$Report += $Results
$Report | Select Name, Value | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\SFTPHostnameModificationCheck.csv" -NoTypeInformation -Append -Verbose
Clear-Variable Results
Clear-Variable Report
}
Catch {
Write-Host ("Error: Data retrieval failed against instance $instance for $db" + " - " + (Get-Date)) -ForegroundColor Red
Write-Output ("Error: Data retrieval failed against $instance on $db" + " - " + (Get-Date)) | Out-File -FilePath $PathFailedLogs -Append
}
}
$attachment = Get-ChildItem -Path "C:\Users\localadmin\Documents\Logs" -Include *.csv -Recurse -Force
Send-MailMessage -From "test#test.com" -To "localadmin#nonprod.com" -Subject "SFTPHostnameCheck" -SmtpServer "localrelay#local.com" -Attachments $attachment
Using PSCustomObject
$obj = New-Object [PSCustomObject] -Property #{
'SQLServer' = $instance
'DBName' = $db
'ExportFolder' = "$ExportFolder
'GlobalExportFolder' = $GlobalExportFolder
'ImportFolder' = $ImportFolder"
}
$list += $obj
$list | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\HostNamecheck.csv" -NoTypeInformation -Append -Verbose

Related

how to execute multiple Invoke-Sqlcmd in one transaction?

I would like to perform a bunch of invoke-sqlcmd in one sql transaction. Here's what I'm doing:
try{
$scope = New-Object -TypeName System.Transactions.TransactionScope
GetFiles $SqlFilesDirectory
$scope.Complete()
}
catch{
$_.exception.message
}
finally{
$scope.Dispose()
}
Here's how GetFiles is defined:
#
# Get SQL Files recursively
#
function GetFiles($path = $pwd)
{
$subFolders = Get-ChildItem -Path $path -Directory | Select-Object FullName,Name | Sort-Object -Property Name
$sqlFiles = Get-ChildItem -Path $path -Filter *.sql | Select-Object FullName,Name | Sort-Object -Property Name
foreach ($file in $sqlFiles)
{
Write-Host "file: " $file.Name
Invoke-Sqlcmd -ServerInstance $ServerInstance -Database $DBName -Username $SvcAdminAccount -Password $SvcAdminPassword -InputFile $file.FullName -QueryTimeout 65535
}
foreach ($folder in $subFolders)
{
Write-Host "`nGetting files for subfolder: " $folder.Name
GetFiles $folder.FullName
}
}
How do we perform a series of invoke-sqlcmd in one transaction?
Here's the output:
The behavior that I want is that ALL
changes are rolled back if a single sql script fails.

Write-SqlTableData : Error Uploading csv to SQL server using PowerShell

I want to write csv file to SQL Server. This is successfully working on a PC having windows 10 with Powershell 5. It is throwing error on a windows server 2012 and windows server 2016 with Povershell 5
Error Msg : Write-SqlTableData : The given value of type String from the data source cannot be converted to type date of the specified target column.
My Code is below :
$DestServer = "servername01.fqdn,1200"
$DestInstance = "instance01"
$DestDatabase = "test"
$ServerDataDir = "\\servername01\folder1"
$ServInstance = "$DestServer\$DestInstance"
$Data_Dir = "\\servername02.fqdn\c$\datafolder"
$Headers_Dir = $Data_Dir + "\Header files"
$ExclSourcename_Head_File = "$Headers_Dir\ExcelSource_Head.txt"
$ExclSourcename_Fields_File = "$Headers_Dir\ExcelSource_Fields.txt"
Out-DataTable.ps1
Clear-Host
write-host "Processing ExcelSource File" -F DarkBlue -BackgroundColor Cyan
$ExclSourcename_Head = (Get-content "$ExclSourcename_Head_File" -Delim ",").Replace(",","")
$ExclSourcename_Fields = #{}
$ExclSourcename_Fields = (Get-content "$ExclSourcename_Fields_File" -Delim ",").Replace(",","")
$ExclSourcename_File = "$Data_Dir\exclfile*.csv"
# Find latest ExclSourcename file to be processed
$ExclSourcename_FileIN = (Get-ChildItem "$ExclSourcename_File" | Sort-Object LastWriteTime -Descending | Select-Object -first 1)
$ExclSourcename_Date = $ExclSourcename_FileIN.BaseName.Replace("BK_","")
$ExclSourcename_Formatted_Date = [datetime]::ParseExact($ExclSourcename_Date,"yyyyMMdd",$null).AddDays(-1).ToString("dd/MM/yyyy")
$ExclSourcename_Temp = "C:\temp\ExclSourcenameFile.txt"
(Get-Content -Path "$ExclSourcename_FileIN").Replace(",",".") | Out-File -FilePath "$ExclSourcename_Temp" -Force
##$ExclSourcename_Data = Import-csv -Path "$ExclSourcename_File" -Delim ";" -Header $ExclSourcename_Head
$ExclSourcename_Data = Import-csv -Path "$ExclSourcename_Temp" -Delim ";" -Header $ExclSourcename_Head
$ExclSourcename_Data = $ExclSourcename_Data[1..($ExclSourcename_Data.Count - 1)]
$ExclSourcename_Count = $ExclSourcename_Data.Count
$ExclSourcename_Data | Add-Member -MemberType NoteProperty -Name Distributor -Value "ExcelSource"
$ExclSourcename_Data | Add-Member -MemberType NoteProperty -Name "File Date" -Value $ExclSourcename_Formatted_Date
Write-Host "ExcelSource File imported - $ExclSourcename_Count records found ... " -NoNewline
Write-Host "Converting to Data Table"
$ExclSourcename_DataTable = $ExclSourcename_Data | Select-Object -Property $ExclSourcename_Fields | Out-DataTable
Write-Host "Uploading ExcelSource data to SQL ... " -ForegroundColor White -BackgroundColor Blue -NoNewline
Invoke-Sqlcmd -Database $DestDatabase -ServerInstance "$DestServer\$DestInstance" -Query "Truncate Table [EU_Staging].tblRBL_Data_ExclSourcename"
Write-SqlTableData -DatabaseName $DestDatabase -ServerInstance "$DestServer\$DestInstance" -SchemaName EU_Staging -TableName tblRBL_Data_ExclSourcename -InputData $ExclSourcename_DataTable -Timeout 900
Write-host "done" -ForegroundColor White -BackgroundColor Red
Write-Host "ExcelSource data loaded"
Write-Host "==========" -ForegroundColor DarkGreen -BackgroundColor Green
Remove-Item -Path "$ExclSourcename_Temp" -Force
Remove-Variable ExclSourcename_Data
Remove-Variable ExclSourcename_DataTable

Powershell Array to csv

I'm pretty new to powershell and I cant figure out how to get my array into a csv file, where each string goes onto a new row. Below is some example code.
$ServerList = "E:\Coding Projects\Powershell\ServerNameList.txt"
$ServerNames = Get-content $ServerList
write-host $ServerNames
$OutputPath = "E:\Coding Projects\Powershell\Output.csv"
$Names = #()
$Outcome = #()
foreach ($Server in $ServerNames){
$Names += $Server
if ($Server -match "Joe"){
$Outcome += "pass"
}else{
$Outcome += "Fail"
}
}
$Names
$Outcome
$csv = New-object psobject -property #{
'User' = $Names -join ','
'Groups' = $Outcome -join ','
}
write-host $csv
$csv | Select-Object -property User, Groups | Export-csv -path $OutputPath -NoTypeInformation
When I check the csv file, all of the outputs appear on one row instead of iterating down the rowin its specific column.
Any help would be very useful and appreciated
Right now you're creating 2 separate arrays of string values - instead, you'll want to create a single array of objects with two properties:
$ServerList = "E:\Coding Projects\Powershell\ServerNameList.txt"
$ServerNames = Get-content $ServerList
write-host $ServerNames
$OutputPath = "E:\Coding Projects\Powershell\Output.csv"
$serversWithOutcome = #()
foreach ($Server in $ServerNames){
$serversWithOutcome += [pscustomobject]#{
User = $Server
Groups = $Server -match "Joe"
}
}
$serversWithOutcome | Export-csv -path $OutputPath -NoTypeInformation

What part of the script is printing the directory information?

I have the following contents in PowerShell script (Process.ps1) that reads from SQL tables and appends results to the variables listed:
function Query($Query) {
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=$Server;Initial Catalog=$Database;Integrated Security=SSPI"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.Connection = $SqlConnection
$SqlCmd.CommandText = $Query
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$a=$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
$DataSet.Tables[0]
}
$Result = Query "SELECT * from [$cubeTable]" | Out-GridView -Wait;
$CUBE = Query "SELECT [cube_name] FROM [$cubeTable] WHERE [cube_name] = '$CUBE_input'" | Select -ExpandProperty cube_name;
$Destination_Server = Query "SELECT [destination_server] FROM [$cubeTable] WHERE [cube_name] = '$CUBE'" | Select -ExpandProperty destination_server;
$BasePath = Query "SELECT [variable_value] FROM [$pathTable] WHERE [variable_name] = 'base_path'" | Select -ExpandProperty variable_value;
$jsonPath = Join-Path -Path $BasePath -ChildPath $jsonDirectory
New-Item -ItemType Directory -Force -Path $jsonPath
$JSON_file = Join-Path $jsonPath $CUBE |
%{ ($_ + ".json") }
$processPATH = Join-Path -Path $BasePath -ChildPath $process_output_Directory
New-Item -ItemType Directory -Force -Path $processPATH
$process_output = Join-Path $processPATH $CUBE |
%{ ($_ + ".txt") }
$autosysPATH = Join-Path -Path $BasePath -ChildPath $AUTOSYS_output_Directory
New-Item -ItemType Directory -Force -Path $autosysPATH
$process_AUTOSYS_output = Join-Path $autosysPATH $CUBE |
%{ ($_ + "_process.txt") }
When I run the script in CMD through a batch file, it runs great as it should, however, it's outputting these directory information somewhere from the following variables:
json_file, process_output, and autosys_output
I have an output image here:
Where exactly is the writing to console happening? I don't have any echo OR a Write-Host! let alone a function to output the directories...
and its definitely not this part: $Result = Query "SELECT * from [$cubeTable]" | Out-GridView -Wait; because I commented it out and it still outputted the directories info as the screenshot shows.
New-Item returns the created FileInfo or DirectoryInfo object. That's what you're seeing in your output. PowerShell default output formating just merges similar consecutive objects to provide more compact output, so you're getting a single table instead of three separate tables with one object each.
You can suppress the output by adding | Out-Null to the New-Item statements:
New-Item -ItemType Directory -Force -Path $jsonPath | Out-Null
Other options would be capturing the output in a variable or using redirection (> $null).

Error executing SQL script via SMO in PowerShell

I have some code that loads a script to a variable, and then I pass the variable to an SMO object. I get the following exception:
Exception calling "ExecuteWithResults" with "1" argument(s): "Execute
with results failed for Database 'Russell_Test'. "
$serverName is the server name.
$databaseName is the database name.
$createScript is the script that was read.
How can I fix this problem?
Below is the relevant portion of the code.
# Load Smo and referenced assemblies.
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.ConnectionInfo');
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.Management.Sdk.Sfc');
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO');
[void][System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMOExtended');
Try{
$server = New-Object Microsoft.SqlServer.Management.Smo.Server $serverName;
$db = $server.Databases.Item($databaseName);
$result = $db.ExecuteWithResults($createScript);
$result | Out-File -Append -FilePath $outputFile;
}
Catch
{
[system.exception]
$_.Exception | Out-File -Append -FilePath $outputFile
}
Here's how to do it:
# If we cast $Content as a [String], all the newlines are lost
# If we don't cast it, it becomes an array and batching breaks
$Content = (Get-Content $SqlScript) -join [Environment]::NewLine
# When using GO, we must set it up as a StringCollection, not a List or Array
$Batch = New-Object -TypeName:Collections.Specialized.StringCollection
$Batch.AddRange($Content)
$result = $Database.ExecuteWithResults($Batch)
Thanks for the help. Ultimately the SMO option would not work so I did this solution:
# Deploy table update scripts
$createScript = Get-Content $scriptFile.FullName | Out-String
################# Script execution to capture errors/messages/warnings ##################
$createScriptList = [regex]::Split($createScript, '\bGO')
$cn2 = new-object system.data.SqlClient.SQLConnection("Data Source=$serverName;Integrated Security=SSPI;Initial Catalog=$databaseName;Connection Timeout=600;Max Pool Size=10");
$cn2.Open();
foreach ($cSL in $createScriptList)
{
Try{
$cmd = new-object system.data.sqlclient.sqlcommand($cSL, $cn2);
$cmd.ExecuteScalar() | Out-File -Append -FilePath $outputFile;
}
Catch
{
[system.exception]
$_.Exception | Out-File -Append -FilePath $outputFile
}
}
$cn2.Close();
###############################################################################################
Try to use Invoke-SqlCmd cmdlet. This cmdlet allows you to run T-SQL code or commands supported be the SQLCMD utility.
Try{
Invoke-SqlCmd `
-Query $createScript `
-ServerInstance $serverName `
-Database $databaseName `
-ErrorAction Stop `
}
Catch
{
[system.exception]
$_.Exception | Out-File -Append -FilePath $outputFile
}

Resources