I want to write csv file to SQL Server. This is successfully working on a PC having windows 10 with Powershell 5. It is throwing error on a windows server 2012 and windows server 2016 with Povershell 5
Error Msg : Write-SqlTableData : The given value of type String from the data source cannot be converted to type date of the specified target column.
My Code is below :
$DestServer = "servername01.fqdn,1200"
$DestInstance = "instance01"
$DestDatabase = "test"
$ServerDataDir = "\\servername01\folder1"
$ServInstance = "$DestServer\$DestInstance"
$Data_Dir = "\\servername02.fqdn\c$\datafolder"
$Headers_Dir = $Data_Dir + "\Header files"
$ExclSourcename_Head_File = "$Headers_Dir\ExcelSource_Head.txt"
$ExclSourcename_Fields_File = "$Headers_Dir\ExcelSource_Fields.txt"
Out-DataTable.ps1
Clear-Host
write-host "Processing ExcelSource File" -F DarkBlue -BackgroundColor Cyan
$ExclSourcename_Head = (Get-content "$ExclSourcename_Head_File" -Delim ",").Replace(",","")
$ExclSourcename_Fields = #{}
$ExclSourcename_Fields = (Get-content "$ExclSourcename_Fields_File" -Delim ",").Replace(",","")
$ExclSourcename_File = "$Data_Dir\exclfile*.csv"
# Find latest ExclSourcename file to be processed
$ExclSourcename_FileIN = (Get-ChildItem "$ExclSourcename_File" | Sort-Object LastWriteTime -Descending | Select-Object -first 1)
$ExclSourcename_Date = $ExclSourcename_FileIN.BaseName.Replace("BK_","")
$ExclSourcename_Formatted_Date = [datetime]::ParseExact($ExclSourcename_Date,"yyyyMMdd",$null).AddDays(-1).ToString("dd/MM/yyyy")
$ExclSourcename_Temp = "C:\temp\ExclSourcenameFile.txt"
(Get-Content -Path "$ExclSourcename_FileIN").Replace(",",".") | Out-File -FilePath "$ExclSourcename_Temp" -Force
##$ExclSourcename_Data = Import-csv -Path "$ExclSourcename_File" -Delim ";" -Header $ExclSourcename_Head
$ExclSourcename_Data = Import-csv -Path "$ExclSourcename_Temp" -Delim ";" -Header $ExclSourcename_Head
$ExclSourcename_Data = $ExclSourcename_Data[1..($ExclSourcename_Data.Count - 1)]
$ExclSourcename_Count = $ExclSourcename_Data.Count
$ExclSourcename_Data | Add-Member -MemberType NoteProperty -Name Distributor -Value "ExcelSource"
$ExclSourcename_Data | Add-Member -MemberType NoteProperty -Name "File Date" -Value $ExclSourcename_Formatted_Date
Write-Host "ExcelSource File imported - $ExclSourcename_Count records found ... " -NoNewline
Write-Host "Converting to Data Table"
$ExclSourcename_DataTable = $ExclSourcename_Data | Select-Object -Property $ExclSourcename_Fields | Out-DataTable
Write-Host "Uploading ExcelSource data to SQL ... " -ForegroundColor White -BackgroundColor Blue -NoNewline
Invoke-Sqlcmd -Database $DestDatabase -ServerInstance "$DestServer\$DestInstance" -Query "Truncate Table [EU_Staging].tblRBL_Data_ExclSourcename"
Write-SqlTableData -DatabaseName $DestDatabase -ServerInstance "$DestServer\$DestInstance" -SchemaName EU_Staging -TableName tblRBL_Data_ExclSourcename -InputData $ExclSourcename_DataTable -Timeout 900
Write-host "done" -ForegroundColor White -BackgroundColor Red
Write-Host "ExcelSource data loaded"
Write-Host "==========" -ForegroundColor DarkGreen -BackgroundColor Green
Remove-Item -Path "$ExclSourcename_Temp" -Force
Remove-Variable ExclSourcename_Data
Remove-Variable ExclSourcename_DataTable
Related
I am trying to output SQL results to a .csv file using Powershell separated by its respective column.
The script I wrote works, but it will duplicate the same result three times in the csv. Even if I have only 1 result from the Select statement from the table, it will output it three times in the .csv file.
I tried using pscustomobject as well. But it throws me an error and does not output anything.
Clear-Variable Results
Clear-Variable Report
[string] $query = "Select Name, Value From options with(nolock) where Name IN('ExportFolder','ImportFolder','GlobalExportFolder'); Select ##ROWCOUNT AS AffectedRows"
[string[]] $servers = #('sqlinstance=mytestdb')
foreach($server in $servers)
{
$instance = ($server -split '=')[0]
$db = ($server -split '=')[1]
Try{
$Results = Invoke-Sqlcmd -ServerInstance $instance -Database $db -Query $query
$ExportFolder = ($Results.ItemArray[1])
$GlobalExportFolder = ($Results.ItemArray[3])
$ImportFolder = ($Results.ItemArray[5])
$Array = '$ExportFolder','$GlobalExportFolder','ImportFolder'
$mail = $Array | Select-Object #{n="SQLServer";e={$instance}},#{n="DBName";e={$db}}, #{n="ExportFolder";e={$ExportFolder}}, #{n="GlobalExportFolder";e={$GlobalExportFolder}}, #{n="ImportFolder";e={$ImportFolder}}
$mail | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\HostNameCheck.csv" -NoTypeInformation -Append -Verbose
$Report += $Results
$Report | Select Name, Value | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\SFTPHostnameModificationCheck.csv" -NoTypeInformation -Append -Verbose
Clear-Variable Results
Clear-Variable Report
}
Catch {
Write-Host ("Error: Data retrieval failed against instance $instance for $db" + " - " + (Get-Date)) -ForegroundColor Red
Write-Output ("Error: Data retrieval failed against $instance on $db" + " - " + (Get-Date)) | Out-File -FilePath $PathFailedLogs -Append
}
}
$attachment = Get-ChildItem -Path "C:\Users\localadmin\Documents\Logs" -Include *.csv -Recurse -Force
Send-MailMessage -From "test#test.com" -To "localadmin#nonprod.com" -Subject "SFTPHostnameCheck" -SmtpServer "localrelay#local.com" -Attachments $attachment
Using PSCustomObject
$obj = New-Object [PSCustomObject] -Property #{
'SQLServer' = $instance
'DBName' = $db
'ExportFolder' = "$ExportFolder
'GlobalExportFolder' = $GlobalExportFolder
'ImportFolder' = $ImportFolder"
}
$list += $obj
$list | Export-Csv -Path "C:\Users\localadmin\Documents\Logs\HostNamecheck.csv" -NoTypeInformation -Append -Verbose
I am trying to load an Excel sheet into SQL server using Import-Excel module in Powershell. There are columns which have some values - #N/A. While loading I need to replace #N/A to something else. I am trying to use below statement to do this but it is giving me the error.
write-host "Processing TD Equipment File" -ForegroundColor DarkBlue -BackgroundColor Cyan
$TD_File = "$Data_Dir\Backlog_Hardware*.xlsx"
$TD_FileIn = (Get-ChildItem "$TD_File" | Sort-Object LastWriteTime -Descending | Select-Object -first 1)
$TD_File_Date = $TD_FileIn.BaseName.Replace("Backlog_Hardware_","")
$TD_Data = Import-Excel -Path $TD_FileIn -WorksheetName Backlog
$TD_Count = $TD_Data.Count
Write-Host "TD Equipment File imported - $TD_Count records found ... " -NoNewline
Write-Host "Converting to Data Table"
$TD_DataTable = ($TD_Data | Where-object { ($_.'E/S' -replace "``#N/A","NA")} -and ($_.'Physical Stock' -replace "``#N/A","0")|
Select-Object -Property Distributor,'File Date',Customer,'Customer Name','Physical Stock') | Out-DataTable
Invoke-Sqlcmd -Database XXX -ServerInstance XX\XX -Query "Truncate Table tbl1"
Write-SqlTableData -DatabaseName XXX -ServerInstance XX\XX -TableName tbl1 -InputData $TD_DataTable -Timeout 900
I am getting below error:
Out-DataTable : Could not add property 'Physical Stock' with value '#N/A' and type 'OfficeOpenXml.ExcelErrorValue'
At line:60 char:812
I would like to perform a bunch of invoke-sqlcmd in one sql transaction. Here's what I'm doing:
try{
$scope = New-Object -TypeName System.Transactions.TransactionScope
GetFiles $SqlFilesDirectory
$scope.Complete()
}
catch{
$_.exception.message
}
finally{
$scope.Dispose()
}
Here's how GetFiles is defined:
#
# Get SQL Files recursively
#
function GetFiles($path = $pwd)
{
$subFolders = Get-ChildItem -Path $path -Directory | Select-Object FullName,Name | Sort-Object -Property Name
$sqlFiles = Get-ChildItem -Path $path -Filter *.sql | Select-Object FullName,Name | Sort-Object -Property Name
foreach ($file in $sqlFiles)
{
Write-Host "file: " $file.Name
Invoke-Sqlcmd -ServerInstance $ServerInstance -Database $DBName -Username $SvcAdminAccount -Password $SvcAdminPassword -InputFile $file.FullName -QueryTimeout 65535
}
foreach ($folder in $subFolders)
{
Write-Host "`nGetting files for subfolder: " $folder.Name
GetFiles $folder.FullName
}
}
How do we perform a series of invoke-sqlcmd in one transaction?
Here's the output:
The behavior that I want is that ALL
changes are rolled back if a single sql script fails.
I'm working on a powershell script to read file attributes filtered by CreationTime on multiple shares. The scripts works, sporadically. It works great when I use a single path but I get mixed results when I add the folders paths to an array. The most disturbing result is when it successfully find and reads all path and then includes everything under c:windows\system32. Same anomaly when shares are empty.
So what I want to accomplish is:
Read list of Shares
Read each share content filtered by 'CreationTime' and 'Archive' attributes.
Save results to a csv file.
If file not empty, write results to event log.
here is the code
$timer = (Get-Date -Format yyy-MM-dd-HHmm)
$Date=(Get-Date).AddHours(-3)
$FolderList = "C:\Software\Scripts\FolderList.txt"
$Folders = get-content $FolderList
$Filepath = "C:\Software\Scripts"
$filename = "$Filepath\" + $timer + "OldFiles.csv"
foreach ($Folder in $Folders)
{
Get-ChildItem $Folder | Where-Object { $_.CreationTime -lt $Date -and $_.Attributes -band [System.IO.FileAttributes]::Archive} | Select Attributes, CreationTime, Fullname | Export-Csv -Path $filename -NoTypeInformation
}
if ( (get-childitem $filename).length -eq 0 )
{
exit
}
else{
#Write to OpsMgr Log
$Message = get-content $filename
Write-EventLog -LogName "Operations Manager" -Source "Health Service Script" -EventID 402 -EntryType Information -Message "Old files found. $Message"
}
This (untested) script might do what you want:
$Date = (Get-Date).AddHours(-3)
$FolderList = "C:\Software\Scripts\FolderList.txt"
$Folders = Get-Content $FolderList
$Filepath = "C:\Software\Scripts"
$timer = (Get-Date -Format yyyy-MM-dd-HHmm)
$filename = Join-Path $Filepath ("{0}_OldFiles.csv" -f $timer)
$Data = foreach ($Folder in $Folders){
Get-ChildItem $Folder |
Where-Object { $_.CreationTime -lt $Date -and
$_.Attributes -band [System.IO.FileAttributes]::Archive} |
Select Attributes, CreationTime, Fullname
}
if ($Data.Count){
#Write to OpsMgr Log
$Data | Export-Csv -Path $filename -NoTypeInformation
$Message = $Data | ConvertTo-Csv
Write-EventLog -LogName "Operations Manager" -Source "Health Service Script" `
-EventID 402 -EntryType Information `
-Message "Old files found. $Message"
}
While trying to create an CSV file with information about certificates I have an issue to store the userrights on the private key.
The problem is that I want to store multiple values in one attribute so I use an array.
At first I had no errors, however the column in my csv-file remained empty even in the case where the array has a value.
With a simple Write-Host I can see my array has the expected value so this part works okay.
For further investigations I have added the line:
Get-Member $certs.GetValue("UserRights")
This gives an error indicating I have to convert my variable to a string-variable.
So next I have tried to convert this array to a single string.
I have tried several ways but my error doesn't disappear so it doesn't work.
Underneath is my full code with some former attempts commented.
cls $certs = Get-ChildItem cert:\LocalMachine -Recurse | Where-Object {-not $_.PSIsContainer} | Select * Write-Host ("There were {0} certificates" -f ($certs | Measure-Object).Count)
foreach($certificate in $certs) {
if($certificate.HasPrivateKey)
{
Write-Host "Certificate's PSChildName is" $certificate.PSChildName
$rsaFile = $certificate.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
$fullPath = "C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\" + $rsaFile
$acl = Get-Acl -Path $fullPath
foreach($accessrule in $acl.Access)
{
Write-Host "User" $accessrule.IdentityReference "has the following rights:" $accessrule.FileSystemRights
}
Write-Host "------"
$UserRechten = #()
foreach($accessrule in $acl.Access)
{
$UserRechten += "{0}:{1};" -f ($accessrule.IdentityReference,$accessrule.FileSystemRights)
}
Write-Host "================================================================"
# -join $UserRechten
# $Userrechten | out-string
# $UserRechten = [system.String]::Join(" ", $UserRechten)
$separator = ";"
[string]::Join($separator,$UserRechten)
$certs | Add-Member -MemberType NoteProperty -Name "UserRights" -Value $UserRechten -Force
Write-Host "UserRechten has value : "$UserRechten
Get-Member $certs.GetValue("UserRights")
Write-Host "================================================================"
} }
$Certs | Add-Member -MemberType NoteProperty -Name "MachineName" -Value $env:COMPUTERNAME -Force
# $certs | Add-Member -MemberType NoteProperty -Name "Store" -Value 'My' -Force $RunDate = Get-Date -Format 'yyyy-MM-dd' $certs | Add-Member -MemberType NoteProperty -Name "RunDate" -Value $RunDate -Force $certs | Add-Member -MemberType NoteProperty -Name "Owner" -Value $env:USERNAME -Force
$Certs | Select * | Export-Csv c:\Certificaten\LocalCertsAll_$env:COMPUTERNAME.csv
$Certs | Select MachineName, Owner, PSParentPath, DnsNameList, PSChildName, NotBefore, NotAfter, Rundate, EnhancedKeyUsageList, HasPrivateKey, SerialNumber, Issuer, Subject, FriendlyName, UserRigthts |
Export-CSV c:\Certificaten\Localcerts_$env:COMPUTERNAME.csv
As noted in the comments, Get-Member is probably not what you're looking for
You (almost certainly) don't want to add the UserRights member property to the $Certs array, but rather to the individual objects in $Certs.
(I removed a bunch of superfluous Write-Host statements for readability):
$CertsAmended = foreach($Certificate in $certs)
{
if($certificate.HasPrivateKey)
{
$rsaFile = $certificate.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
$fullPath = "C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\" + $rsaFile
$acl = Get-Acl -Path $fullPath
# Create the UserRights value using -join
$UserRechten = #(foreach($accessrule in $acl.Access){
Write-Host "User" $accessrule.IdentityReference "has the following rights:" $accessrule.FileSystemRights
"{0}:{1}" -f ($accessrule.IdentityReference,$accessrule.FileSystemRights)
}) -join ";"
# Add the property to the individual object
$Certificate | Add-Member -MemberType NoteProperty -Name "UserRights" -Value $UserRechten
Write-Host "Userrights: " $UserRechten
# "Drop" the certificate object (now with a UserRights value) back onto the pipeline
$Certificate
}
}
Now you can export the $CertsAmended array to CSV all you want
If you find the $var = #(foreach($item in $collection){}) -join ';' displeasing to the eye, break it into two statements:
$UserRechten = foreach($accessrule in $acl.Access)
{
# Create UserRight string here, without ;
}
$UserRechten = $UserRechten -join ';'
For the $fullPath variable, you may want to use the Join-Path cmdlet:
$fullPath = Join-Path "C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\" $rsaFile