Parameterised powershell script through SSIS Execute Process Task (with UNC) - sql-server

I've done varied amounts of research into how to run a parameterised PS script from SSIS. I am having issues getting a parameterised script running. PS script is as follows, if I hard code the parameters into the script it behaves as expected:
Param ([string]$filepath,[string]$filename)
$Path = $filepath
$InputFile = (Join-Path $Path $filename)
$Reader = New-Object System.IO.StreamReader($InputFile)
While (($Line = $Reader.ReadLine()) -ne $null) {
If ($Line -match 'FILE\|([^\|]+)') {
$OutputFile = "$($matches[1]).txt"
}
Add-Content (Join-Path $Path $OutputFile) $Line
}
Running in SSIS execute process task, I am trying to build the Arguments command via an expression as follows:
"-ExecutionPolicy ByPass -File " + #[User::vPSScriptLocation] + " " + #[User::vFilePath]+ " "+ #[User::vFileName]
Evaluating the expression gives the following:
-ExecutionPolicy ByPass -File \\WorkDirectory\Script.ps1 \\transfer datafile.data
Upon execution, the task fails. The .ps1 is deleted from the work directory and SSIS gives the following error code:
Error: 0xC0029151 at Execute powershell script, Execute Process Task: In Executing "C:\Windows\System32\WindowsPowerShell\v1.0\PowerShell.exe" "-ExecutionPolicy ByPass -File \\WorkDirectory\Script.ps1 \\transfer datafile.data" at "", The process exit code was "-196608" while the expected was "0".
Looks like it's getting an empty string where it shouldn't? Any pointers appreciated.

OK so as it appears I could not call a UNC path to execute this using an Execute Process Task, I decided to execute this within a Script Task with a reference added to System.Management.Automation which allowed me to create a PowerShell instance. This is far from my ideal solution as I really wanted to call a .ps1 file, but looks like this is my only solution given I need to use a UNC path.
I build the PS script with my Dts variables and then executed it within the instance, which achieved the desired result:
public void Main()
{
string filepath = Dts.Variables["User::vUNCPath"].Value.ToString();
string filename = Dts.Variables["User::vFileName"].Value.ToString();
string searchterm = Dts.Variables["User::vSearchTerm"].Value.ToString();
bool fireAgain = true;
// Build powershell script
string script = "$Path = \""+filepath+"\";"+
"$InputFile = (Join-Path $Path \""+ filename+"\");" +
"$Reader = New-Object System.IO.StreamReader($InputFile);" +
"While (($Line = $Reader.ReadLine()) -ne $null) {" +
"If ($Line -match '"+searchterm+"') { "+
"$OutputFile = \"$($matches[1]).txt\"};" +
"Add-Content (Join-Path $Path $OutputFile) $Line}";
Dts.Events.FireInformation(0, "Info", "Powershell script built: " + script, String.Empty, 0, ref fireAgain);
try
{
// Create instance to run script
using (PowerShell psinstance = PowerShell.Create())
{
//Assign built script to this instance
psinstance.AddScript(script);
//Run powershell script
psinstance.Invoke();
}
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception ex)
{
Dts.Events.FireError(0, "Error", ex.Message, String.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
}

If you're running your script as a ps1-File with a param-block like this, your execution-call should name the parameters by their name:
"-ExecutionPolicy ByPass -File " + #[User::vPSScriptLocation] + " -filepath " + #[User::vFilePath]+ " -filename "+ #[User::vFileName]
This should do it, if you use a valid filepath and filename.
If it doesn't work, please try to write your script as a function and try it in a powershell-console. Your script as a function looks like this:
function SearchLines
{
Param (
[string]$filepath,
[string]$filename
)
$Path = $filepath
$InputFile = (Join-Path $Path $filename)
$Reader = New-Object System.IO.StreamReader($InputFile)
While (($Line = $Reader.ReadLine()) -ne $null) {
If ($Line -match 'FILE\|([^\|]+)') {
$OutputFile = "$($matches[1]).txt"
}
Add-Content (Join-Path $Path $OutputFile) $Line
}
}
Usage:
SearchLines -filepath "\\your\unc\path\here" -filename "filename.txt"
If this don't work for you, please let us know which error you got.
Thx.
UPDATE:
Based on your comments, i wrote your function new in the hope, it meets your requirements as close as possible. The function now looks like this:
function SearchLines
{
Param (
[string]$InputFile
)
$FileContent = Get-Content $InputFile
foreach($Line in $FileContent)
{
If ($Line -match 'FILE\|([^\|]+)')
{
$OutputFile = "$($matches[1]).txt"
}
Add-Content -Path $OutputFile -Value $Line
}
}
Usage:
SearchLines -InputFile c:\your\path\to\your\file.log
This function creates for every line in your given file a new file in the actual folder named what is written in the line. The Cmdlet Join-Path simply adds the two strings together, without any check for plausibility. That's why you can simply commit the full path to your file instead of the path and the file in separate parameters.
If you need the path of the inputfile to set it for your outputfiles, you can get it with these lines:
$tmpPath = Get-Childitem $InputFullPath
$Path = $tmpPath.Directory.FullName
Because you didn't explained what exactly this script should do, i hope you can use this to get what you wanted.
Greetings

Related

Powershell script to install multiple applications using an array and scriptblock

Need help troubleshooting an the Array and Scriptblock
OR Maybe this is better using param and functions???
Script Objective: To easily update the list of applications to be installed
Getting error below.
'
At C:\Temp\appinstall.ps1:7 char:10
$Firefox={
~
The assignment expression is not valid. The input to an assignment operator must be an object that is able to accept
assignments, such as a variable or a property.
+ CategoryInfo : ParserError: (:) [], ParseException
+ FullyQualifiedErrorId : InvalidLeftHandSide
'
Start-Transcript -Append c:\Deploy\log.txt
$ProgressPreference = 'SilentlyContinue';
#Change App Name, Source, MSI/EXE, Argument
$AppArray= (
$Firefox={
$App= "Firefox";
$App_source= "https://download.mozilla.org/?product=firefox-latest&os=win64&lang=en-US";
$destination = "c:\Deploy\$App.exe";
$Argument= "/S";
},
$Chrome=
{
$App= "Chrome";
$App_source= "https://dl.google.com/tag/s/defaultbrowser/edgedl/chrome/install/GoogleChromeStandaloneEnterprise64.msi";
$destination = "c:\Deploy\$App.exe";
$Argument= "/norestart","/qn";
}
)
$InstallScriptBlock=
{
$installed = (Get-ItemProperty HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | Where { $_.DisplayName -Match "$App" });
$installed.displayname
if ($installed.displayname -Match $App) {
Write-Host "$software installed"
}else{
If ((Test-Path $destination) -eq $false) {
New-Item -ItemType File -Path $destination -Force
}
#install software
Invoke-WebRequest $App_source -OutFile $destination
Start-Process -FilePath "$destination" -ArgumentList "$Argument" -Wait
#Delete installer
Remove-Item -recurse "$destination"
}
}
ForEach ($Program in $AppArray) {Invoke-Command -ScriptBlock $InstallScriptBlock}
Stop-Transcript
It looks like you're trying to create a nested hashtable (#{ ... }), but your syntax is flawed - see the linked docs.
However:
It should suffice in your case to create an array of hashtables to iterate over with foreach
There's no need to use a separate script block ({ ... }) - just use the body of the foreach loop statement.
As an aside: While using Invoke-Command for local invocation of script blocks works, it usually isn't necessary, because &, the call operator, will do (e.g. $sb = { 'hi' }; & $sb). Invoke-Command's primary purpose is to execute a script block on a remote machine.
Generally, you can use variables as-is as command arguments, without enclosing them in "..." - even if their values contain spaces. E.g., Write-Output $foo is sufficient, no need for Write-Output "$foo"
To put it all together:
# Create an array whose elements are hashtables.
$appArray = (
#{
App = ($thisApp = 'Firefox')
App_source = 'https://download.mozilla.org/?product=firefox-latest&os=win64&lang=en-US'
Destination = "c:\Deploy\$thisApp.exe"
Argument = '/S'
},
#{
App = ($thisApp = 'Chrome')
App_source = 'https://dl.google.com/tag/s/defaultbrowser/edgedl/chrome/install/GoogleChromeStandaloneEnterprise64.msi'
Destination = "c:\Deploy\$thisApp.exe"
Argument = '/norestart /qn'
}
)
foreach ($app in $appArray) {
# Note how $app.<key> is used to refer to the entries of the hashtable at hand,
# e.g. $app.App yields "Firefox" for the first hashtable.
$installed = Get-ItemProperty HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* | Where-Object { $_.DisplayName -Match $app.App }
$installed.displayname
if ($installed.displayname -Match $app.App) {
Write-Host "$($app.App) already installed."
}
else {
if ((Test-Path $app.Destination) -eq $false) {
New-Item -ItemType File -Path $app.Destination -Force
}
#install software
Invoke-WebRequest $app.App_source -OutFile $app.Destination
Start-Process -FilePath $app.Destination -ArgumentList $app.Argument -Wait
#Delete installer
Remove-Item -Recurse $app.Destination
}
}
Note:
I've removed unnecessary ; and I've switched to using verbatim (single-quoted) strings ('...') when no string interpolation via expandable (double-quoted) strings ("...") is required, both for conceptual clarity and to avoid potentially unwanted expansions.
Note the use of aux. variable $thisApp in the App key, which allows referencing it in the later Destination key, in an expandable string ("c:\Deploy\$thisApp.exe").
GitHub suggestion #13782 looks for a more elegant way to allow hashtable entries to reference one another.

Why the powershell script is not invoke inside my powershell program

Hi i am quite frustrated with powershell. i want to use my powershell script to call another powershell script inside it.
My powershell only do the first part which is download files and does not invoke the following powershell script...
Can not think the reason behind it.. Could anyone help me with that? Thanks!
The Program looks like :
$invocation = (Get-Variable MyInvocation).Value
$directorypath = Split-Path $invocation.MyCommand.Path
Set-Location $directorypath
if ($StepNumber -ne 2) {
$MyName = $MyInvocation.InvocationName
$LogFileName = ".\Logs\" + [System.IO.Path]::GetFileNameWithoutExtension($MyName) + [String]::Format("_{0:yyyy-MM-dd_HH-mm-ss}.Log", (Get-Date))
if (!(Test-Path ".\Logs")) {
$X = New-Item -path . -name Logs -ItemType directory
}
&PowerShell.exe -noProfile -File NSCCMPIDDownload.ps1 2 *>`&1 | Tee $LogFileName
exit
}
## ## ---- STEP2 ------------DownLoad the file from Website
$ErrorActionPreference = "Continue"
Write-Host Running NSCC-MPID-Download at $(Get-Date)
Write-Host ""
## Download the NSCC-MPID-Directory.xls from website
$datestr=$(Get-Date).ToString("yyyy-MM-dd")
$url="http://www.dtcc.com/~/media/Files/Downloads/client-center/NSCC/NSCC-MPID-Directory.xls"
$destination = $directorypath +"/NSCC-MPID-Directory"+"_"+$datestr+".xls"
## if destination does not exsist, creat a new one.
if (!(Test-Path $destination) ) {
New-Item $destination -type file -force
}
$client = new-object System.Net.WebClient
$client.DownloadFile( $url, $destination)
Write-Host Running NSCCMPIDInsertion.ps1
$cmd = "$directorypath\NSCCMPIDInsertion.ps1"
Invoke-Expression "$cmd"
Write-Host Running NSCCMPIDAfterParse.ps1
$cmd = "$directorypath\NSCCMPIDAfterParse.ps1"
Invoke-Expression "$cmd"
Write-Host --------------------------------------------------------------------------
Write-Host Process ended at $(Get-Date)

Powershell Job Step in SQL Error

I'm new to PowerShell and still early in SQL Server, but I'm trying to write a PowerShell step for a SQL Agent job that looks at a CSV file that contains names of .sql files.
It should then look at a different directory and if the names from the CSV file exist in that directory it should open the .sql file and execute the function inside.
I'm getting an error:
Unable to cast object of type 'System.String to type System.Type
Any help would be greatly appreciated.
$excelFile = "C:/ExcelTest/Test.csv"
$functionDirectory = "some directory"
$excel_Array = (Get-Content $excelFile)[0].split(",")
foreach ($sqlName in $excel_Array)
{
if($sqlName::exists($functionDirectory + "/" + $sqlName) -ne $true)
{
invoke-sqlcmd -inputfile $functionDirectory + "/" + $sqlName -serverinstance "serverinstance" -database "database"
}
}
If understand the question correctly, you need to use Test-Path not ::exists
$excelFile = "C:/ExcelTest/Test.csv"
$functionDirectory = "some directory"
Import-Csv $excelFile |
Foreach-Object {
$filename = $functionDirectory + '\' + $_[0]
if (Test-Path $filename) {
invoke-sqlcmd -inputfile $filename -serverinstance "serverinstance" -database "database"
}
}
I would adjust a few things in your script to properly handle a CSV and then utilize built-in cmdlet for testing the path of a given file.
[cmdletbinding()]
param()
Import-Module SQLPS -DisableNameChecking
$functionDirectory = "C:\temp\PowerShell_Testing2"
$excelFile = Import-Csv "C:\temp\PowerShell_Testing\SQLFileList.csv"
foreach ($e in $excelFile) {
$fileonly = Split-Path $e.SQLFile -Leaf
$fdFile = $functionDirectory + "\" + $fileonly
if (Test-Path $fdFile) {
Write-Host "Found File $fdFile"
Invoke-Sqlcmd -ServerInstance "MANATARMS\SQL12" -InputFile $fdFile -Database master
}
}
In my setup there is only one file that just runs:
SELECT TOP 1 name FROM sys.databases
If your CSV contains say a ServerName, and then SQLFile you can adjust your script to also pull the ServerInstance value like this:
[cmdletbinding()]
param()
Import-Module SQLPS -DisableNameChecking
$functionDirectory = "C:\temp\PowerShell_Testing2"
$excelFile = Import-Csv "C:\temp\PowerShell_Testing\SQLFileList.csv"
foreach ($e in $excelFile) {
$fileonly = Split-Path $e.SQLFile -Leaf
$fdFile = $functionDirectory + "\" + $fileonly
if (Test-Path $fdFile) {
Write-Host "Found File $fdFile"
Invoke-Sqlcmd -ServerInstance $e.ServerName -InputFile $fdFile -Database master
}
}

Look through text files for a certain text and insert to SQL

I'm trying to optimize my Powershell Script a little.
I have a lot of log (text) files, that i need to search through the content of, for a specific text entry.
If the entry is found, I need the script to trigger with an inset to an sql databse.
This is what I have for now:
$tidnu = (Get-Date -f dd.MM.yyyy)
$Text = "ERROR MESSAGE STACK"
$PathArray = #()
$NodeName = "SomeName"
$Logfil = "SomeLogFile"
Get-ChildItem $Path -Filter "*ORA11*.log" |
Where-Object { $_.Attributes -ne "Directory"} |
ForEach-Object {
If (Get-Content $_.FullName | Select-String -Pattern $Text)
{
$PathArray += $_.FullName
$cmd.commandtext = "INSERT INTO ErrorTabel (Datotid, Nodename, Logfil, ErrorFound) VALUES('{0}','{1}','{2}','{3}')" -f $tidnu, $NodeName, $Logfil, "Yes"
$cmd.ExecuteNonQuery()
}
else
{
$cmd.commandtext = "INSERT INTO ErrorTabel (Datotid, Nodename, ErrorFound) VALUES('{0}','{1}','{2}')" -f $tidnu, $NodeName, "No"
$cmd.ExecuteNonQuery()
}
}
This is working okay, but when i need to move to another log file name, i have simply made the same code again with different inputs.
What i would like to do, is to use an Array, and a foreach loop, so i could specify all the log files in one array, like:
$LogArray = #(Log1.log, log2.log, log3.log)
And specify all the Nodenames like:
$NodeArray = #(Node1, Node2, Node3)
And then make a foreach loop that will go through the logfiles one by one and insert into the databse, with it's matching nodename every time the loop runs through.
Can someone help me to make this happen? I have the idea on how it should be done, but I can't figure out how to write the code. All help would be much appreciated.
EDIT:
Ok, this is what i have now then, but i'm not sure that it's correct put together. Its giving me some strange results.
$conn = New-Object System.Data.SqlClient.SqlConnection
$conn.ConnectionString = "Data Source=PCDK03918;Initial Catalog=Rman;Integrated Security=SSPI;"
$conn.open()
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.connection = $conn
$tidnu = (Get-Date -f dd.MM.yyyy)
$Path = "C:\RMAN"
$Text = "ERROR MESSAGE STACK"
$nodes = #{
'NodeName1' = 'Node1log1.log', 'Node1log2.log', 'Node1log3.log'
'NodeName2' = 'Node2log1.log', 'Node2log2.log'
}
foreach ($NodeName in $nodes.Keys) {
foreach ($Logfil in $nodes[$NodeName]) {
Get-ChildItem $Path -Filter "*.log" |
ForEach-Object {
If (Get-Content $_.FullName | Select-String -Pattern $Text)
{
$cmd.commandtext = "INSERT INTO Error (Datotid, Nodename, Logfil, Error) VALUES('{0}','{1}','{2}','{3}')" -f $tidnu, $NodeName, $Logfil, "Yes"
$cmd.ExecuteNonQuery()
}
else
{
$cmd.commandtext = "INSERT INTO Error (Datotid, Nodename, Logfil, Error) VALUES('{0}','{1}','{2}','{3}')" -f $tidnu, $NodeName, $Logfil, "No"
$cmd.ExecuteNonQuery()
}
}
}
}
$conn.close()
I have created the log files mentioned in $nodes, in the folder, and put the "ERROR MESSAGE STACK" into Node1log1.log and Node1log2.log The rest of the log files are with no "ERROR MESSAGE STACK" inside.
But the result in the database is strange. It says Error = Yes to log files with no "ERROR MESSAGE STACK" inside, and it says Error = No to the same log files some rows down. Plus its inserting double rows and all in all its not doing as intended.
could it be because my
Get-ChildItem $Path -Filter "*.log" |
is wrong by using *.log ?
Or am I simply going completely wrong about this?
EDIT Once more:
Not sure what I was thinking yesterday, but I believe i have solved it now.
Get-ChildItem $Path -Filter "*.log" |
Will of course not work.
Get-ChildItem $Path -Filter $logfil |
Gives much more sense, and now my databse output is looking much more correct.
#Ansgar Wiechers - Thank you for pointing me in the right direction. I learned alot from this.
Consider using a hashtable for this:
$logs = #{
'Log1.log' = 'Node1'
'Log2.log' = 'Node2'
'Log3.log' = 'Node3'
}
That way you can iterate over the logs like this:
foreach ($Logfil in $logs.Keys) {
$NodeName = $logs[$Logfil]
...
}
If you have more than one log file per node name, it would be more efficient to reverse the mapping and store the log file names in an array:
$nodes = #{
'Node1' = 'Log1.log', 'Log2.log', 'Log3.log'
'Node2' = 'Log4.log', 'Log5.log'
}
Then you can process the logfiles with a nested loop like this:
foreach ($NodeName in $nodes.Keys) {
foreach ($Logfil in $nodes[$NodeName]) {
...
}
}
You should be able to fit your pipeline into either loop without further modifications.
Edit: As an optimization you could do something like this to avoid needlessly fetchin logs with each iteration of the outer loop:
$logs = Get-ChildItem $Path -Filter '*.log'
foreach ($NodeName in $nodes.Keys) {
$logs | ? { $nodes[$NodeName] -contains $_.Name } | % {
...
}
}

PowerShell: Set-Content having issues with "file already in use"

I'm working on a PowerShell script that finds all the files with PATTERN within a given DIRECTORY, prints out the relevant lines of the document with the PATTERN highlighted, and then replaces the PATTERN with a provided REPLACE word, then saves the file back. So it actually edits the file.
Except I can't get it to alter the file, because Windows complains about the file already being open. I tried several methods to solve this, but keep running into the issue. Perhaps someone can help:
param(
[string] $pattern = ""
,[string] $replace = ""
,[string] $directory ="."
,[switch] $recurse = $false
,[switch] $caseSensitive = $false)
if($pattern -eq $null -or $pattern -eq "")
{
Write-Error "Please provide a search pattern." ; return
}
if($directory -eq $null -or $directory -eq "")
{
Write-Error "Please provide a directory." ; return
}
if($replace -eq $null -or $replace -eq "")
{
Write-Error "Please provide a string to replace." ; return
}
$regexPattern = $pattern
if($caseSensitive -eq $false) { $regexPattern = "(?i)$regexPattern" }
$regex = New-Object System.Text.RegularExpressions.Regex $regexPattern
function Write-HostAndHighlightPattern([string] $inputText)
{
$index = 0
$length = $inputText.Length
while($index -lt $length)
{
$match = $regex.Match($inputText, $index)
if($match.Success -and $match.Length -gt 0)
{
Write-Host $inputText.SubString($index, $match.Index) -nonewline
Write-Host $match.Value.ToString() -ForegroundColor Red -nonewline
$index = $match.Index + $match.Length
}
else
{
Write-Host $inputText.SubString($index) -nonewline
$index = $inputText.Length
}
}
}
Get-ChildItem $directory -recurse:$recurse |
Select-String -caseSensitive:$caseSensitive -pattern:$pattern |
foreach {
$file = ($directory + $_.FileName)
Write-Host "$($_.FileName)($($_.LineNumber)): " -nonewline
Write-HostAndHighlightPattern $_.Line
%{ Set-Content $file ((Get-Content $file) -replace ([Regex]::Escape("[$pattern]")),"[$replace]")}
Write-Host "`n"
Write-Host "Processed: $($file)"
}
The issue is located within the final block of code, right at the Get-ChildItem call. Of course, some of the code in that block is now a bit mangled due to me trying to fix the problem then stopping, but keep in mind the intent of that part of the script. I want to get the content, replace the words, then save the altered text back to the file I got it from.
Any help at all would be greatly appreciated.
Removed my previous answer, replacing it with this:
Get-ChildItem $directory -recurse:$recurse
foreach {
$file = ($directory + $_.FileName)
(Get-Content $file) | Foreach-object {
$_ -replace ([Regex]::Escape("[$pattern]")),"[$replace]")
} | Set-Content $file
}
Note:
The parentheses around Get-Content to ensure the file is slurped in one go (and therefore closed).
The piping to subsequent commands rather than inlining.
Some of your commands have been removed to ensure it's a simple test.
Just a suggestion but you might try looking at the documentation for the parameters code block. There is a more efficient way to ensure that a parameter is entered if you require it and to throw an error message if the user doesn't.
About_throw: http://technet.microsoft.com/en-us/library/dd819510.aspx
About_functions_advanced_parameters: http://technet.microsoft.com/en-us/library/dd347600.aspx
And then about using Write-Host all the time: http://powershell.com/cs/blogs/donjones/archive/2012/04/06/2012-scripting-games-commentary-stop-using-write-host.aspx
Alright, I finally sat down and just typed everything sequentially in PowerShell, then used that to make my script.
It was actually really simple;
$items = Get-ChildItem $directory -recurse:$recurse
$items |
foreach {
$file = $_.FullName
$content = get-content $file
$newContent = $content -replace $pattern, $replace
Set-Content $file $newcontent
}
Thanks for all your help guys.

Resources