i have a number of strings like this this that i would like powershell to reevaluate\convert to an array (like what would happen if you just wrote the same code in ISE without the single quotes).
$String = '#("a value","b value","c value")'
Is there a easier way to do this than stripping the '#' & '()' out of the string and using -split?
Thanks for the help in advance.
As long as the string contains a valid expression, you can use the [scriptblock]::Create(..) method:
$String = '#("a value","b value","c value")'
& ([scriptblock]::Create($String))
Invoke-Expression would also work and in this case would be mostly the same thing.
However, the nice feature about Script Blocks, as zett42 pointed out in a comment, is that with them we can validate that arbitrary code execution is forbidden with it's CheckRestrictedLanguage method.
In example below, Write-Host is an allowed command and a string containing only 'Write-Host "Hello world!"' would not throw an exception however, assignment statements or any other command not listed in $allowedCommands will throw and the script block will not be executed.
$String = #'
Write-Host "Hello world!"
$stream = [System.Net.Sockets.TcpClient]::new('google.com', 80).GetStream()
'#
[string[]] $allowedCommands = 'Write-Host'
[string[]] $allowedVaribales = ''
try {
$scriptblock = [scriptblock]::Create($String)
$scriptblock.CheckRestrictedLanguage(
$allowedCommands,
$allowedVaribales,
$false # Argument to allow Environmental Variables
)
& $scriptblock
}
catch {
Write-Warning $_.Exception.Message
}
Another alternative is to run the expression in a Runspace with ConstrainedLanguage Mode. This function can make it really easy.
using namespace System.Management.Automation.Language
using namespace System.Collections.Generic
using namespace System.Management.Automation.Runspaces
function Invoke-ConstrainedExpression {
[CmdletBinding(DefaultParameterSetName = 'ScriptBlock')]
param(
[Parameter(ParameterSetName = 'Command', Mandatory, ValueFromPipeline, Position = 0)]
[string] $Command,
[Parameter(ParameterSetName = 'ScriptBlock', Mandatory, Position = 0)]
[scriptblock] $ScriptBlock,
[Parameter()]
[Management.Automation.PSLanguageMode] $LanguageMode = 'ConstrainedLanguage',
# When using this switch, the function inspects the AST to find any variable
# not being an assigned one in the expression, queries the local state to find
# it's value and injects that variable to the Initial Session State of the Runspace.
[Parameter()]
[switch] $InjectLocalVariables
)
process {
try {
$Expression = $ScriptBlock
if($PSBoundParameters.ContainsKey('Command')) {
$Expression = [scriptblock]::Create($Command)
}
# bare minimum for the session state
$iss = [initialsessionstate]::CreateDefault2()
# set `ContrainedLanguage` for this session
$iss.LanguageMode = $LanguageMode
if($InjectLocalVariables.IsPresent) {
$ast = $Expression.Ast
$map = [HashSet[string]]::new([StringComparer]::InvariantCultureIgnoreCase)
$ast.FindAll({ $args[0] -is [AssignmentStatementAst] }, $true).Left.Extent.Text |
ForEach-Object { $null = $map.Add($_) }
$variablesToInject = $ast.FindAll({
$args[0] -is [VariableExpressionAst] -and
-not $map.Contains($args[0].Extent.Text)
}, $true).VariablePath.UserPath
foreach($var in $variablesToInject) {
$value = $ExecutionContext.SessionState.PSVariable.GetValue($var)
$entry = [SessionStateVariableEntry]::new($var, $value, '')
$iss.Variables.Add($entry)
}
}
# create the PS Instance and add the expression to invoke
$ps = [powershell]::Create($iss).AddScript($Expression)
# invoke the expression
[Collections.Generic.List[object]] $stdout = $ps.Invoke()
$streams = $ps.Streams
$streams.PSObject.Properties.Add([psnoteproperty]::new('Success', $stdout))
$streams
}
finally {
if($ps) { $ps.Dispose() }
}
}
}
Now we can test the expression using Constrained Language:
Invoke-ConstrainedExpression {
Write-Host 'Starting script'
[System.Net.WebClient]::new().DownloadString($uri) | iex
'Hello world!'
}
The output would look like this:
Success : {Hello world!}
Error : {Cannot create type. Only core types are supported in this language mode.}
Progress : {}
Verbose : {}
Debug : {}
Warning : {}
Information : {Starting script}
DevBlogs Article: PowerShell Constrained Language Mode has some nice information and is definitely worth a read.
Related
I've been trying to write a utility function that will return initialized [System.Data.DataTable] object data type.
I am aware of the annoying PowerShell functions return behavior where it tries to "unroll" everything into a [System.Array] return type.
Previously I have always managed to get around that problem.
Usually using the "comma trick" to return your own array #(,$result) aways worked - however this time this does not seem to make a difference and I've run out of options...
Another trick I normally use is the $null assignments within the Process block (see my code bellow) - this way I fool the pipeline there is nothing to "unroll" on the output...
I am not taking impossible for an answer, so far based on my experience nothing is impossible in PowerShell :)
Here is my code:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
return ,$datatable
}
}
I hope someone can get this code working...
Rather than return use Write-Output -NoEnumerate. For example:
function New-DataTable {
$datatable = New-Object System.Data.DataTable
$null = $datatable.Columns.Add("x",[int])
$null = $datatable.Columns.Add("y",[int])
$null = $datatable.Rows.Add(#(1,2))
$null = $dataTable.Rows.Add(#(3,4))
Write-Output -NoEnumerate $datatable
}
New-DataTable | Get-Member
Note however that if you just type New-DataTable, it might look like enumberated rows, but Get-Member tells you the actual type returned.
I got the function from the question to return DataTable type when I used LoadWithPartialName to load the assembly containing the type and pipe it out with Out-Null.
Don't ask my why, but feel free to comment if you know the reason.
The working function code is below. Note the return statement is not necessary I only use it to improve code readability:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
[System.Reflection.Assembly]::LoadWithPartialName("System.Data") | Out-Null
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
return ,$datatable
}
}
To summarize all known possible solutions to the problem of forcing PowerShell function to return specific data type:
use $null assignments
use comma to return an array ,$variable
use LoadWithPartialName("Assembly.Name") | Out-Null
use Write-Output -NoEnumerate $variable to return the type - credit goes to Burt_Harris
Finally, after the input from Burt_Harris (THANKS Burt!) the final working version of the function from this question is this:
function Get-SourceDataTable
{
[OutputType([System.Data.DataTable])]
[cmdletbinding()]
param(
[parameter(Mandatory=$true, Position=0)]
[System.Data.SqlClient.SqlBulkCopy] $Destination,
[parameter(Mandatory=$true, Position=1)]
[System.Collections.Specialized.OrderedDictionary] $ColumnDefinition,
[parameter(Mandatory=$false, Position=2)]
[int]$ColumnStartIndex = 0
)
BEGIN{
#[System.Reflection.Assembly]::LoadWithPartialName("System.Data") | Out-Null
$datatable = New-Object System.Data.DataTable
$colIndex = $ColumnStartIndex
}
PROCESS{
$ColumnDefinition.Keys |
foreach {
$null = $datatable.Columns.Add($_, $ColumnDefinition[$_]) # define each column name and data type
$null = $Destination.ColumnMappings.Add($_, $colIndex) # map column to destination table
$colIndex++
}
}
END{
#return ,$datatable
Write-Output -NoEnumerate $datatable
}
}
I am writing a PowerShell function/script (using the version that ships with Windows 10, which I believe is 5.0) to take in a GZip compressed Base64 string and decompress it, and then decode it under the assumption the original uncompressed/decoded string was Unicode encoded.
I am trying to instantiate a new object instance of type MemoryStream using this constructor and the New-Object cmdlet. It takes one parameter, which is an array of bytes.
PowerShell is unable to find a valid overload that accepts the array of bytes I am trying to pass as the constructor's parameter. I believe the issue is due to the array's relatively large length. Please see my code below:
Function DecompressString()
{
param([parameter(Mandatory)][string]$TextToDecompress)
try
{
$bytes = [Convert]::FromBase64String($TextToDecompress)
$srcStreamParameters = #{
TypeName = 'System.IO.MemoryStream'
ArgumentList = ([byte[]]$bytes)
}
$srcStream = New-Object #srcStreamParameters
$dstStream = New-Object -TypeName System.IO.MemoryStream
$gzipParameters = #{
TypeName = 'System.IO.Compression.GZipStream'
ArgumentList = ([System.IO.Stream]$srcStream, [System.IO.Compression.CompressionMode]::Decompress)
}
$gzip = New-Object #gzipParameters
$gzip.CopyTo($dstStream)
$output = [Text.Encoding]::Unicode.GetString($dstStream.ToArray())
Write-Output $output
}
catch
{
Write-Host "$_" -ForegroundColor Red
}
finally
{
if ($gzip -ne $null) { $gzip.Dispose() }
if ($srcStream -ne $null) { $srcStream.Dispose() }
if ($dstStream -ne $null) { $dstStream.Dispose() }
}
}
DecompressString
$ExitPrompt = Read-Host -Prompt 'Press Enter to Exit'
The error message I get is: Cannot find an overload for "MemoryStream" and the argument count: "1764".
Can anyone please clarify how I can get the script interpreter to use the constructor correctly with a large byte array?
I thought I would share an answer to this question which I found very interesting, solution to the error has already been provided by Lance U. Matthews in his helpful comment, by adding the unary operator , before the $bytes assigned to the ArgumentList of New-Object, by doing so, the $bytes are passed as a single argument (array of bytes) and not as individual elements to the constructor of System.IO.MemoryStream:
$bytes = [Convert]::FromBase64String($compressedString)
$srcStreamParameters = #{
TypeName = 'System.IO.MemoryStream'
ArgumentList = , $bytes
}
$srcStream = New-Object #srcStreamParameters
Beginning in PowerShell 5.0 and going forward, you can construct your memory stream with the following syntax, which is more efficient and straight forward:
$srcStream = [System.IO.MemoryStream]::new($bytes)
As for the functions (Compress & Expand), I would like to share my take on these cool functions.
Required using statements and ArgumentCompleter for -Encoding Parameter.
using namespace System.Text
using namespace System.IO
using namespace System.IO.Compression
using namespace System.Collections
using namespace System.Management.Automation
using namespace System.Collections.Generic
using namespace System.Management.Automation.Language
Add-Type -AssemblyName System.IO.Compression
class EncodingCompleter : IArgumentCompleter {
[IEnumerable[CompletionResult]] CompleteArgument (
[string] $commandName,
[string] $parameterName,
[string] $wordToComplete,
[CommandAst] $commandAst,
[IDictionary] $fakeBoundParameters
) {
[CompletionResult[]] $arguments = foreach($enc in [Encoding]::GetEncodings().Name) {
if($enc.StartsWith($wordToComplete)) {
[CompletionResult]::new($enc)
}
}
return $arguments
}
}
Compression from string to Base64 GZip compressed string:
function Compress-GzipString {
[cmdletbinding()]
param(
[Parameter(Mandatory, ValueFromPipeline)]
[string] $String,
[Parameter()]
[ArgumentCompleter([EncodingCompleter])]
[string] $Encoding = 'utf-8',
[Parameter()]
[CompressionLevel] $CompressionLevel = 'Optimal'
)
try {
$enc = [Encoding]::GetEncoding($Encoding)
$outStream = [MemoryStream]::new()
$gzip = [GZipStream]::new($outStream, [CompressionMode]::Compress, $CompressionLevel)
$inStream = [MemoryStream]::new($enc.GetBytes($string))
$inStream.CopyTo($gzip)
}
catch {
$PSCmdlet.WriteError($_)
}
finally {
$gzip, $outStream, $inStream | ForEach-Object Dispose
}
try {
[Convert]::ToBase64String($outStream.ToArray())
}
catch {
$PSCmdlet.WriteError($_)
}
}
Expansion from Base64 GZip compressed string to string:
function Expand-GzipString {
[cmdletbinding()]
param(
[Parameter(Mandatory, ValueFromPipeline)]
[string] $String,
[Parameter()]
[ArgumentCompleter([EncodingCompleter])]
[string] $Encoding = 'utf-8'
)
try {
$enc = [Encoding]::GetEncoding($Encoding)
$bytes = [Convert]::FromBase64String($String)
$outStream = [MemoryStream]::new()
$inStream = [MemoryStream]::new($bytes)
$gzip = [GZipStream]::new($inStream, [CompressionMode]::Decompress)
$gzip.CopyTo($outStream)
$enc.GetString($outStream.ToArray())
}
catch {
$PSCmdlet.WriteError($_)
}
finally {
$gzip, $outStream, $inStream | ForEach-Object Dispose
}
}
And for the little Length comparison, querying the Loripsum API:
$loremIp = Invoke-RestMethod loripsum.net/api/10/long
$compressedLoremIp = Compress-GzipString $loremIp
$loremIp, $compressedLoremIp | Select-Object Length
Length
------
8353
4940
(Expand-GzipString $compressedLoremIp) -eq $loremIp # => Should be True
These 2 functions as well as Compression From File Path and Expansion from File Path can be found on this repo.
I need to store data like below for TextToColumns Excel automation.
I need to implement Code-2 or Code-3 or Code-4 is that any way to achieve?
I have more than 350+ data so I cant use Code-1, that's not fair for me.
Code-1: working fine
$var = (1,2),(2,2),(3,2),(4,2),(5,2),(6,2)........(300,2)
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-2: not Working
$var = #((1,2)..(300,2))
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-3: not Working
$var = #()
#forloop upto 300
{ $var += ($i,2) }
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
Code-4: not Working
[array]$var = 1..300 | foreach-object { ,#($_, 2) }
$ColumnA.texttocolumns($colrange,1,-412,$false,$false,$false,$false,$false,$true,"|",$var)
I can't fully explain what happens here but I guess that it is related to the fact that the texttocolumns requires an (deferred) expression rather than an (evaluated) object.
Meaning that the following appears to work for the Minimal, Reproducible Example from #mclayton:
$Var = Invoke-Expression ((1..6 |% { "($_, `$xlTextFormat)" }) -Join ',')
And expect the following to work around the issue in the initial question:
$Var = Invoke-Expression ((1..300 |% { "($_, 2)" }) -Join ',')
Not an answer - just documenting some research to save others some time...
I can repro the issue here with the following code:
$xl = new-object -com excel.application;
$xl.Visible = $true;
$workbook = $xl.Workbooks.Add();
$worksheet = $workbook.Worksheets.Item(1);
$worksheet.Range("A1") = "aaa|111";
$worksheet.Range("A2") = "bbb|222";
$worksheet.Range("A3") = "ccc|333";
$worksheet.Range("A4") = "ddd|444";
$worksheet.Range("A5") = "eee|555";
$worksheet.Range("A6") = "fff|666";
which builds a new spreadsheet like this:
If you then run the following it will parse the contents of column A and put the results into columns B and C:
$range = $worksheet.Range("A:A");
$target = $worksheet.Range("B1");
# XlColumnDataType enumeration
# see https://learn.microsoft.com/en-us/office/vba/api/excel.xlcolumndatatype
$xlTextFormat = 2;
# XlTextParsingType enumeration
# see https://learn.microsoft.com/en-us/office/vba/api/excel.xltextparsingtype
$xlDelimited = 1;
# XlTextQualifier enumeration
# https://learn.microsoft.com/en-us/office/vba/api/excel.xltextqualifier
$xlTextQualifierNone = -4142;
$var = (1,$xlTextFormat),(2,$xlTextFormat),(3,$xlTextFormat),(4,$xlTextFormat),(5,$xlTextFormat),(6,$xlTextFormat);
# parse the values in A1:A6 and puts the values in a 2-dimensional array starting at B1
# see https://learn.microsoft.com/en-us/office/vba/api/excel.range.texttocolumns
$result = $range.TextToColumns(
$target, # Destination
$xlDelimited, # DataType
$xlTextQualifierNone, # TextQualifier
$false, # ConsecutiveDelimiter
$false, # Tab
$false, # Semicolon
$false, # Comma
$false, # Space
$true, # Other
"|", # OtherChar
$var # FieldInfo
);
which then looks like this:
However, if you change the declaration for $var to
$var = 1..6 | % { ,#($_, $xlTextFormat) };
you get the following error:
OperationStopped: The remote procedure call failed. (0x800706BE)
and the Excel instance terminates.
So there's something different about these two declarations:
$var = (1,$xlTextFormat),(2,$xlTextFormat),(3,$xlTextFormat),(4,$xlTextFormat),(5,$xlTextFormat),(6,$xlTextFormat);
$var = 1..6 | % { ,#($_, $xlTextFormat) };
but what that is eludes me :-S
I have a function that replaces PackageID in a SCCM task sequence, I would like to capture all those package IDs into a variable, so I would be able to create a report based on that.
The problem is that I already have a foreach loop doing the work, and I can't figure out how to not overwrite the values.
$Driver.PackageID comes from a foreach loop based on $Drivers, which contains
If I run the code I get this as I have Write-Output defined:
Updated code:
function Set-Drivers{
foreach ($Driver in $Drivers) {
Write-Output "Driver Name: $($Driver.Name)"
Write-Output "DriverPackageID: $($Driver.PackageID)"
}
}
$array = #()
$array = Set-Drivers
$hash = [ordered]#{
'DriverName' = $Driver.Name
'DriverID' = $Driver.PackageID
}
$array += New-Object -Typename PSObject -Property $hash
Can someone explain, why I only get the first result in my $array? I can see the values are being overwritten if I run it in debug mode.
Your code is not iterating over the results, but instead only using one of them. This what you intended.
$array = $drivers | foreach {
[ordered]#{
DriverName = $_.Name
DriverID = $_.PackageID
}
}
Your function doesn't return anything. It only writes lines to the console. Then after the function is finished, you create a single object and add that to your array.
Try something like
function Set-Drivers{
$result = foreach ($Driver in $Drivers) {
[PsCustomObject]#{
'DriverName' = $Driver.Name
'DriverID' = $Driver.PackageID
}
}
# output the result
# the comma wraps the result in a single element array, even if it has only one element.
# PowerShell 'flattens' that upon return from the function, leaving the actual resulting array.
,$result
}
$array = Set-Drivers
# show what you've got
$array
I'm trying to set up replication in RavenDB by using PowerShell DSC, but I get this error in the TestScript scriptblock when I try to compile the configuration:
PSDesiredStateConfiguration\Node : Error formatting a string: Input string was not in a correct format.
Here is my scriptblock:
TestScript = {
$result = Invoke-WebRequest -Method GET "http://localhost:8080/Databases/Test/Docs/Raven/Replication/Destinations" -UseBasicParsing
$ravenSlaves = "{0}".Split(",")
foreach($ravenSlave in $ravenSlaves)
{
if($result -notmatch $ravenSlave)
{
return $false
}
}
return $true
} -f ($Node.RavenSlaves)
And RavenSlaves is defined like a string in my ConfigurationData for the nodes like this:
#{
NodeName = "localhost"
WebApplication = "test"
Role = "Master server"
RavenSlaves = "server1,server2"
}
The problem seems to be connected to when I'm using the foreach to iterate over the $ravenSlaves variable, because if I remove the foreach (and the if inside the foreach) the configuration compiles and the mof file is created.
Kiran led me to the right solution by his comment about using the $using modifier in the configuration.
I edited the RavenSlaves property on the node to be an array like this:
#{
NodeName = "localhost"
WebApplication = "test"
Role = "Master server"
RavenSlaves = #("server1,server2")
}
And then I changed the TestScript-block to be like this:
TestScript = {
$result = Invoke-WebRequest -Method GET "http://localhost:8080/Databases/Test/Docs/Raven/Replication/Destinations" -UseBasicParsing
$ravenSlaves = $Node.RavenSlaves
foreach($ravenSlave in $using:ravenSlaves)
{
if($result -notmatch $ravenSlave)
{
return $false
}
}
return $true
}
The script compiled and ran on the server and the replication document in RavenDB was correct.