In C, I have the following code
struct _file_header* file_header = (struct _file_header*)(cfg_bytes + (cfg_size - 16));
Which effectively fills the _file_header structure with the bytes from memory. I want to do the same thing in Powershell, but I am confused with the [System.Text.Encoding]::ASCII.GetBytes returning some bytes that aren't related to the data I see in cfg_bytes
Take the following PowerShell code,
$cfg_bytes = (Get-Content $env:ProgramData'\config.bin' -Raw)
$cfg_size = [System.Text.Encoding]::ASCI.GetByteCount($cfg_bytes)
$file_header = $cfg_bytes.Substring($cfg_size - 16, 16)
When I Write-Output [System.Text.Encoding]::ASCII.GetBytes($file_header), the output is not the same as I see from my debugger memory viewer. How can I obtain the bytes in the same format as the C example, such that I'd be able to read the same structure in PowerShell?
As Santiago Squarzon
suggests, use the following to directly get a file's raw bytes in PowerShell:
In Windows PowerShell:
$cfg_bytes = Get-Content -Encoding Byte -Raw $env:ProgramData\config.bin
In PowerShell (Core) 7+:
$cfg_bytes = Get-Content -AsByteStream -Raw $env:ProgramData\config.bin
Note: This breaking change in syntax between the two PowerShell editions is unfortunate, and arguably should never have happened - see GitHub issue #7986.
Note that adding -Raw to the Get-Content call - when combined with -Encoding Byte / -AsByteStream - efficiently returns all bytes as a single array, strongly typed as [byte[]].
Without -Raw, the bytes would be streamed - i.e., output to the pipeline one by one - which, if the output isn't accumulated, keeps memory use constant - but is much slower.
Related
Using BorderAround emits "True" to the console.
$range = $sum_wksht.Range('B{0}:G{0}' -f ($crow))
$range.BorderAround(1, -4138)
This can be overcome by using one of the following.
$wasted = $range.BorderAround(1, -4138)
[void]$range.BorderAround(1, -4138)
Why is this needed? Am I not creating the range correctly? Is there a better workaround?
Why is this needed?
It is needed, because the BorderAround method has a return value and, in PowerShell, any command or expression that outputs (returns) data is implicitly output to the (success) output stream, which by default goes to the host, which is typically the console window (terminal) in which a PowerShell session runs.
That is, the data shows in the console/terminal, unless it is:
captured ($var = ...)
sent through the pipeline for further processing (... | ...; the last pipeline segment's command may or may not produce output itself)
redirected (... >)
or any combination thereof.
That is:
$range.BorderAround(1, -4138)
is (more efficient) shorthand for:
Write-Output $range.BorderAround(1, -4138)
(Explicit use of Write-Output is rarely needed.)
Since you don't want that output, you must suppress it, for which you have several options:
$null = ...
[void] (...)
... > $null
... | Out-Null
$null = ... may be the best overall choice, because:
It conveys the intent to suppress up front
While [void] = (...) does that too, it often requires you to enclose an expression in (...) for syntactic reasons; e.g., [void] 1 + 2 doesn't work as intended, only [void] (1 + 2); similarly, a command must always be enclosed in (...); [void] New-Item test.txt doesn't work, only [void] (New-Item test.txt) does.
It performs well with both command output (e.g., $null = Get-AdUser ...) and expression output (e.g., $null = $range.BorderAround(1, -4138)).
Conversely, avoid ... | Out-Null, because it is generally much slower (except in the edge case of a side effect-free expression's output in PowerShell (Core) 6+)[1].
However, if you need to silence all output streams - not just the success output, but also errors, verbose output, ... - you must use *> $null
Why does PowerShell produce output implicitly?
As a shell, PowerShell's output behavior is based on streams, as in traditional shells such as cmd.exe or Bash. (While traditional shells have 2 output streams - stdout and stderr - PowerShell has 6, so as to provide more sophisticated functionality - see about_Redirection.)
A cmdlet, script, or function can write to the output streams as often as it wants, and such output is usually instantly available for display but notably also to potential consumers, which enables the streaming, one-by-one processing that the pipeline provides.
This contrasts with traditional programming languages, whose output behavior is based on return values, typically provided via the return keyword, which conflates output data (the return value) with flow control (exit the scope and return to the caller).
A frequent pitfall is to expect PowerShell's return statement to act the same, but it doesn't: return <val> is just syntactic sugar for <val>; return, i.e., implicit output of <val> followed by an unconditional return of control to the caller; notably, the use of return does not preclude generation of output from earlier statements in the same scope.
Unlike traditional shells, PowerShell doesn't require an explicit write-to-the-output stream command in order to produce output:
While PowerShell does have a counterpart to echo, namely Write-Output, its use is rarely needed.
Among the rare cases where Write-Output is useful is preventing enumeration of a collection on output with -NoEnumerate, or to use common parameter -OutVariable to both output data and capture it in a variable (which is generally only needed for expressions, because cmdlets and advanced functions / scripts themselves support -OutVariable).
The implicit output behavior:
is generally a blessing:
for interactive experimentation - just type any statement - notably including expressions such as [IO.Path]::GetExtension('foo.txt') and [math]::Pow(2, 32) - and see its output (akin to the behavior of a REPL).
for writing concise code that doesn't need to spell out implied behavior (see example below).
can occasionally be a pitfall:
for users accustomed to the semantics of traditional programming languages.
due to the potential for accidental pollution of the output stream from statements that one doesn't expect to produce output, such as in your case; a more typical example is the .Add() method of the [System.Collections.ArrayList] class unexpectedly producing output.
Example:
# Define a function that takes an array of integers and
# outputs their hex representation (e.g., '0xa' for decimal 10)
function Get-HexNumber {
param([int[]] $numbers)
foreach ($i in $numbers) {
# Format the integer at hand
# *and implicitly output it*.
'0x{0}' -f $i.ToString('x')
}
}
# Call the function with integers 0 to 16 and loop over the
# results, sleeping 1 second between numbers.
Get-HexNumber (0..16) | ForEach-Object { "[$_]"; Start-Sleep 1 }
The above yields the following:
[0x0]
# 1-second pause
[0x1]
# 1-second pause
[0x2]
...
[0x10]
This demonstrates the streaming aspect of the behavior: Get-HexNumber's output is available to the ForEach-Object cmdlet call as it is being produced, not after Get-HexNumber has exited.
[1] In PowerShell (Core) 6+, Out-Null has an optimization if the only preceding pipeline segment is a side effect-free expression rather than a method or command call; e.g., 1..1e6 | Out-Null executes in almost no time, because the expression is seemingly not even executed. However, such a scenario is atypical, and the functionally equivalent Write-Output (1..1e6) | Out-Null takes a long time to run, much longer than $null = Write-Output (1..1e6).
I have an array that contains MBs already in the values. This is how MS DPM returns data written to a tape. I would like to sum them together. Is there an easy one liner to accommodate for this?
MB is a recognized numeric suffix in PowerShell's native grammar, so you can parse and evaluate your size strings with Invoke-Expression:
PS ~> Invoke-Expression '2401927.56MB'
2517924115906.56
You'll want to do some basic input validation to make sure it's actually a numeric sequence, and remove the thousand separator:
$Tapes.DataWrittenDisplayString |ForEach-Object {
# remove commas and whitespace
$dataWritten = $_ -replace '[,\s]'
# ensure it's actually a number in the expected format
if($dataWritten -match '^\d+(?:\.\d+)?[kmgtp]b$'){
# let PowerShell do the rest
$dataWritten |Invoke-Expression
}
}
Here's something I don't understand.
When I define a variable:
$v = [byte]2, [byte]3
and check its type:
$v.getType().name
I get
Object[]
I then format $v:
'{0} {1}' -f $v
which prints
2 3
Now, if I get a file's first two bytes:
$f = (get-content 'xyz.txt' -encoding byte -readCount 2 -totalCount 2)
and check its type:
$f.getType().name
I get the same type as before: Object[].
However, unlike with $v, I cannot format $f:
'{0} {1}' -f $f
I get the error message Error formatting a string: Index (zero based) must be greater than or equal to zero and less than the size of the, although the length of the array is 2:
$f.length
returns
2
I don't understand why this is and would appreciate an explanation.
The behavior should be considered a bug in the -f operator; it is present as of v7.1 and reported in GitHub issue #14355; it does not affect other operators with array operands, such as -split or -in.
The workaround is to cast $f to [array] or, if creating a copy of the array is acceptable, #($f):
'abc' > xyz.txt
$f = get-content 'xyz.txt' -encoding byte -readCount 2 -totalCount 2
'{0} {1}' -f ([array] $f)
Note: Using #(), the array-subexpression operator - ... - #($f) - as Mathias R. Jessen notes - is the even simpler option, but do note that using #() involves cloning (creating a shallow copy of) the array, whereas the [array] cast in this case does not.
The alternative is to apply the [array] cast as a type constraint (by placing it to the left of the $f = ... assignment):
'abc' > xyz.txt
[array] $f = (get-content 'xyz.txt' -encoding byte -readCount 2 -totalCount 2)
'{0} {1}' -f $f
Note:
In PowerShell [Core] v6+, you must use -AsByteStream in lieu of -Encoding Byte.
The problem can also be avoided if -ReadCount 2 is omitted, but note that that decreases the performance of the command, because the bytes are then emitted one by one; that is, with -ReadCount 2 -TotalCount 2 a single object is emitted that is a 2-byte array as a whole, whereas just -TotalCount 2 emits the individual bytes, one by one to the pipeline, in which case it is then the PowerShell engine itself that collects these bytes in an [object[]] array for the assignment.
Note that applying #() directly to the command - #(get-content ...) - would not work in this case, because #(), due to parameter combination -ReadCount 2 -TotalCount 2, receives a single output object that happens to be an array as a whole and therefore wraps that single object in another array. This results in a single-element array whose element is the original 2-element array of bytes; for more information about how #(...) works, see this answer.
Background information:
The problem is an invisible [psobject] wrapper around each array returned by Get-Content -ReadCount (just one in this case), which unexpectedly causes the $f array passed to -f not to be recognized as such.
Note that PowerShell's other array-based operators, such as -in and -replace, are not affected.
The wrapper can be bypassed in two ways:
$f.psobject.BaseObject
casting to [array], as shown at the top.
Note:
Generally, output objects produced by cmdlets - as opposed to output produced by PowerShell code - have generally invisible [psobject] wrappers; mostly, they are benign, because PowerShell usually just cares about the .NET object being wrapped, not about the wrapper, but on occasion problems arise, such as in this case - see GitHub issue #5579 for a discussion of the problem and other contexts in which it manifests.
In order to test if a given object has a [psobject] wrapper, use -is [psobject]; e.g.:
$var = 1
$var -is [psobject] # -> $false
$var = Write-Output 1
$var -is [psobject] # -> $true, due to use of a cmdlet.
# You can also test command output directly.
(Write-Output 1) -is [psobject] # -> $true
I want to compare two txt files with process information in PowerShell.
I looked up plenty of websites and there is always this simple
Compare-Object $(Get-Content [RandomFilePath]) $(Get-Content [RandomFilePath])
quote.
Now for some reason, whenever I try to do this with txt files that contain process information, the shell always outputs the whole content of both files, instead of the differences. However when I compare two txt files with random words in each line, the output correctly states the differences.
For some reason this only happens, when I compare two txt files, that contain process information.
Here's the code I used. I already changed directory beforehand, however I also tried it with the whole file path just in case but got the same result.
Compare-Object $(Get-Content proc.txt) $(Get-Content proc1.txt)
The content of both files is just a plain (with different processes running for each file)
Get-Process > proc.txt
I expect output like this:
InputObject SideIndicator
----------- -------------
System.Diagnostics.Process (EXCEL) =>
System.Diagnostics.Process (freecell) =>
System.Diagnostics.Process (notepad) =>
System.Diagnostics.Process (dexplore) <=
However what I get is exactly what gets written in a txt file if you enter
Get-Process > file.txt
Edit:
When I write it like this:
$a = Get-Process
notepad
$b = Get-Process
Compare-Object $a $b
I get:
System.Diagnostics.Process (notepad) =>
However, when I used txt files with the information from Get-Process in them instead of these variables, the console outputs the whole content of the file instead of the difference in them like the above.
The output redirection operator writes the default textual representation of the Get-Process output to a file. Meaning your output is text in tabular form, not Process objects. The tabular output contains information like CPU and memory usage, which vary over time, as well as PID and Handles, which change across invocations of a process. Hence it's pretty likely that most (if not all) of the lines in your input files are differing in at least one of these values. Meaning you simply don't have matching lines. Period.
What you actually seem to want to compare is just the process names. Of course you could parse those out of the text files, but I wouldn't recommend it. It's far better to fix your input:
Get-Process | Select-Object -Expand Name > proc.txt
Then you can compare 2 output files like this:
Compare-Object (Get-Content proc.txt) (Get-Content proc1.txt)
However, if you also require the other process information you may want to consider using Export-Clixml or Export-Csv instead:
Get-Process | Export-Clixml proc.xml
# or
Get-Process | Export-Csv proc.csv -NoType
Then you can compare 2 output files like this:
Compare-Object (Import-Clixml proc.xml) (Import-Clixml proc1.xml) -Property Name -PassThru
# or
Compare-Object (Import-Csv proc.csv) (Import-Csv proc1.csv) -Property Name -PassThru
Not sure what causes this but here's a troubleshooting tip.
Use following:
Compare-Object $(Get-Content proc.txt) $(Get-Content proc.txt)
There should be no difference now. Just to test if Compare works correctly using the process file.
Export processes to .csv file format and compare them again. that might work as well.
I have what may be an odd issue. I've got a Powershell script that's supposed to watch a directory for files, then move and rename them. It checks the output directory to see if a file with that name already exists with the form "Trip ID X Receipts Batch Y.pdf" (original output from the web form will always be that Y=1) and if it does replace Y with whatever the highest existing number of Y for other files with Trip ID X is. If there isn't one already, it'll just stay that Y=1. It does this successfully except on the second match, where instead of 2 Y will equal a number that varies depending on the file. This seems to be the file size in bytes plus 1. Example results of the script (from copy/pasting the same source file into the watched directory):
Trip ID 7 Receipts Batch 1.pdf
Trip ID 7 Receipts Batch 126973.pdf
Trip ID 7 Receipts Batch 3.pdf
Trip ID 7 Receipts Batch 4.pdf
The relevant portion of my code is here:
$REFile = "Trip ID " + $TripID + " Receipts Batch "
$TripIDCheck = "Trip ID " + $TripID
$TripFileCount = Get-ChildItem $destination |Where-Object {$_.Name -match $TripIDCheck}
$BatchCount = $TripFileCount.GetUpperBound(0) + 1
$destinationRegEx = $destination + $REFile + $BatchCount + ".pdf"
Move-Item -Path $path -Destination $destinationRegEx -Force
For counting the number of items in the array, I've used what you see above as well as $TripFileCount.Length, and $TripFileCount.Count. They all behave the same, seemingly taking the file, examining its size, setting the Y value for the second item to that, but then treating the third, fourth, etc. items as expected. For the life of me, I can't figure out what's going on. Have any of you ever seen something like this?
Edit: Trying to force $TripFileCount as an array with
$TripFileCount = #(Get-ChildItem $destination |Where-Object {$_.Name -match $TripIDCheck})
doesn't work either. It still does this.
As TessellatingHeckler states, your symptom indeed suggests that you're not accounting for the fact that cmdlets such as Get-ChildItem do not always return an array, but may return a single, scalar item (or no items at all).
Therefore, you cannot blindly invoke methods / properties such as .GetUpperBound() or .Length on such a cmdlet's output. There are two workarounds:
Use array subexpression operator #(...) to ensure that the enclosed command's output is treated as an array, even if only a single object is returned or none at all.
In PSv3+, use the .Count property even on a single object or $null to implicitly treat them as if they were an array.
The following streamlined solution uses the .Count property on the output from the Get-ChildItem call, which works as intended in all 3 scenarios: Get-ChildItem matches nothing, 1 file, or multiple files.
$prefix = "Trip ID $TripID Receipts Batch "
$suffix = '.pdf'
$pattern = "$prefix*$suffix"
$count = (Get-ChildItem $destination -Filter $pattern).Count
Move-Item -Path $path -Destination (Join-Path $destination "$prefix$($count+1)$suffix")
Note:
If you're using PowerShell v2, then prepend # to (...). #(...), the array subexpression operator, ensures that the output from the enclosed command is always treated as an array, even if it comprise just 1 object or none at all.
In PowerShell v3 and above, this behavior is conveniently implicit, although there are caveats - see this answer of mine.