PS noob here (as will be obvious shortly) but trying hard to get better. In my exchange 2010 environment I import and export huge numbers of .pst files. Many will randomly fail to queue up and once they're not in the queue it's very tedious to sort through the source files to determine which ones need to be run again so I'm trying to write a script to do it.
first I run a dir on the list of pst files and fill a variable with the associated aliases of the accounts:
$vInputlist = dir $vPath -Filter *.pst |%{ get-mailbox -Identity $_.basename| select alias}
Then I fill a variable with the aliases of all the files/accounts that successfully queued:
$vBatch = foreach ($a in (Get-MailboxImportRequest -BatchName $vBatchname)) {get-mailbox $a.mailbox | select alias}
Then I compare the two arrays to see which files I need to queue up again:
foreach($should in $vInputlist){if ($vBatch -notcontains $should){Write-Host $should ""}}
It seems simple enough yet the values in the arrays never match, or not match, as the case may be. I've tried both -contains and -notcontains. I have put in a few sanity checks along the way like exporting the variables to the screen and/or to csv files and the data looks fine.
For instance, when $vInputlist is first filled I send it to the screen and it looks like this:
Alias
MapiEnableTester1.psiloveyou.com
MapiEnableTester2.psiloveyou.com
MapiEnableTester3.psiloveyou.com
MapiEnableTester4.psiloveyou.com
Yet that last line of code I displayed above (..write-host $should,"") will output this:
#{Alias=MapiEnableTester1.psiloveyou.com}
#{Alias=MapiEnableTester2.psiloveyou.com}
#{Alias=MapiEnableTester3.psiloveyou.com}
#{Alias=MapiEnableTester4.psiloveyou.com}
(those all display as a column, not sure why they won't show that way here)
I've tried declaring the arrays like this, $vInputlist = #()
I've tried instead of searching for the alias just cleaning .pst off off the $_.basename using .replace
I've searched on comparing arrays til I'm blue in the fingers and I don't think my comparison is wrong, I believe that somehow no matter how I fill these variables I am corrupting or changing the data so that seemingly matching data simply doesn't.
Any help would be greatly appreciated. TIA
Using -contains to compare objects aren't easy because the objects are never identical even though they have the same property with the same value. When you use select alias you get an array of pscustomobjects with the property alias.
Try using the -expand parameter in select, like
select -expand alias
Using -expand will extract the value of the alias property, and your lists will be two arrays of strings instead, which can be compared using -contains and -notcontains.
UPDATE I've added a sample to show you what happends with your code.
#I'm creating objects that are EQUAL to the ones you have in your code
#This will simulate the objects that get through the "$vbatch -notcontains $should" test
PS > $arr = #()
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester1.psiloveyou.com" }
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester2.psiloveyou.com" }
PS > $arr += New-Object psobject -Property #{ Alias="MapiEnableTester3.psiloveyou.com" }
PS > $arr | ForEach-Object { Write-Host $_ }
#{Alias=MapiEnableTester1.psiloveyou.com}
#{Alias=MapiEnableTester2.psiloveyou.com}
#{Alias=MapiEnableTester3.psiloveyou.com}
#Now this is what you will get if you use "... | select -expand alias" instead of "... | select alias"
PS > $arrWithExpand = $arr | select -expand alias
PS > $arrWithExpand | ForEach-Object { Write-Host $_ }
MapiEnableTester1.psiloveyou.com
MapiEnableTester2.psiloveyou.com
MapiEnableTester3.psiloveyou.com
Related
Is there a non for-loop way to remove some items from a arrayList?
$remotesumerrors = $remoteFiles | Select-String -Pattern '^[a-f0-9]{32}( )' -NotMatch
I want to remove the output of the above from the $remoteFiles var.. is there some pipe way to remove them?
Assuming all of the following:
you do need the results captured in $remotesumerrors separately
that $remoteFiles is a collection of System.IO.FileInfo instances, as output by Get-ChildItem, for instance
it is acceptable to save the result as an invariably new collection back to $remoteFiles,
you can use the .Where() array method as follows (this outperforms a pipeline-based solution based on the Where-Object cmdlet):
# Get the distinct set of the full paths of the files of origin
# from the Select-String results stored in $remotesumerrors
# as a hash set, which allows efficient lookup.
$errorFilePaths =
[System.Collections.Generic.HashSet[string]] $remotesumerrors.Path
# Get those file-info objects from $remoteFiles
# whose paths aren't in the list of the paths obtained above.
$remoteFiles = $remoteFiles.Where({ -not $errorFilePaths.Contains($_.FullName) })
As an aside:
Casting a collection to [System.Collections.Generic.HashSet[T]] is a fast and convenient way to get a set of distinct values (duplicates removed), but note that the resulting hash set's elements are invariably unordered and that, with strings, lookups are by default case-sensitive - see this answer for more information.
Use the Where-Object cmdlet to filter the list:
$remoteFiles = $remoteFiles |Where-Object { $_ |Select-String -Pattern '^[a-f0-9]{32}( )' -NotMatch }
If it truly was a [collections.arraylist], you could remove an element by value. There's also .RemoveAt(), to remove by array index.
[System.Collections.ArrayList]$array = 'a','b','c','d','e'
$array.remove
OverloadDefinitions
-------------------
void Remove(System.Object obj)
void IList.Remove(System.Object value)
$array.remove('c')
$array
a
b
d
e
Let assume that $remoteFiles is a file object of type System.IO.FileInfo. I also assume that you want to filter based on filename.
$remotesumerrors = $remoteFiles.name | Select-String -Pattern '^[a-f0-9]{32}' -NotMatch
What are trying to do with "( )" or what is query that you want to do.
edit: corrected answer based on comment
I'm new to PowerShell. I'm trying to pull a users name and place it in a file. I have two corresponding arrays, $title and $csvfile. $title[0] corresponds with $csvfile[0] and $title[1] to $csvfile[1] and so on. Is it possible to loop this script while increasing the index for both at the same time so that each index runs once but also in sync?
$title = #('jim' 'john' 'james')
$csvfile = #('jim.csv' 'john.csv' 'james.csv')
Get -ADUser -filter {(Title -like "$title") -and (Company -like "Location1")} | Export-Csv c:\temp\$csvfile
Problem 2
The $title array doesn't seem to be iterating one at a time. If I replace $title[$] with any of the listed array items it works. Funny thing is that it DOES create all my .csv files from the $csvfile array, they are just empty. I've done some looking on the web, not sure if my array items are to long or the quotations are not parsing right. Any help would be muchly appreciated.
$title = #(
'Director of Nursing'
'Assistant Director of Nursing'
'Activities Director'
)
$csvfile = #(
'DON'
'ADON'
'ACTIVITIES'
)
for($i=0; $i -lt $title.Count; $i++)
{
#Get-ADUser -Filter { (Title -like "Director of Nursing") -and (Company -like "location1") }| Export-Csv c:\temp\$($csvfile[$i]).csv"
Get-ADUser -filter { (Title -like "$($title[$i])") -and (Company -like "Location1") }| Export-Csv "c:\tempPath\$($csvfile[$i]).csv"
}
If I'm understanding this correctly. You'd like to append the user information from Get-Aduser to the corresponding csv of the users name you got it from? Such as, jim info goes to jim.csv, and so on for each one?
Seems like you're looking for the Foreach loop.
$title = #('jim','john','james')
#$csvfile = #('jim.csv' 'john.csv' 'james.csv')
Foreach($user in $title){
Get-ADUser -filter {(Title -like "$title") -and (Company -like "Location1")} | Export-Csv "c:\temp\$title.csv"}
A Foreach loop goes through a list of objects and performs the same action for every object, ending when it's finished with the last one. The list of objects is typically an array. When you run a loop over a list of objects, we say you're iterating over the list.
The Foreach loop can be used in 3 different ways: Foreach Statement, Foreach-Object cmdlet, or as a foreach() method.
What we're using here is the Foreach statement which is followed by parentheses that contain three elements, in order: a variable, the keyword in, and the object or array to iterate over. As it moves through list ($title-array in this case), Powershell will copy the object it's looking at into the Variable defining each item in the list, $user.
Note: because the variables is just a copy, you cannot directly change the item in the original list.
Please note as well, the items in an array are read separately by adding a comma to the end of each item in the list(if its not in a new line). In the code above, we're appending the same name you're iterating with to the csv file as well.
EDIT: Using for loop. . .
$title = #('jim','john','james')
$csvfile = #('CEO','CFO','CIO')
For($i=0; $i -lt $title.Count; $i++){
Get-ADUser -filter {(Title -like "$($title[$i])") -and (Company -like "Location1")} | Export-Csv "c:\temp\$($csvfile[$i]).csv"}
Matches the output like so:
jim - CEO.csv
john - CFO.csv
james - CIO.csv
With a powershell script
I`m importing a CSV file in an array, everything works fine.
$csv = Import-Csv "C:\test.csv" -delimiter ";"
But I'm not able to find easily a value in a field name PeopleID directly.
The only best method is to loop through all array line and look if the item I`m looking for exist like :
foreach($item in $csv)
{
if ($csv | where {$item.PeopleID -eq 100263} | select *) {
#Good I found it!!!
}
$List.add($item.PeopleID) > $null
}
Instead I decide to import only my column PeopleID directly in an array to make it faster:
$csvPeople = Import-Csv "C:\test.csv" -delimiter ";" | select PeopleID
If you see higher, I also create an array $List that add every PeopleID in my loop.
So I have 2 arrays that are identically
The problem if I use the CONTAINS command:
if($csvPeople -contains 100263)
the result is false
if($List -contains 100263)
the result is true
What can I do to have my $csvPeople array working with "contains" ?
Importing a csv column is faster than looping through result and adding it to a new array, but this array is working.
Do I'm missing somthing when I import my CSV column to have a "working" array ?
thanks
I think you are looking for -like not -contains. -contains is a bit finicky about how it is used. If you replace you if syntax with this:
if($csvPeople -like "*100263*")
You should be good to go. Note the wildcards on either side, I'm putting these here for you since I don't know exactly what your data looks like. You might be either able to remove them or able to change them.
Obligatory -like vs -contains article if you are interested: http://windowsitpro.com/blog/powershell-contains
Also, #AnsgarWiechers comment above will work. I believe you will still need to wrap your number in quotes though. I don't like to do this as it requires an exact match. If you are working with the CSV in excel or elsewhere and you have whitespace or oddball line ending characters then you might not get a hit with -contains.
I just noticed this in your script above where you are doubling your efforts:
foreach($item in $csv) # <-- Here $item is a record in the CSV
{
if ($csv | where {$item.PeopleID -eq 100263} | select *) { # <-- Here you are re-parsing the CSV even though you already have the data in $item
}
$List.add($item.Matricule) > $null
}
Since I don't have the full picture of what you are trying to do I'm providing a solution that allows you to match and take action per record.
foreach($item in $csv)
{
if ($item.PeopleID -eq 100263) {
# Do something with $item here now that you matched it
}
}
To Address your 191 users. I would take a different approach. You should be able to load a raw list of PeopleID's into a variable and then use the -in feature of Where to do a list-to-list comparison.
First load your target PeopleID's into a list. Assuming it is just a flat list, not a csv you could do your comparison like this:
$PeopleID_List = Get-Content YourList.txt
$csv | where { $_.PeopleID -in $PeopleID_List } | % { # Do something with the matched record here. Reference it with $_ }
This basically says for each record in the CSV I want you to check if that record's peopleID is in the $PeopleID_List. If it is, take action on that record.
There is probably a simple way to do this, but I've been hitting my head against a wall for hours at this point. I'm trying to grab several user attributes out of AD, compare two of those attributes, and then modify them based on the differences. However since some users have null values for either their office or department fields which causes compare-object to fail, I have those going into other arrays with a -replace to get rid of the nulls, so my variables look like this:
$UserData = Get-ADuser -filter * -properties physicaldeliveryofficename,department | select samaccountname,physicaldeliveryofficename,department
$Offices = $UserData.physicaldeliveryofficename -replace "^$","N/A"
$Departments = $UserData.department -replace "^$","N/A"
So far so good, but when I loop through to compare values, I start to run into trouble. Looping through the users like this seems to be comparing every element to every other element:
Foreach ($user in $UserData.samaccountname) {
Compare-object $offices $departments -includeqeual}
While not having a loop and using compare-object by itself gives accurate results, but then I'd need a loop to check for matches anyway.
Assuming I just want to determine which users have matching office and department fields (and based off that do a pretty simple Set-ADUser command), how would I go about comparing the values without checking every element against every other element?
Your ForEach loop won't work properly because even though you are going through each user account, you are always comparing the same collection of offices and departments. I wrote this that might give you better results and saves the compare results as part of an object so you can see the user account as well.
Get-ADuser -Filter * -properties physicaldeliveryofficename,department | ForEach {
$Offices = $_.physicaldeliveryofficename -replace "^$","N/A"
$Departments = $_.department -replace "^$","N/A"
$Results = Compare-object $offices $departments -IncludeEqual
[pscustomobject]#{
User = $_.samaccountname
compare = $Results
}
}
I'm looking for the negative intersection of two arrays. Each array has about 20k elements. I'm using a foreach loop over one array and looking each value up in the other array. I'm only keeping elements in the first array not found in the second array:
$deadpaths=#()
$ix=0
ForEach ($f in $FSBuildIDs)
{
if (-not($blArray -like $f)) {$deadpaths+=$paths[$ix]}
$ix++
}
$blArray contains valid IDs. $FSBuildIDs contains the IDs corresponding to the file system paths in $paths. The intent is to only keep the elements in $paths where the corresponding ID in $FSBuildIDS is NOT in $blArray.
Is there a better way to do this? The processing here takes an extremely long time. Both $blArray and $FSBuildIDs have about 20k elements and I suspect I'm looking at On^2 comparisons.
I thought about using a Dictionary with the elements of $FSBuildIDs as the keys and $paths as the values, but I can't figure out from the docs how to initialize and load the Dictionary (assuming this approach would speed things up). Obviously negative set intersection would be best but this isn't TSQL and I'm painfully aware that even V4 of PS doesn't support set operations.
Would using a dictionary in this problem speed up the comparisons? If so how do I create it from $FSBuildIDs and $paths? Any other techniques that might give me a performance boost vs. just iterating over these large(ish) lists?
Sample data for $blArray:
51012
51044
51049
51055
51058
51060
51073
51074
51077
51085
Sample data for $FSBuildIDs:
51001
51003
51005
51009
51013
51017
51018
51020
51021
51024
51026
Sample data for $paths:
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2335
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2336
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2337
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2338
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2339
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2340
\\server1\d$\software\anthill\var\artifacts\0000\3774\0000\3792\0005\2341
This is similar to the question posed previously, but different in some aspects. I'm essentially looking for guidance on constructing a dictionary from two existing arrays. I realized after posting that I really need a dictionary from $blarray as the keys and maybe $True as the value. The value is irrelevant. The important test is whether or not the current value in $FSBuildIDs is found in $blarray. That could be a dictionary lookup based on the ID as the key. That should speed up the processing, right?
I'm not clear on the comment that I'm destroying and recreating the array each time. Is that the $deadPaths array? Simply adding to it causes that? If so would I be better using a .Net ArrayList?
You could achieve a significant improvement by using the -contains operator instead of -like.
When the left-hand side of a -like operation is an array, PowerShell will iterate the array and perform a -like comparison against each and every entry.
-contains, on the other hand, returns as soon as a match is found.
Consider the following example:
$array1 = 1..2000
$array2 = 2..2001
$like = Measure-Command {
foreach($i in $array2){
$array1 -like $i
}
} |Select -Expand TotalMilliseconds
$contains = Measure-Command {
foreach($i in $array2){
$array1 -contains $i
}
} |Select -Expand TotalMilliseconds
Write-Host "Operation with -like took: $($like)ms"
Write-Host "Operation with -contains took: $($contains)ms"
Just like in your real-world example, we have 2 integer arrays with a large overlap. Let's see how it performs on my Windows 7 laptop (PowerShell 4.0):
I think the result speaks for itself :-)
That being said, you could, as you seem to anticipate, achieve an even greater improvement by populating a hashtable, using the values from the first array as keys:
$hashtable = $array1 |ForEach-Object -Begin {$t = #{}} -Process {
$t[$_] = $null
# the value doesn't matter, we're only interested in the key lookup
} -End { $t }
and then use the ContainsKey() method on the hashtable instead of -like:
foreach($i in $array2){
if($hashtable.ContainsKey($i)) { # do stuff }
}
You'll need to bump up the size of the array to see the actual difference (here using 20K items in the first array):
Final test script can be found here
I think this would be the start of what you are looking for. As discussed in comments we are going to do two comparisons. First to get the BuildID's we need to compare from from $FSBuildIDs and $blArray then we take the result of that to compare against the list of $paths. I am going to assume that it is just a string array of paths for now. Note there is room for error prevention and correction here. Still just testing for now.
$parsedIDs = Compare-Object $blArray $FSBuildIDs | Where{$_.SideIndicator -eq "=>"} | Select-Object -ExpandProperty InputObject
$paths = $paths | ForEach-Object{
$_ | Add-Member -MemberType NoteProperty -Name BuildID -Value (($_.Parent.Name + $_.Name) -as [int32]) -PassThru
}
$paths | Where-Object{$_.BuildID -in $parsedIDs}
First we compare the two ID arrays and keep the unique elements of $FSBuildIDs.
Next we go through the $paths. For each one we add a property that contains buildid. Where the buildid is the last two path elements concatenated and converted to an integer.
Once we have that a simple Where-Object give us the paths that have an id present from the first comparison.
To answer the question about building a hashtable:
$keyEnumerator = $FSBuildIDs.GetEnumerator()
$valEnumerator = $paths.GetEnumerator()
$idPathHash = #{}
foreach ($key in $keyEnumerator ) {
$null = $valEnumerator.movenext()
$idPathHash[$key] = $valEnumerator.current
}
Running this code on my system with a 20000 element array of fake data took 138ms.
To build the list of build ids not in the $idPathHash:
$buildIDsNotIn =
foreach ($buildId in $blArray) {
if (!$idPathHash.ContainsKey($buildId )) {
$buildId
}
}
This took 50ms on my system, with 20000 items in $blArray, again with fake data.