Bulk regex removals against large array very slow in PowerShell - arrays

Question

I am trying to find the quickest / most efficient way to run many regex removals against an array.
My $hosts array contains tens of thousands of individual items, in domain format. E.g:
test.domain.xyz
domain.xyz
something.com
anotherdomain.net
My $local_regex array contains ~1000 indivdual regexes, in multi-line format. E.g:
^ad. (ad.*)
domain.xyz$ (*domain.xyz)
I am currently trying to exclude any regex matches in the following way, but it is EXTREMELY slow with a large array and many regexes to match:
Function Regex-Remove
{
Param
(
[Parameter(Mandatory=$true)]
$local_regex,
[Parameter(Mandatory=$true)]
$hosts
)
# Loop through each regex and select only non-matching items
foreach($regex in $local_regex)
{
# Multi line, case insensitive
$regex = "(?im)$regex"
# Select hosts that do not match regex
$hosts = $hosts -notmatch $regex
}
return $hosts
}
Is there a better way to do this?

Answer 1

Reassigning a large array is going to be costly. Changing an array's size requires allocating a new array and copying the contents into it. If you have, say, 10 000 hostnames and 1 000 regexes, you have 10 000 000 copy operations. That's going to have some measurable effect. There is a cmdlet Measure-Command which is used to time execution times.
As an alternative approach, try to use indexed an array and overwrite undesired values with $null values. Like so,
foreach($regex in $local_regex) {
$regex = "(?im)$regex"
for($i=0;$i -lt $hosts.length; ++$i) {
if( $hosts[$i] -match $regex) {
$hosts[$i] = $null
}
}
}

Answer 2

You can use System.Collections.ArrayList objects instead of arrays, this will make the process much faster, and you have methods to add / remove items without rebuilding the whole array
$var = New-Object System.Collections.ArrayList
$var.Add()
$var.AddRange()
$var.Remove()
$var.RemoveRange()

Answer 3

As suggested by #Roberto, I switched the $hosts array to a New-Object System.Collections.ArrayList
The ability to remove from the ArrayList on the fly is exactly what I needed, and the while loop makes sure to remove duplicate values.
Function Regex-Remove
{
Param
(
[Parameter(Mandatory=$true)]
$local_regex,
[Parameter(Mandatory=$true)]
$hosts
)
# Loop through each regex and select only non-matching items
foreach($regex in $local_regex)
{
# Multi line, case insensitive
$regex = "(?i)$regex"
# Select hosts that do not match regex
$hosts -match $regex | % {
while($hosts.Contains($_))
{
$hosts.Remove($_)
}
}
}
return $hosts
}