I have inherited one ETL process and one of the new aims is try to get all the data from a text file and insert it into a SQL Server Instance. The problem is that the new format of the text file changed and my stored procedure does not work anymore so I trying to change the format of the txt file to fit to my SP.
First of all I was able to delete some data with this powershell script
$newdata=foreach($line in get-content "C:\users\user12\text_file1.txt")
{
$line -replace 'text to delete1',' '
}
else
{
$line
}
$newdata+=$newdata1
$newdata1=foreach($line in $newdata)
{
$line -replace 'text to delete2',' '
}
else
{
$line
}
$newdata1 | set-content "C:\users\user12\text_file1.txt" -force
$newdata1 | set-contect "C:\users\user12\text_file1.txt" -WorksheetName "datachanged" -AutoSize
The challenge begins when I exported to excel
Here a pseudo-example about how looks the excel and what I expect
How actual looks
Time
Action
User
Size
Data
Folder
SO
Direction
Age
10:00 PM
Delete
Admin
3KB
PDF
/pdf/system_data.pdf
WO
10.0.0.0
14
10:30 PM
Created
Admin
5KB
XLSX
UB
10.1.0.0
12
10:45 PM
Sent
User
8KB
CSV
/pdf/system_data1.csv
WO
10.0.1.0
07
10:30 PM
Created
Admin
12KB
XLSX
UB
10.1.0.0
12
What I expect
Time
Action
User
Size
Data
Folder
SO
Direction
Age
10:00 PM
Delete
Admin
3KB
PDF
/pdf/system_data.pdf
WO
10.0.0.0
14
10:30 PM
Created
Admin
5KB
XLSX
UB
10.1.0.0
12
10:45 PM
Sent
User
8KB
CSV
/pdf/system_data1.csv
WO
10.0.1.0
07
10:30 PM
Created
Admin
12KB
XLSX
UB
10.1.0.0
12
I need to create an empty field or move to the right side all the values from "folder" to be able to read the file correctly with an OPENROWSET function and insert into a table.
Assuming that your data is tab-delimited, and that you with to insert an extra tab in lines containing exactly 8 fields (7 tabs), the following may work.
$Header = "Time", "Action", "User", "Size", "Data", "Folder", "SO", "Direction", "Age"
$Content = `
"10:00 PM`tDelete`tAdmin`t3KB`tPDF`t/pdf/system_data.pdf`tWO`t10.0.0.0`t14",
"10:30 PM`tCreated`tAdmin`t5KB`tXLSX`tUB`t10.1.0.0`t12",
"10:45 PM`tSent`tUser`t8KB`tCSV`t/pdf/system_data1.csv`tWO`t10.0.1.0`t07",
"10:30 PM`tCreated`tAdmin`t12KB`tXLSX`tUB`t10.1.0.0`t12"
$Content | ConvertFrom-Csv -Delimiter "`t" -Header $Header | Format-Table
$match = '^([^\t]*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([^\t]*)$'
$replace = '$1\t$2\t$3\t$4\t$5\t\t$6\t$7\t$8' -replace '\\t',"`t" # Is there a better way to code this?
$Content = $Content | foreach { $_ -replace $match,$replace }
$Content | ConvertFrom-Csv -Delimiter "`t" -Header $Header | Format-Table
The match string matches an alternating sequence of non-tab characters and tabs, capturing the text between the tabs. The replace string just rebuilds the line with the extra tab. (I couldn't figure out how to include tabs and replacement strings in the same powershell string literal, so I had to actually do a -replace on the replace string.)
Caution: The above may not properly handle quoted values that themselves contain delimiters.
I'm stuck trying to find a better way of modifying large (1000+ rows) CSV file than multiple foreach loops.
I have a CSV file with:
Name,login,ID,MGRID,MGRNAME
Bob Smith,bsmith,101,201,Drake Suzy
Suzy Drake,sdrake,201,300,Long Jane
John Bass,jbass,102,201,Drake Suzy
Jane Long,jlong,300,300,Long Jane
I'm trying to find the best way to import the csv and then set the MGRNAME for each employee to match the login of the corresponding MGR. I'd like to see:
Name,login,ID,MGRID,MGRNAME
Bob Smith,bsmith,101,201,sdrake
Suzy Drake,sdrake,201,300,jlong
John Bass,jbass,102,201,sdrake
Jane Long,jlong,300,300,jlong
I've tried importing the csv and then replacing the MGRNAME using:
$Sup = Import-Csv $Csvfile
Foreach ($MGR in $SUP){
$SUP1 = $MGR.MGRId
$SupID = ($sup | Where-Object {$_.login -eq $Sup1}).Login
Foreach ($ID in $Sup) {
($Sup | Where-Object {$_.MGRID -eq $SupID}).MGRNAME = $supId
}
}
I've also tried using something like:
$Users = Import-Csv $Csvfile
Foreach ($MGR in $users){
$supID=$MGR.MGRID
$RowIndex=[array]::Indexof($MGR.MGR.NAME,"$supID")
}
Any helpful suggestions welcome.
Thank you
here's one way to get the job done. [grin] what it does ...
fakes reading in a CSV
when ready to do this with real data, replace the entire #region/#endregion with a call to Import-Csv.
iterates thru the imported list
if the current user ID is the same as the MgrId, set the MgrName to the current login
otherwise, look up the Mgr with a .Where({}) method call with the mode set to First & set the MgrName with the resulting login
shows the result
the .Where({}) lookup could be slow if the import is large. if so, you likely otta replace that with a hashtable for lookups. however, premature optimization is bad coding, so i did not do that. [grin]
also, the export to CSV seems well documented, so i did not include that.
the code ...
#region >>> fake reading in a CSV file
# in real life, use Import-CSV
$UserList = #'
Name,login,ID,MGRID,MGRNAME
Bob Smith,bsmith,101,201,Drake Suzy
Suzy Drake,sdrake,201,300,Long Jane
John Bass,jbass,102,201,Drake Suzy
Jane Long,jlong,300,300,Long Jane
'# | ConvertFrom-Csv
#endregion >>> fake reading in a CSV file
foreach ($UL_Item in $UserList)
{
if ($UL_Item.Id -eq $UL_Item.MgrId)
{
$UL_Item.MgrName = $UL_Item.Login
}
else
{
$UL_Item.MgrName = $UserList.Where({$_.Id -eq $UL_Item.MgrId}, 'First', 1).Login
}
}
$UserList |
# remember - DO NOT use the "Format-*" cmdlets for anything other than _final output as plain text_ [*grin*]
Format-Table
the output ...
Name login ID MGRID MGRNAME
---- ----- -- ----- -------
Bob Smith bsmith 101 201 sdrake
Suzy Drake sdrake 201 300 jlong
John Bass jbass 102 201 sdrake
Jane Long jlong 300 300 jlong
Joining objects is a quite common activity in PowerShell, therefore I have created a 'Join-Object' cmdlet sometime ago which I still maintaining by adding features along with using a hashtable (for performance reasons as #Lee_Dailey mentiones) and adding self-join capabilities:
InnerJoin $UserList -on MgrId -eq id -discern '',MGR |
Select-Object Name, Login, ID, MGRID, MGRLogin | Format-Table
Name login ID MGRID MGRlogin
---- ----- -- ----- --------
Bob Smith bsmith 101 201 sdrake
Suzy Drake sdrake 201 300 jlong
John Bass jbass 102 201 sdrake
Jane Long jlong 300 300 jlong
Note that the column header MGRLogin doesn't exactly match the example in your question (which is actually inconsistent with the employee Name and - Login headers), but if you really want the same header you might do:
InnerJoin $UserList -on MgrId -eq id -discern '',MGR |
Select-Object Name, Login, ID, MGRID, #{n='MGRName'; e={$_.MGRLogin}} | Format-Table
For more details, see the embedded help (Help Join-Object) and In Powershell, what's the best way to join two tables into one?
everyone. I am newbie to Powershell. So your help is much appreciated.
My question is the following. I have a powershell small script that provides me with server Start, Stop, Kill ... tasks.
Here's the command
PS P:\> Get-ScheduledTask -TaskPath "*$someString*" | Get-ScheduledTaskInfo | Select TaskName,LastRunTime,NextRunTime
TaskName LastRunTime NextRunTime
-------- ----------- -----------
Archive 10/14/2017 7:00:00 AM 10/21/2017 7:00:00 AM
Kill 10/15/2017 12:50:50 PM 10/22/2017 12:50:50 PM
Restart 10/20/2017 12:00:00 AM 10/23/2017 12:00:00 AM
Start 10/15/2017 1:00:00 PM 10/22/2017 1:00:00 PM
Stop 10/13/2017 5:30:30 PM 10/20/2017 5:30:30 PM
I would like to extract the value of LastRunTime that corresponds to Start and the value of NextRunTime that corresponds to stop, and have them converted to a String.
Thank you.
Not very useful in the requested OP format, but here you go
C:\> $TaskNames = "Start","Stop"
C:\> $data = Get-ScheduledTask -TaskName $TaskNames | Get-ScheduledTaskInfo
C:\> [string]"$(($data | sort)[0].LastRunTime) $(($data | sort)[1].NextRunTime)"
10/20/2017 08:15:15 10/21/2017 09:16:16
this will only work right if your task names are actually in alphabetical order. Change the indexes (numbers in []) if the sort comes out in a different order.
I am working with a software that will output a text file containing various information regarding the measurement runs. I want to edit this file so that a certain set of strings can be deleted and then save this new file. Below is sample of the text file.
featureListName All Characteristics
meMode cf
naviMode automatic
pdfExport false
presProtocol false
printer false
printPlots false
protNumbers 1
protocolDefinition nil
protocolHead true
psExport false
ptxExport false
runMode slowToFirst
selection nil
speed 300
changedate July 5, 2017 10:00:35 am
changeoper Master
changeswi 6.2.1602
changeTimestamp 3676701635
changeTimestampOld 3676701635
creationdate July 5, 2017 9:57:53 am
creationdme Prismo
creationswi 6.2.1602
date 2017-7-5
dateandtime July 5, 2017 5:05:53 pm
datedayofmonth 5
datemonthindex 7
datesecondclock 3676745153
datestring July 5, 2017
dateyear 2017
dmeid Prismo
dmesn 000000
drawingno
mccplanversion
nameOfFile c:\Users\Public\Documents\Zeiss\CALYPSO\workarea\inspections\test
operid Master
order
partnbinc 3
partnbpre 3
planid test
produceoper Master
runswv 6.2.1602
time 5:05:53 pm
u_field1
u_field2
u_field3
u_field4 Contura1
My goal in editing this file is to remove the string "Contura1" and then save the file in the same directory with the same name. Any insight as to how I can accomplish this is very much appreciated.
I want a script to list all files in folder/subfolder with a particular last access date.
I have a script to show a subfolder:
get-childitem -Recurse | Where {$_.psIsContainer -eq $true} | select {$_.Fullname}
Thank you for your help.
Get-ChildItem -Recurse | ? { $_.lastaccesstime -ge [datetime]"12/18/11"} | select fullname
this give you a files's list with lastaccesstime attribute greater or equal 18 december 2011 00.00 am.
Get-ChildItem -Recurse | ? { $_.lastaccesstime -ge [datetime]"12/01/11" -and $_.lastaccesstime -lt [datetime]"12/02/11"} | select fullname
this give all file with lastaccesstime in date december 01 2011 from 00.00am to 11.59pm
lastaccesstime is a complete reference to date and time, that's why you need to specifies a range for a single date match.