I'm using a Powershell script to import API data into SQL Server. At the beginning of the script, the table within the database is truncated--
$TruncateTable = #"
TRUNCATE TABLE [dbo].[Table1]
"#
Invoke-sqlcmd #params -Query $TruncateTable
Then, the data is imported into SQL Server using a foreach loop with a nested if-else statement and some regex to isolate needed data.
foreach ($r in $roles) {
$InsertRoleResults = #"
INSERT INTO [dbo].[Table1]([roleID],[roleLocation],[roleName])
VALUES ('$roleId','$rolelocation','$rolename')
"#
# If role value contains brackets: assign variable to data within brackets/ variable to data after brackets
if($r.name -match '\[')
{
[int]$roleId = $r.id
$rolelocation = $r.name -replace '(\[)(.+)(\])(.+)', '$2'
$rolename = $r.name -replace '\[.+\]\ ', ''
Invoke-sqlcmd #params -Query $InsertRoleResults
}
else
# if data does not contain brackets, assign Null to rolelocation, entire string to rolename
{
[int]$roleId = $r.id
$rolelocation = $null
$rolename = $r.name
Invoke-sqlcmd #params -Query $InsertRoleResults
}
}
In both blocks, I also take the role id number ($r.id) and convert to INT and then reassign to $roleID.
Then it inserts into SQL Server. I have this problem where I keep getting this happening:
If I Write-Host the output of the id values, it starts with 1, not zero. So my question is, where is this zero value coming from?
I have code where I'm creating an index file by parsing out pieces of a file name.
This file name:
25643245_AjaWar_Prav_2_FT_20200701.pdf
Will create this line in the index file:
256432245|ST|W-HIGH SCHOOL TRANSCRIPT|##TEST-LOCATION\25643245_AjaWar_Prav_2_FT_20200701.pdf
The problem is that the first parse of '256432245' isn't a primary key in our database, so I have to convert to a primary key, then store the primary key into the index file in place of '256432245'
I have part of the query that builds the index file working correctly, but not the part that queries and returns the converted ID. If I run only the part of the query that returns just one ID, that also works. I'm having a problem getting the query to work within the "foreach".
I currently get this a result:
|ST|W-HIGH SCHOOL TRANSCRIPT|##TEST-LOCATION\25643245_AjaWar_Prav_2_FT_20200701.pdf
When I want to get:
8992004|ST|W-HIGH SCHOOL TRANSCRIPT|##TEST-LOCATION\25643245_AjaWar_Prav_2_FT_20200701.pdf
Where '8992004' is the SPRIDEN_ID is the result of the sql query.
Thanks for any help you can provide.
foreach ($Filename in Get-ChildItem $ImagePath)
{
$Arr = $Filename -split '_'
$reworkedfilename = $Arr[0] + '_' + $Arr[1] + '_' + $Arr[2] + '_' + $Arr[3] + '_' + $Arr[4] + '_' + $Arr[5]
##$reworkedarray2 = $Arr[0] -replace ".pdf", "";
Write-host $Arr[0] ##this works because I can see the non-primary ID being returned
#Find Each SPRIDEN_ID translated from CAID
add-type -AssemblyName System.Data.OracleClient
$username = "U"
$password = "P"
$data_source = "DS"
$connection_string = "User Id=$username;Password=$password;Data Source=$data_source"
$statement = "
Select Distinct SPRIDEN_ID
from SARACMT, SPRIDEN
where
SPRIDEN_PIDM = SARACMT_PIDM
and SPRIDEN_CHANGE_IND is null
AND SARACMT_COMMENT_TEXT = '$Arr[0]'
"
##The "AND SARACMT_COMMENT_TEXT = '$Arr[0]'" doesn't work because nothing is being returned in the index file
try{
$con = New-Object System.Data.OracleClient.OracleConnection($connection_string)
$con.Open()
$cmd = $con.CreateCommand()
$cmd.CommandText = $statement
$result = $cmd.ExecuteReader()
# Do something with the results...
$ArrConverted = while ($result.Read()) {
$result.GetString(0)
}
} catch {
Write-Error (“Database Exception: {0}`n{1}” -f `
$con.ConnectionString, $_.Exception.ToString())
} finally{
if ($con.State -eq ‘Open’) { $con.close() }
}
$outputline = $ArrConverted + '|' + $Arr[4] + '|' + $DocType + '|##'+ $ImagePath + $reworkedfilename | out-file -filepath $IndexFilePath -Encoding "ascii" -append
#>
}
Your issue is to do with how you’re trying to inject variable values into your sql query string:
$statement = "
Select Distinct SPRIDEN_ID
from SARACMT, SPRIDEN
where
SPRIDEN_PIDM = SARACMT_PIDM
and SPRIDEN_CHANGE_IND is null
AND SARACMT_COMMENT_TEXT = '$Arr[0]'
"
If you add a write-host $statement after this line you’ll see it’s replacing just the $Arr part and not the $Arr[0] part, so your query contains something like:
Select Distinct SPRIDEN_ID
from SARACMT, SPRIDEN
where
SPRIDEN_PIDM = SARACMT_PIDM
and SPRIDEN_CHANGE_IND is null
AND SARACMT_COMMENT_TEXT = '25643245 AjaWar Prav 2 FT 20200701[0]'
instead of:
Select Distinct SPRIDEN_ID
from SARACMT, SPRIDEN
where
SPRIDEN_PIDM = SARACMT_PIDM
and SPRIDEN_CHANGE_IND is null
AND SARACMT_COMMENT_TEXT = '25643245’
To get it to replace the value of $Arr[0] you can surround it with $( ... ) to use “command substitution” (see https://learn.microsoft.com/en-us/powershell/scripting/learn/deep-dives/everything-about-string-substitutions?view=powershell-7#command-substitution) so your query becomes:
$statement = "
Select Distinct SPRIDEN_ID
from SARACMT, SPRIDEN
where
SPRIDEN_PIDM = SARACMT_PIDM
and SPRIDEN_CHANGE_IND is null
AND SARACMT_COMMENT_TEXT = '$($Arr[0])’
Having said that, you’d be much better off using a parameterised query rather than building a dynamic sql string (see Oracle Parameterized query in c#) because as your code stands it’s vulnerable to a sql injection attack if you come across a deliberately mischievous file name (e.g. ‘ or 1='1_AjaWar_Prav_2_FT_20200701.pdf).
See https://blogs.oracle.com/sql/what-is-sql-injection-and-how-to-stop-it for more about sql injection attacks.
Just by sheer luck I figured it out.
I created a variable for $Arr[0]
$Arr0 = $Arr[0]
Then put the new variable in the where clause of where clause of the sql statement:
AND SARACMT_COMMENT_TEXT = '$Arr0'
This ran the query on each item parsed out during the foreach.
Can anyone help with this problem? I am referring to a example from the internet for executing T-SQL statements in parallel.
https://www.mssqltips.com/sqlservertip/3539/complete-common-sql-server-database-administration-tasks-in-parallel-with-powershell-v3-workflow/
I want to be able to execute the same T-SQL on the same instance at once for proof of concept work on locking. In order to do this, I have tweaked the script so that I can execute any number of iterations by changing the
while ($counter -le 5)
Here is the full script. Basically the primary statement can be whatever T-SQL you want and this will populate $sqlcmds to have that statement passed through as many iterations as you wish.
Import-Module sqlps -DisableNameChecking;
Set-Location c:
# create a workflow to run multiple sql in parallel
workflow Run-PSQL #PSQL means Parallel SQL {
Param(
[Parameter(Mandatory=$true)]
[string]$ServerInstance,
[Parameter(Mandatory=$false)]
[string]$Database,
[Parameter(Mandatory=$true)]
[string[]]$Query # a string array to hold t-sqls
)
foreach -parallel ($q in $query) {
Invoke-Sqlcmd -ServerInstance $ServerInstance -Database $Database -Query $q -QueryTimeout 60000;
}
} # Run-PSQL
# prepare a bunch of sql commands in a string arrary
#####new bit to make it dynamic sql multiple times
[string[]]$sqlcmds
$sqlcmds = ""
$counter = 0
do {
"Starting Loop $Counter"
$PrimaryStatement = '"SELECT TOP 1 * FROM sys.objects"'
if ($counter -eq 5) {
$sqlcmds = $sqlcmds + "$PrimaryStatement"
Write-Host "this is what sqlcmds is $sqlcmds loop 5"
} else {
$sqlcmds = $sqlcmds + "$PrimaryStatement,``"
Write-Host "this is what sqlcmds is now $sqlcmds"
}
$counter++
} while ($counter -le 5)
# now we can run the workflow and measure its execution duration
$dt_start = Get-Date; #start time
Run-PSQL -Server &&&&&&& -Database master -Query $sqlcmds;
$dt_end = Get-Date; #end time
$dt_end - $dt_start; # find execution duration
When this is executed, I get this message:
Run-PSQL : Cannot bind argument to parameter 'Query' because it is an empty string.
There are a few minor corrections I had to make and below is the final code that seems to work as expected
Moved the first { since it got commented!
Removed $sqlcmds = ""
Changed how SQL is aggregated in the array $sqlcmds
Removed the if/else inside as it did not seem to serve a
purpose
Changed the text printed
Import-Module sqlps -DisableNameChecking;
Set-Location c:
# create a workflow to run multiple sql in parallel
workflow Run-PSQL #PSQL means Parallel SQL
{
Param(
[Parameter(Mandatory=$true)]
[string]$ServerInstance,
[Parameter(Mandatory=$false)]
[string]$Database,
[Parameter(Mandatory=$true)]
[string[]]$Query #a string array to hold t-sqls
)
foreach -parallel ($q in $query) {
Invoke-Sqlcmd -ServerInstance $ServerInstance -Database $Database -Query $q -QueryTimeout 60000;
}
} # Run-PSQL
# prepare a bunch of sql commands in a string arrary
##### new bit to make it dynamic sql multiple times
[string[]]$sqlcmds = #()
$counter = 0
do {
"Starting Loop $Counter"
$PrimaryStatement = 'SELECT TOP 1 * FROM sys.objects'
$sqlcmds += "$PrimaryStatement"
Write-Host ("this is what sqlcmds has [$($sqlcmds.Count)] statements at loop counter [$Counter]")
$counter++
} while ($counter -le 5)
# now we can run the workflow and measure its execution duration
$dt_start = Get-Date; #start time
Run-PSQL -Server 'myserver\myinstance' -Database master -Query $sqlcmds;
$dt_end = Get-Date; #end time
$dt_end - $dt_start; # find execution duration
I have PowerShell command that's executing the SQL Query and it is returning this query as XML.
$query = #(Invoke-Sqlcmd -Query "select * from LS.dbo.live_Holdbacks FOR XML PATH('test');" -ServerInstance 'localhost')
$query | FORMAT-TABLE -Wrap
The problem is I have to print this result on the screen and then something strange happens. There are white marks in random places. You can see it marked on red. Word "Number" is literally split in the middle for no reason. What may cause it and how should I deal with it?
Apparently I dealt with it. Script below returns whole XML without those white marks.
$query = #(Invoke-Sqlcmd -Query "select top 100 * from LS.dbo.live_Holdbacks FOR XML PATH('test');" -MaxCharLength 100000 -ServerInstance 'localhost' )
$count = $query.Count
$out_string = ''
for($i=0; $i -lt $count; $i++){
$r = $query[$i].ItemArray[0].Trim()
$out_string += $r
}
$out_string
I have a SQL 2008 Ent server with the databases "DBOne", "DBTwo", "DBThree" on the server DEVSQLSRV.
Here is my Powershell script:
$DBNameList = (Invoke-SQLCmd -query "select Name from sysdatabases" -Server DEVSQLSRV)
This produces my desired list of database names as:
Name
-----
DBOne
DBTwo
DBThree
I has been my assumption that anything that is returned as a list is an Array in Powershell. However, when I then try this in Powershell:
$DBNameList -contains 'DBTwo'
It comes back has "False" instead of "True" which is leading me to believe that my list is not an actual array.
Any idea what I'm missing here?
Thanks so much!
Emo
I'd do this:
$DBNameList = #(Invoke-SQLCmd -query "select Name from sysdatabases" -Server DEVSQLSRV) | select-object -expand Name
That will give you an array of names.The option -contains should work fine.
What's missing from the original post is some type of conversion, from an object to an array.
Powershell outputs the result of $DBNameList because it kinda interprets the object. But if you need to manipulate this object and identify a specific item from it, this is the method I use:
$Itm = "DBTwo"
$DBNameList = #(Invoke-SQLCmd -query "select Name from sysdatabases" -Server DEVSQLSRV)
$NameList = #($DBNameList | select-object -ExpandProperty Name)
$Name = ($NameList.Split()).Contains($Itm)
Write-Output $Name
True
I have been looking for this myself for a while and finally worked it out, so I hope it helps someone else!
The Name header suggests it's a single object with a Name property which is an array.
I reckon initialize an empty PS array :
$DBNameList = (Invoke-SQLCmd -query "select Name from sysdatabases" -Server DEVSQLSRV)
[Array]$DbNames = #()
$DBNameList.Name | ForEach-Object {$DbNames += $_}
$DbNames -contains "DBTwo"
Any luck?
Your code...
$DBNameList = (Invoke-SQLCmd -query "select Name from sysdatabases" -Server DEVSQLSRV)
...gives you Datarows back..
You can check this with
$DBNameList | Get-Member
You can also see that there is a property wit the name "Name".
If you want to check if one of the datarows in your $DBNameList contains the Name of 'DBTwo' you would need to write it the following:
$DBNameList.Name -contains 'DBTwo'
In case anybody else ended up here because they were terrified they were going to have to hand-type the name of every property in the DataRow object response in order to get several columns into an array, have no fear, there is a handy property called "ItemArray" that provides what you need.
(Invoke-SQLCmd -query "select Name from sysdatabases").ItemArray -contains 'DBTwo'
True
There are plenty of great answers here that solve this particular OP's quandry, but when the column list get's long, this makes things a lot simpler.
(Invoke-SQLCmd -query "select DBID,Name,Version from sysdatabases")[0].ItemArray -join ','
1,master,852
Reading all these complicated answers and I realized there is a simple one:
$DBNameList.Name -contains 'DBTwo'
This should return true.
Your initial pull of the data (the database names) returns an array of objects. Each object has many properties. But in your logic test, you are trying to compare the entire object to a single string. You need to compare a single property of the object (.Name) to the string.
Still very new to Powershell (less than two weeks): I suggest that you try this, if your query contains multiple columns and rows... Multi-Dimensional Arrays. This was my first attempt at this, and after checking the web, given that I could not find a simple straight forward solution, I ended up writing my own solution. Here's the full set of sample code for you to experiment with and use.
Full set of sample code below....
#############################################################################################
# RDSago
# RDSago#gmail.com
# 09/20/2014
#############################################################################################
#
# Capturing database size information from a collection of servers
# and returning that back to an array that can be used to populate
# a SQL table that can be used for monitoring database growth remotely.
# RDSago, RDSago#gmail.com
#
# Note, SQL data retrieved in this manner, does not have to be parsed
# before it is consumed and used elsewhere, just like any array you have defined.
# The data only needs to be addressed by its ".identityname" captured in the
# array $queryResults (shown below).
#
############################################################################################
#############################################################################################
# T-SQL for creating table to hold data returned
#
# CREATE TABLE [dba].[tbl_dbfilesize](
# [ServerNameInstance] [varchar](20) NULL,
# [DatabaseName] [varchar](30) NULL,
# [DataFileSizeMB] [numeric](20, 0) NULL,
# [LogFileSizeMB] [numeric](20, 0) NULL,
# [TotalDatabaseSizeMB] [numeric](20, 0) NULL,
# [CollectionDate] [date] NULL
# ) ON [PRIMARY]
#############################################################################################
Try
{
#define your connection points
# first create an array that will hold the server/instance name of the servers you wish to audit
# the first sever assumes a named instance, the second a default instance name.
$SourceServerName = #("ServerName01/InstanceName", "ServerName02", "ServerName03") # Server you will retrieve data from
#next define the server connection for where you will write your data back to
$TargetServerInstance = "TaretServerName"
# define your sql query that will be used to pull data from SQL on the Source Server
$qryDatabaseInfo = "
SELECT ##ServerName as ServerNameInstance,
DB.name as DatabaseName,
SUM(CASE WHEN type = 0 THEN MF.size * 8 / 1024 ELSE 0 END) AS DataFileSizeMB,
SUM(CASE WHEN type = 1 THEN MF.size * 8 / 1024 ELSE 0 END) AS LogFileSizeMB,
SUM(CASE WHEN type = 1 THEN MF.size * 8 / 1024 ELSE 0 END) + SUM(CASE WHEN type = 0 THEN MF.size * 8 / 1024 ELSE 0 END) AS TotalDatabaseSizeMB
FROM sys.master_files MF
JOIN sys.databases DB ON DB.database_id = MF.database_id
GROUP BY DB.name
ORDER BY DB.NAME ASC
"
#Loop through all the servers you wish to audit
ForEach ($SourceServerName in $SourceServerNames)
{
#execute query to pull data from server into an array
$queryResults = #(Invoke-SQLCmd -query $qryDatabaseInfo -Server $SourceServerInstance)
# Next, construct your insert statement from data in your $queryresults array.
Foreach ($queryResult in $queryResults)
{
$query = "
Insert Into [DBS_AUDIT_SERVERS].[dba].[tbl_dbfilesize]
([ServerNameInstance],
[DatabaseName],
[DataFileSizeMB],
[LogFileSizeMB],
[TotalDatabaseSizeMB],
[CollectionDate])
Values
(" +
"'" + $SourceServerInstance + "'," +
"'" + $queryResult.DatabaseName + "'," +
"'" + $queryResult.DataFileSizeMB + "'," +
"'" + $queryResult.LogFileSizeMB + "'," +
"'" + $queryResult.TotalDatabaseSizeMB + "'," +
"'" + $Date + "'" +
")"
""
#execute insert statement for sql
Invoke-Sqlcmd -Query $query -ServerInstance $TargetServerInstance
}
}
}
Catch [Exception]
{
$ErrorMessage = $_.Exception.Message
Write-Host $ErrorMessage
}
Finally
{
Write-Host "Completed Successfully"
}
Return 0;