Powershell Invoke-Sqlcmd - return multiple datasets - sql-server

I'm looking for suggestions on either returning multiple datasets, or keeping the session open, with Invoke-SqlCmd?
I have a series of SQL queries that use temporary tables to generate a few related views of the data that I am sending on (via Excel) to a manager. As we work on what is required from the datasets, I am getting a little tired of cutting and pasting samples into Excel.
I thought to use Powershell to simply send the results to HTML files as output for the manager, however I ran into a couple of problems
If I put the final extracts into one SQL file, Powershell appends all of the data into a single result set (sort of a union of the tables)
If I attempt to build the temporary tables and then extract each query individually, each Invoke-Sqlcmd is a seperate session, meaning my Temporary tables get dropped.
I'm looking for suggestions on either returning multiple datasets, or keeping the session open?
Invoke-Sqlcmd -InputFile .\GenerateTimecard.sql -Variable $params | Out-Null;
#{
'Summary' = 'select * from #WeeklyTimeSummary;'
'ByDay' = 'select * from #WeeklyTimeDaily order by postdate desc;'
'ByTask' = 'select * from #WeeklyTimeEvents order by HoursSpent desc;'
'Detail' = 'select * from #WeeklyTimeDetail order by postdate desc;'
}.GetEnumerator() | ForEach-Object {
Write-Output $_.Name;
$fname = $_.Name + '.html';
Invoke-Sqlcmd -Query $_.Value | ConvertTo-Html | Out-File -Encoding ascii $fname;
};

The Description section from Get-Help Invoke-Sqlcmd says it supports GO commands so you could try running everything at once. Personally I'd use the -InputFile parameter and pipe the results to Out-File.

You can specify the ApplicationName parameter for Invoke-SqlCmd, which results in a different SQL connection.
Omitting ApplicationName will result in the temp tables getting removed the second time you call Invoke-SqlCmd.
Something like:
Invoke-SqlCmd -ApplicationName CreateTable -Query 'CREATE TABLE ##FooTable (FooKey INT)
Invoke-SqlCmd -ApplicationName SelectTable -Query 'SELECT * FROM ##FooTable'

Related

Powershell - Export SQL to XML to file - no clean output

I am trying to export a sql query to an xml file with Powershell using the below script:
$SQLResult = Invoke-Sqlcmd -MaxCharLength ([int]::MaxValue) -AbortOnError -EncryptConnection -ConnectionTimeout $TIMEOUT -Database $Database -ServerInstance $SQL_SERVER_FULLNAME_SOURCE -Username $SQL_ACCOUNT_NAME_SOURCE -Password $SQL_ACCOUNT_PASSWORD_SOURCE -Query $QUERY
$PropertyName = ($SQLResult | Get-Member -MemberType Property | Where {$_.Name -like "XML*"}).Name
$SQLResult.$PropertyName | Out-File -FilePath $OutputFile -Force
It should generate a file like below:
<vendors>
<vendor>
<administration>YIT</administration>
<FISCALCODE>804658055B01</FISCALCODE>
<accountNumber>10001</accountNumber>
<Offset_LedgerAccount></Offset_LedgerAccount>
<name>Comp Europe B.V.</name>
<shortName>Comp Europe B.V.</shortName>
<TAXEXEMPTNUMBER>NL801238055B01</TAXEXEMPTNUMBER>
<DefaultDescription></DefaultDescription>
<invoiceType>Non-PO CEE-FN-ACQ</invoiceType>
<IBAN>NL57MHCB0212303590</IBAN>
<CUSTOMERACCOUNT></CUSTOMERACCOUNT>
<TAXITEMGROUP></TAXITEMGROUP>
<IncludeOrderReference>Unknown</IncludeOrderReference>
<lineText></lineText>
<lines>Unknown</lines>
</vendor>
<vendor>
<administration>YIT</administration>
<FISCALCODE>03840123961</FISCALCODE>
<accountNumber>20001</accountNumber>
<Offset_LedgerAccount></Offset_LedgerAccount>
<name>4ABCD - For Marketing s.r.l.</name>
<shortName>4ABCD-FOR MARKETING</shortName>
<TAXEXEMPTNUMBER>03840123961</TAXEXEMPTNUMBER>
<DefaultDescription></DefaultDescription>
<invoiceType>Purchase invoice</invoiceType>
<IBAN>IT93M0306912330615256048252</IBAN>
<CUSTOMERACCOUNT></CUSTOMERACCOUNT>
<TAXITEMGROUP></TAXITEMGROUP>
<IncludeOrderReference>Unknown</IncludeOrderReference>
<lineText></lineText>
<lines>Unknown</lines>
</vendor>
<vendors>
In real it can contain a few thousand records.
The problem with the various other options that I used is that the output file is either truncated, or it contains double quotes. With the above script suddenly extra spaces are added that are repeated every certain block of output. If I run the query in MSMS it generates a perfectly clean result that I can save as .xml file.
<vendor>
<administration>YIT</administration>
<FISCALCODE>0381230961</FISCALCODE>
<accountNumber>20001</accountNumber>
<Offset_LedgerAccount/>
<name>4ABCD - For Marketing s.r.l.</name>
<shortName>4ABCD-FOR MARKETING</shortName>
<TAXEXEMPTNUMBER>03840123961</TAXEXEMPTNUMBER>
<DefaultDescription/>
<invoiceType>Purchase invoice</invoiceType>
<IBAN>IT93M03069121238252</IBAN>
<CUST OMERACCOUNT>< CUSTOMERACCOUNT><TAXITEMGROUP>< TAXITEMGROUP><IncludeOrderReference>Unknown< IncludeOrderReference><lineText>< lineText>
As you can see above it starts okay but then suddenly it starts adding a single space to certain nodes, not to all. Then it continues okay for some records and then a gain a line with a single space in some of the nodes.
I tried various solutions but none of them seem to work for me. Can somebody explain what I am doing wrong? The actual query I use is below.
SELECT
v.[DATAAREAID] AS [administration]
,v.[FISCALCODE]
,v.[VENDORACCOUNTNUMBER] AS [accountNumber]
,v.[DEFAULTOFFSETLEDGERACCOUNTDISPLAYVALUE] AS [Offset_LedgerAccount]
,v.[VENDORORGANIZATIONNAME] AS [name]
,v.[VENDORSEARCHNAME] AS [shortName]
,CASE WHEN LEFT(v.[TAXEXEMPTNUMBER],2)='IT' THEN RIGHT(v.[TAXEXEMPTNUMBER],LEN(v.[TAXEXEMPTNUMBER])-2) ELSE v.[TAXEXEMPTNUMBER] END AS [TAXEXEMPTNUMBER]
,ISNULL(v.[NOTES],'') AS [DefaultDescription]
,ISNULL(ipvp.[INVOICETYPENAME],'') AS [invoiceType]
,ISNULL(vba.[IBAN],'') AS [IBAN]
,ISNULL(c.[CUSTOMERACCOUNT],'') AS [CUSTOMERACCOUNT]
,ISNULL(itc.[TAXITEMGROUP],'') AS [TAXITEMGROUP]
,'Unknown' AS [IncludeOrderReference]
,v.[DESTINATIONCODE] AS [lineText]
,'Unknown' AS [lines]
FROM [dbo].[RetailVendVendorV3Staging] v
LEFT OUTER JOIN [dbo].[AXTip_ParametersVendTableStaging] ipvp ON v.[VENDORACCOUNTNUMBER]=ipvp.[VENDORACCOUNT] AND v.[DATAAREAID]=ipvp.[DATAAREAID]
LEFT OUTER JOIN [dbo].[VendVendorBankAccountStaging] vba ON vba.[DATAAREAID]=v.[DATAAREAID] AND vba.[VENDORACCOUNTNUMBER]=v.[VENDORACCOUNTNUMBER] AND vba.[VENDORBANKACCOUNTID]=v.[BANKACCOUNTID]
LEFT OUTER JOIN [dbo].[CustCustomerV3Staging] c ON c.[VENDORACCOUNT]=v.[VENDORACCOUNTNUMBER] AND v.[DATAAREAID]=c.[DATAAREAID]
LEFT OUTER JOIN [dbo].[AXTip_ImportTaxcodeStaging] itc ON itc.[ACCOUNTRELATION]=v.[VENDORACCOUNTNUMBER] AND itc.[DATAAREAID]=v.[DATAAREAID] AND itc.[EXTERNALTAXCODE]='I'
WHERE v.[DATAAREAID] ='YIT' AND v.[FISCALCODE]!=''
FOR XML PATH('vendor'), root('vendors')
I read quite a lot of issues using the method that I used in the opening post. The method described in the link below seems a more "powershell" way to generate an xml file using sql-data. I copied this method and it works well.
stuartsplace.com: powershell-and-sql-server-exporting-data-xml

powershell query of sys.sql_logins

I'm trying automate some queries and am querying sys.sql_logins and get different results between a query in ssms and powershell.
My query is this:
$SQLquery =#"
Select 'sql_logins' [sql_logins], ##SERVERNAME [Hostname], DB_NAME() [Database], * from sys.sql_logins;
"#
When I run that from a query in ssms i the SID result as 0xA1E3(some long alphanumeric string). When i run the same thing via powershell - i get a complete different format for the SID. Instead of the long alphanumeric string I end up with something like this {123, 45, 70, 16...} and i have no idea why or how to make it match the result i get via a manual query in ssms.
why does it change? how do i get it to NOT change and remain like 0xA1E3(super long alphanumeric string)?
[edit - as requested]
Really not doing anything special to get or display the data. Invoke-sqlcmd and then spit it out once returned.
$result = invoke-sqlcmd -query $SQLquery -serverinstance $computername -database $dbname -Username $dbuser -Password $dbpass
$result | Format-Table -Property Hostname, Database, name, principal_id, sid,`
type, type_desc, is_disabled, create_date, modify_date,`
default_database_name, default_language_name, credential_id,`
is_policy_checked, is_expiration_checked, password_hash -Autosize
The problem is correctly identified in another answer to this question in that the varbinary value being returned from the query is being treated as a byte array by powershell. My recommendation as to a fix is different, though - change the formatting in the query. If I change your query to:
Select 'sql_logins' [sql_logins],
##SERVERNAME [Hostname],
DB_NAME() [Database],
convert(varchar(172), sid, 1) as sid
from sys.sql_logins;
And then run it through the rest of your code, it works for me. NB: I didn't do select * in my query - if you really need all of the columns, you should list them explicitly. Lastly, one observation - sys.sql_logins is a server-level DMV; querying it on a per-database basis is going to be duplicative for all databases on the same server.
Edit - changed the length of the varchar to accommodate the varbinary(85) SID as the fine manual says. Showing my work, (85 bytes * 2 characters/byte) + (2 characters for '0x') = 172.
The SID which you get on the result window of SSMS is a 16 byte (binary(16)) literal value based on a GUID, while the one which you get in PowerShell is as mentioned in the comments, a default view is a comma separated list of byte values surrounded with brackets. PowerShell is better at string manipulation than SQL Server hence it internally converts the binary(16) value to a byte value.
Here is what you can do -
$SQLquery = "Select 'sql_logins' [sql_logins], ##SERVERNAME [Hostname], DB_NAME() [Database], * from sys.sql_logins;"
$Result = Invoke-Sqlcmd -query $SQLquery -ServerInstance ServerName -Database master
$Result[0].sid
$Result[0].sid.Length would give you 1024, implying that the length is indeed a KB. Use the ConvertTo-SQLHashString function from Mike Fal's blog where he talks about copying SQL Logins via PowerShell -
function ConvertTo-SQLHashString{
param([parameter(Mandatory=$true)] $binhash)
$outstring = '0x'
$binhash | ForEach-Object {$outstring += ('{0:X}' -f $_).PadLeft(2, '0')}
return $outstring
}
ConvertTo-SQLHashString $Result[0].sid
That would give you a long hex value. You could use the PowerShell TrimEnd() function to get the results like those in SSMS window -
(ConvertTo-SQLHashString $Result[0].sid).TrimEnd('0')

Capturing logs for .sql statements though powershell

I'm trying to figure out a good way to run insert/select/update statements into a MSSQL database through PowerShell scripting and log it thoroughly. I have managed to use Invoke-Sqlcmd which does the inserts and select statements pretty fine. But my concern is to capture the logs some where as an output file. That is when doing inserts the log should capture the no. of rows affected/inserted with the data that was inserted.
Invoke-Sqlcmd used:
PS C:\Users\moshink\Desktop\Powershell SQL> Invoke-Sqlcmd -Server $server -Database $db -Username $user -Password $pass -InputFile ".\test.sql" | Out-File -FilePath ".\test_$datetime.rpt" >> $LogFile
test.sql file query below:
use TCS_MIS
select * from tblMasterAccountType where acctType in ('0121') and intCat in ('0020')
--insert into tblMasterAccountType values ('DEP','0121','0020','PARENTHOOD ASSISTANCE ACCOUNT')
the test.sql file is called in the Invoke-Sqlcmd.
This file can be dynamic with any query going in there.
The Out-File ".\test_$datetime.rpt" does capture the select statement outputs if the data exits matching the criteria, but it will be blank if no data.
Is there something that can be ran to capture instantly when running a Insert .sql file/script? i.e. which will say 20 rows inserted and listing out the data inserted.
Basically what I'm after is a thorough logging when running any .sql scripts through PowerShell. It should capture no of rows affected with the data inserted/updated/deleted and the user who performed it.

A positional parameter cannot be found that accepts argument "from" when doing db2 query to get all schemas

I am getting an error when trying to run a db2 select query to get all the schemas listed in a db2 database.
Here is the error:
Select-Object : A positional parameter cannot be found that accepts
argument 'from' .At line:5 char:1...
I don't see the issue here since this appears to be the correct format. My code is below:
$conn = New-Object System.Data.OleDb.OleDbConnection("Provider=IBMDADB2;Database=TESTDATABASE;HostName=DB2HOST.db2domain.net;Protocol=TCPIP;Port=50002;Uid=adminID;Pwd=aPassWord;")
$ds = New-Object System.Data.DataSet
$conn.Open();
set-item -path env:DB2CLP -value "**$$**"
Select schemaname from syscat.schemata;
Your Select is a bare-line command in PowerShell. Select is aliased to Select-Object and that's why you get the error. You need to issue your Select to the database.
See examples like:
How To Execute MS Access Query with OLEDB.12
https://technet.microsoft.com/en-us/library/hh855069.aspx
Note that this one incorrectly marks the PowerShell examples as C#.

Pass a powershell variable into a SQL value during out-datatable (invoke-sqlcmd2)

I want to insert a PowerShell variable value with a Select as I build a datatable from a SQL query.
Borrowed function invoke-sqlcmd2 from TechNet gallery and dot-sourced it in.
$NewSequenceID = invoke-sqlcmd2 -ServerInstance "MyServer" -Database "MyDB" -Query "INSERT INTO [Sequence] (TimeStarted) SELECT GETDATE(); SELECT max(SequenceID) as SequenceID FROM [Sequence]" | foreach { $_.SequenceID }
This generates a new sequence ID and stamps the time we started the batch. Results in a single number which will identify this run. Verified with 'write $NewSequenceID'.
I want to keep later results from queries together with this SequenceID for analysis.
Then I have this:
$PollTime = Get-Date -format "yyyy-MM-dd HH:mm:ss"
Then I want to do this: (Edit: This statement is not working - error message at the bottom)
$AuditUserOutput = invoke-sqlcmd2 -ServerInstance "MyServer2" -Database "MyDB2" -Query "SELECT $NewSequenceID, $PollTime, [USERID], [PID], [UDATE] FROM [MyTable]" -As 'Datatable'
And do some things with the table, then write it after with write-datatable.
If I select NULL for the first two values and grab the other three from the existing table, it works fine. I want to add the $NewSequenceID and $PollTime from the previous statements.
I've read a dozen pages about using ` (backtick), $, {}, and on and on, but I haven't gotten it right. Can someone help with the correct syntax for inserting these variable values into the selection?
PS Error is: Exception calling "Fill" with "1" argument(s): "Invalid pseudocolumn "$NewSequenceID"."
You're interpolating the variables correctly in PowerShell. If I'm understanding this correctly, the problem is with your SQL query. I'm going to make an inference here, but I think this is probably what you want:
$AuditUserOutput = invoke-sqlcmd2 -ServerInstance "MyServer2" -Database "MyDB2" -Query "SELECT [NewSequenceID], [PollTime], [USERID], [PID], [UDATE] FROM [MyTable] WHERE NewSequenceID = '$NewSequenceID' AND PollTime = '$PollTime'" -As 'Datatable'
If not, please clarify by responding to the questions above.
I was able to work around this by first creating a variable to store the query text, which allowed for the natural substitution I needed:
$AuditUserQuery = "SELECT '$NewSequenceID', '$PollTime', [USERID], [PID], [UDATE] FROM [AUDITUSER]"
Then calling that variable as the $query when building the datatable.
This avoided the parameterization problem experienced before.

Resources