Loading URL result in SQL Server table with PowerShell - sql-server

I have a PowerShell script that gives the latest SP and CU available of all the versions of SQL Server by directly querying over the web by using the below method and saving the output into an HTML file.
Invoke-WebRequest -UseBasicParsing -Uri $URI
I want the same result in a SQL Server table but I'm unable to do so. please help me understand how to achieve this.
Below is the complete code:
Note: You can directly execute this script.
-----------------------------------------------------------------------------------------------------
# Script to display all SQL Server Service Packs per version available on microsoft technet
$URI = "https://learn.microsoft.com/en-us/sql/database-engine/install-windows/latest-updates-for-microsoft-sql-server?view=sql-server-ver15"
$WebResponse = Invoke-WebRequest -UseBasicParsing -Uri $URI
# $WebResponse
# -----------------------------------------------------------------------------------------------------
# -------------------------------------------------
# Format the Output
# -------------------------------------------------
$content = $WebResponse.RawContent -replace "\s*`n", " "
$Tab=$content -match "<table(.*)</table>"
$Output=$matches[0]
while ($Output.Contains(" ")){
$Output = $Output -replace " "," "
}
$Output= $Output.replace('class="grid"','id="grid"').replace("<td> <strong>","<th>").replace("</strong> </td>","</th>")
$OutputFile=".\SQLServer-Latest-SP-CU.html"
$PageTableHeader | out-file $OutputFile -encoding default
out-file $OutputFile -encoding default -Append
$Output | out-file $OutputFile -encoding default -Append
out-file $OutputFile -encoding default -Append
Write-Host "Output written to $OutputFile"
start $OutputFile

Since your html is well-formed, you can use XML methods in T-SQL to parse and insert into a table. The example below take your existing $Output string with html and passes it as an XML parameter.
$sql = #"
CREATE TABLE dbo.SqlPatches(
ProductVersion varchar(30)
, LatestServicePack varchar(30)
, LatestCumulativeUpdate varchar(30)
, GeneralGuidance varchar(30)
);
INSERT INTO dbo.SqlPatches
SELECT
b.value('./td[1]','varchar(100)') AS ProductVersion
, b.value('./td[2]','varchar(100)') AS LatestServicePack
, b.value('./td[2]','varchar(100)') AS LatestCumulativeUpdate
, b.value('./td[2]','varchar(100)') AS GeneralGuidance
FROM #SqlPatches.nodes('/table/tbody/tr') AS a(b);
"#
$connection = New-Object System.Data.SqlClient.SqlConnection("Data Source=.;Initial Catalog=tempdb;Integrated Security=SSPI")
$command = New-Object System.Data.SqlClient.SqlCommand($sql, $connection)
$command.Parameters.Add("#SqlPatches", [System.Data.SqlDbType]::Xml).Value = $Output
$connection.Open()
[void]$command.ExecuteNonQuery()
$connection.Close()

Related

How to read SQL rows as PowerShell parameters

On Azure, I am running multiple .sql files from a container in 100s of Azure SQL Databases via Powershell runbook.
I want Powershell to read the server name and the database name to run the scripts from my SQL Server table that looks like this:
Servername
Databasename
Status
Server-01
DB-01
Process
Server-01
DB-02
Skip
Server-02
DB-03
Process
In my current version of the Powershell script, it can read the files in the container and run them in a given server and database:
# Get the blob container
$blobs = Get-AzStorageContainer -Name $containerName -Context $ctx | Get-AzStorageBlob
# Download the blob content to localhost and execute each one
foreach ($blob in $blobs)
{
$file = Get-AzStorageBlobContent -Container $containerName -Blob $blob.Name -Destination "." -Context $ctx
Write-Output ("Processing file :" + $file.Name)
$query = Get-Content -Path $file.Name
Invoke-Sqlcmd -ServerInstance "Server-01.database.windows.net" -Database "DB-01" -Query $query -AccessToken $access_token
Write-Output ("This file is executed :" + $file.Name)
}
I am looking for a method that will read the rows from the table and feed them into the -ServerInstance and -Database fields in the Invoke-Sqlcmd. Ideally it can filter out the Skip rows.
One method is to load the database list into a DataTable and iterate over the list for each query. Change the $databaseListConnectionString in the example code below per your authentication method and set the connection AccessToken if/as needed.
# get database list
$databaseListConnectionString = "Data Source=YourServer;Initial Catalog=YourDatabase"
$databaseListQuery = "SELECT ServerName, DatabaseName FROM dbo.DatabaseList WHERE Status = 'Process';"
$dataAdapter = New-Object System.Data.SqlClient.SqlDataAdapter($databaseListQuery, $databaseListConnectionString)
$dataAdapter.SelectCommand.Connection.AccessToken = $access_token
$databaseList = New-Object System.Data.DataTable
[void]$dataAdapter.Fill($databaseList)
# Get the blob container
$blobs = Get-AzStorageContainer -Name $containerName -Context $ctx | Get-AzStorageBlob
# Download the blob content to localhost and execute each one
foreach ($blob in $blobs) {
{
$file = Get-AzStorageBlobContent -Container $containerName -Blob $blob.Name -Destination "." -Context $ctx
Write-Output ("Processing file :" + $file.Name)
$query = Get-Content -Path $file.Name
foreach($database in $databaseList.Rows) {
Invoke-Sqlcmd -ServerInstance "$($database.ServerName)" -Database "$($database.DatabaseName)" -Query $query -AccessToken $access_token
Write-Output ("This file is executed :" + $file.Name)
}
}
}

Generate Scripts for Tables and all Extended Properties

Looking for a way to generate individual scripts for each table and include any relationships or extended properties. (After getting this working I will try to generate scripts for stored procedure, and function)
I am aware of the Sql-Server UI Generator (Database=>Tasks=>Generate Scripts) but that does one big file, not individuals. If there is a way to make this produce individual files (with out doing them 1 at a time) that would be best.
I have used Powershell package DBATools with some limited success. The following will make a file that contains create scripts for the table and the table's extended property but not the column extended properties.
$server = "sql01"
$database = "MyDatabase"
$table = "MyTable"
Get-DbaDbTable -SqlInstance $server -Database $database -Table $table | Export-DbaScript -FilePath ($database + "\" + $table +".sql")
Get-DbaDbTable -SqlInstance $server -Database $database -Table $table | Get-DbaExtendedProperty | ForEach-Object { Export-DbaScript -InputObject $_ -FilePath ($database + "\" + $table +".sql") -Append }
The answer was given by Larnu in the commnets.
The comment pointed out the option to save scripts individually and I have been over looking it for years.
While you found the option in SSMS, I wanted to say that this is also possible using the dbatools approach you tried. The "secret sauce" is specifying a ScriptingOptions object that controls the scripting behavior.
$so = New-DbaScriptingOption;
$so.ExtendedProperties = $true;
foreach ($table in Get-DbaDbTable -SqlInstance . -Database AdventureWorks2019){
$path = '{0}.{1}.sql' -f $table.Schema, $table.Name;
Export-DbaScript -InputObject $table -FilePath $path -ScriptingOptionsObject $so;
}

How may I export the output of SQL query in Excel fetched through PowerShell?

I am using this power-shell script to fetch the versions of all SQL Servers in a list.
How may I export the result columns (only query output not error messages) into excel and send to email after the script is run?
Can someone help me add the required script please?
Import-Module SQLPS -DisableNameChecking
$ServerInstences = Get-Content "D:\DBA\All_Server_monitoring\ServerList.txt"
$SQLQuery = #"
Select ##Servername 'Server Name' ,##version 'Version'
"#
$DBName = "master"
$ServerInstences |
ForEach-Object {
$ServerObject = New-Object -TypeName Microsoft.SqlServer.Management.Smo.Server -ArgumentList $_
Invoke-Sqlcmd -ServerInstance $_ -Database $DBName -Query $SQLQuery
}
The easiest way to export data to a csv file is by using Export-CSV which takes an input object (or object array) and a path and can fill out the csv file from that object/array. For you, it would look like this:
$results = Invoke-Sqlcmd -ServerInstance $_ -Database $DBName -Query $SQLQuery
New-Item -Path "MYPATH.csv"
Export-CSV -Path "MYPATH.csv" -InputObject $results -Append
CSV files are versatile and can be opened with the most lightweight text editors. They also can be easily emailed and opened with Excel.

Join SQL query Results and Get-ChildItem Results

Background: I have a directory with a number of files that are imported to SQL server.
Task: Creating a PowerShell script which will pick up files within this directory and use the filenames as in the SQL query.
Ultimate objective: To display SQL results besides the filenames but the resultset being displayed should also show files having no entries in SQL server. Something like RIGHT JOIN in SQL server queries.
Powershell Code
$files = Get-ChildItem -Recurse -Force $filePath -ErrorAction SilentlyContinue | Where-Object { ($_.PSIsContainer -eq $false) } | Select-Object Name
$Server = "Loadv1"
$DB = "LoadDB"
$dbResults = #()
ForEach ($file in $files)
{
$fileName = $file.name
write-host $fileName
if($fileName.Length -gt 1)
{
$Query = "
SELECT FileName,CurrentStatus
FROM LogStatus
WHERE FileName LIKE '$fileName%'
"
# Write-host $Query
}
$dbResults += Invoke-Sqlcmd -ServerInstance $Server -Database $DB -Query $Query
}
$dispResults = $dbResults,$file
$dispResults | Format-Table -autosize
Work done so far: I have been able to fetch the file names using Get-ChildItem and loop them to get the query results. However, the result I am currently getting does not show the files that don't have corresponding entry in SQL server table
Current Result
OperationalLanding20150622061502.dat
OperationalLandingAudit20150622061502.dat
OperativeThird_Party_System20150616090701.dat
FileName CurrentStatus
OperationalLandingAudit20150622061502.dat SSIS Package Complete
OperativeThird_Party_System20150616090701.dat SSIS Package Complete
Expected Result
OperationalLanding20150622061502.dat
OperationalLandingAudit20150622061502.dat
OperativeThird_Party_System20150616090701.dat
FileName CurrentStatus
OperationalLanding20150622061502.dat NULL
OperationalLandingAudit20150622061502.dat SSIS Package Complete
OperativeThird_Party_System20150616090701.dat SSIS Package Complete
Hoping I was able to explain my requirement above.
OK so if the SQL query does not have results then NULL is returned and, in essence, nothing is added to the $dbResults array. Instead lets append the results to a custom object. I don't know what PowerShell version you have so I needed to do something that I know should work. I also don't use the SQL cmdlets much so I had to guess for some of this.
$files = Get-ChildItem -Recurse -Force $filePath -ErrorAction SilentlyContinue |
Where-Object {$_.PSIsContainer -eq $false -and $_.Length -gt 1} |
Select-Object -ExpandProperty Name
$Server = "Loadv1"
$DB = "LoadDB"
$files | ForEach-Object{
write-host $_
$Query = "
SELECT FileName,CurrentStatus
FROM LogStatus
WHERE FileName LIKE '$_%'
"
$Results = Invoke-Sqlcmd -ServerInstance $Server -Database $DB -Query $Query
$props = #{Name = $_}
If($Results){
$props.CurrentStatus = $Results.CurrentStatus
} Else {
$props.CurrentStatus = "Null"
}
New-Object -TypeName PSCustomObject -Property $props
} | Format-Table -autosize
What this does is create a custom object that contains the results of the sql query (Which I did not change for reasons stated above). If there are no results returned we use the string "null" as a filler.
I cleaned up how you generated the $files variable by making is a simple string array with -Expand and moved the length condition there as well.
You should now have all the expected results. I say should since I am assuming what the return object looks like.
$Query = "
SELECT isNull(A.FileName, b.FileName) FileName,ISNULL(A.CurrentStatus,B.CurrentStatus) CurrentStatus
FROM LogStatus A
Right JOIN (SELECT '$filename' FileName,NULL CurrentStatus) B
ON a.Filename like '$filename%'
"
This should pad out the filenames for you. A little tough to prototype since it's in powershell but I might be able to come up with a sql fiddle to prove it.
EDIT
Answer edited, with sql fiddle:
http://sqlfiddle.com/#!3/12b43/9
Obviously, since you're in a cursor, we can only prove one query at a time.

XML data coming in newline using invoke-sql command in powershell

I am using a powershell script which will return an XML from a sqlquery of FOR XML EXPLICIT type.
Now, the query returned the XML in a column with header and dashed lines
XML 32776498797309
---------------------
<data>asdsafafaf<\data><value>dfsasfasfdfasdf....
dfdsfsdgregrge<\value><value>asdfasdfadfadfsda<\v
alue>
Here, some how we were able to remove header , dashed line and truncating of data.
but still the data comes in new line so if we open the XML it throws error because at some places the tag gets distributed as shown above.
Basically, new line issue is there.
We tried -width 4096
but since XML is huge it is not proper.
Please help, stuck big time.
Query used :
invoke-sqlcmd -inputFile $inputFilePath -serverinstance $dbHostName
-database $dbName -username $userName -password $password
| Format-Table -hide -Wrap -AutoSize
| Out-File -filePath $outputFilePath -width 4096
Try using set-content instead of out-file. Out-file will use the host/console formatting while set-content does not. You'll need to handle outputting XML to string
invoke-sqlcmd ....| select -expandproperty XML | set-content -path $outputFilePath
Edited. Added full working example:
#Contents of my test inputfile.sql:
SELECT CAST ('<EVENT_INSTANCE>
<EventType>CREATE_TABLE</EventType>
<PostTime>2011-04-26T15:05:21.333</PostTime>
<SPID>56</SPID>
<ServerName>Z001\SQL1</ServerName>
<LoginName>Contoso\u00</LoginName>
<UserName>dbo</UserName>
<DatabaseName>AdventureWorksDW2008R2</DatabaseName>
<SchemaName>dbo</SchemaName>
<ObjectName>AdventureWorksDWBuildVersion</ObjectName>
<ObjectType>TABLE</ObjectType>
<TSQLCommand>
<SetOptions ANSI_NULLS="ON" ANSI_NULL_DEFAULT="ON" ANSI_PADDING="ON" QUOTED_IDENTIFIER="ON" ENCRYPTED="FALSE" />
<CommandText>CREATE TABLE [dbo].[AdventureWorksDWBuildVersion] (
[DBVersion] [nvarchar] (50) NULL,
[VersionDate] [datetime] NULL
) ON [PRIMARY];
</CommandText>
</TSQLCommand>
</EVENT_INSTANCE>' AS XML)
$inputFilePath = "C:\Temp\inputfile.sql"
$dbHostName = "$env:computername\sql1"
$outputFilePath = "C:\Temp\output.txt"
#Notice Extra Line Breaks
invoke-sqlcmd -inputFile $inputFilePath -serverinstance $dbHostName |
Format-Table -hide -Wrap -AutoSize |
Out-File -filePath $outputFilePath -width 4096
#No Extra line breaks using set-content
invoke-sqlcmd -inputFile $inputFilePath -serverinstance $dbHostName | select -expandproperty Column1 | set-content -Path $outputFilePath
Use -MaxCharLength parameter of Invoke-SQLCMD command. By default it 4000. it will truncate big XMLs.
See Invoke-SqlCmd doesn't return long string?

Resources