Powershell select value from datatable where - sql-server

I have a data table that I have imported from SQL which has two columns. One is country code and the other is country name.
I have a CVS import of country names and I want to go through the data table with each country name and get its corresponding country code.
If it were SQL I could do this but I can't think how best to achieve this in PowerShell. I could of course pass each country name to SQL and query the country code but there are approx. 7000 entries in the csv file.
I have the following code which imports the csv file and I can iterate through the objects within it
$path = "C:\Temp\myImport.csv"
$import = import-csv $path
foreach($object in $import){
}

Simply execute insert for every row.
$SQLInsertFormat="insert into [table name goes there]([columns goes there]) values ({0},{1})"
$SQLConnectionString = "[connection string goes there]"
$SQLCmd = New-Object System.Data.SqlClient.SqlCommand
$SQLCmd.CommandType = [System.Data.CommandType]::Text
$SQLCmd.Connection = $SQLConnectionString
$SQLCmd.Connection.Open()
$path = "C:\Temp\myImport.csv"
$import = import-csv $path
foreach($object in $import){
$SQLCmd.CommandText = $SQLInsertFormat -f $object.CountryCode, $object.UpperCase
$SQLCmd.ExecuteNonQuery()
}
$SQLCmd.Connection.Close()

Related

How do I execute a SELECT query against a SQLServer database and iterate results using PowerShell

Say I have a table with 3 columns - "Column1", "Column2", and "Column3" - datatype is varchar(100) for all 3.
Using PowerShell, how do I connect to SQL Server and use SqlDataReader and ForEach operator to view the contents of "Column2"?
Here's roughly how I'm doing it:
$SqlServer = 'sql.example.com';
$SqlDatabase = 'MyDB';
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI' -f $SqlServer, $SqlDatabase;
$SqlQuery = "SELECT Name FROM dbo.Person ORDER BY Name;";
$SqlConnection = New-Object -TypeName System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;
$SqlCommand = $SqlConnection.CreateCommand();
$SqlCommand.CommandText = $SqlQuery;
$SqlConnection.Open();
$SqlDataReader = $SqlCommand.ExecuteReader();
#Fetch data and write out to files
while ($SqlDataReader.Read()) {
Write-Output $SqlDataReader['Name'];
}
$SqlConnection.Close();
$SqlConnection.Dispose();
If I remember right, I basically refactored the code from the MSDN example.
For those wondering why I'm using SqlDataReader: Most of my scripts use SqlDataAdapter, but this one retrieves about 8,000 PDFs from a database so I wasn't really interested in calling SqlDataAdapter.Fill(). In exchange for holding shared locks on the table much longer than SqlDataAdapter.Fill() would, SqlDataReader.Read() keeps memory usage down to a manageable level for the client by fetching one record at a time.

Pass a query and access SQL Server Database in Powershell

I have a database named 'SQLDatabase'. It has many folders like 'System _Database', 'R_Database','ReportServer' etc.
I need to access 'R_Database'.
Now 'R_Database' has many folders like 'Storage', 'Security', 'Usage', 'Tables'.
I need to access 'Tables'.
Again, 'Tables' have many tables in it.
I need to access a particular table named 'DB_Batch', and update it.
How should I?
This is the code I did for connecting to the SQL Server, and then the Table. But it fails.
$dataSource = ".\MSSQL"
$user = "userID"
$pwd = "password01"
$database = "SQLDatabase"
$connectionString = "Server=$dataSource;uid=$user; pwd=$pwd;Database=$database;Integrated Security=False;"
$connection = New-Object System.Data.SqlClient.SqlConnection
$connection.ConnectionString = $connectionString
#$connection.ConnectionString = "Server=$dataSource;Database=$database;Integrated Security=True;"
$connection.Open()
query = <query>..????
Now I suppose I need to pass a query. How should I?
I need to access the table first and then update it.
I even wrote the following code to end the above code, but not able to test as I am stuck.
$command = $connection.CreateCommand()
$command.CommandText = $query
$result = $command.ExecuteReader()
$result
I request, please help me with this. I have been trying this for days. I am using Powershell v2.0
In order to insert/update a table, use a SqlCommand and its ExecuteNonQuery().
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.connection = $connection
$cmd.commandtext = "INSERT INTO myTable (myColumn) values ({0})" -f $myValue
$cmd.ExecuteNonQuery()

In powershell, how can I append one table to another and export to CSV?

I'm relatively new to Powershell, but here's what I'm trying to do:
I need to make multiple SQL queries and store the results to all of the queries in the same csv. With Powershell 3.0 I could use Export-CSV -Append, but unfortunately I need to use Powershell 2.0. Here is what I have right now.
#Connection Strings
$Database = "DB"
$Server = "localhost"
#Export File
$AttachmentPath = "C:\Users\Administrator\Desktop\SQLData.csv"
# Connect to SQL and query data, extract data to SQL Adapter
$SqlQuery = "select * from DB.dbo.DB1"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$Server;Initial Catalog=$Database;Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$nRecs = $SqlAdapter.Fill($DataSet)
$nRecs | Out-Null
#Populate Hash Table
$objTable = $DataSet.Tables[0]
#Export Hash Table to CSV File
$objTable | Export-CSV $AttachmentPath
Write-Output "REPORT: Successfully created ${AttachmentPath}"
This will successfully create and export my table into a csv file that looks like this:
Table 1
----------------
col1(int) col2(string) col3(string)
col1(int) col2(string) col3(string)
but now I want to run a different query the exact same way
# Connect to SQL and query data, extract data to SQL Adapter
$SqlQuery = "select * from DB.dbo.DB2"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$Server;Initial Catalog=$Database;Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$nRecs = $SqlAdapter.Fill($DataSet)
$nRecs | Out-Null
#Populate Hash Table
$objTable = $DataSet.Tables[0]
#IN POWERSHELL 3.0 THIS WOULD WORK
$objTable | Export-CSV $AttachmentPath -Append
and append it onto the first table. The final CSV would look like this:
Table 1
----------------
col1(int) col2(string) col3(string)
col1(int) col2(string) col3(string)
Table 2
----------------
col1(string) col2(int) col3(int)
col1(string) col2(int) col3(int)
I can't find any way to do this, any help is appreciated!
EDIT
Found a workaround by exporting the second query to a csv,
$objTable | Export-CSV $AttachmentPath
Then I used the Get-Content and Add-Content cmdlets to append to the first csv. It's hacky but it works. If you have anything better let me know!
$file2=Get-Content $AttachmentPath
Add-Content "SQLData.csv" "`n"
Add-Content "SQLData.csv" $file1
Why can't you use UNION in your query like below to get a merged data from both table
$SqlQuery = "select * from DB.dbo.DB1 UNION select * from DB.dbo.DB2"
Then you can use Export-CSV filename.csv commandlet to export the data to a CSV file (OR) see this thread How to export data to CSV in PowerShell?
While Union might not be acceptable for all situations, it is possible to merge two queries... just cast the output of numbers and dates to varchar/char.
SELECT Cast(col1(int) as varchar(7)) as col1, col2(string) as col2,
col3(string)
FROM [Table 1]
UNION ALL SELECT col1(string), Cast(col2(int) as varchar(7)),
Cast(col3(int) as varchar(7))
Just make sure you aren't truncating any data and you can mash em together.
Of course with 15 tables joining them in powershell would seem easier to maintain.

Remove column header from SQL Server query result

I want to remove column header from SQL Server query output. I did the search but not found any solution. I have a query eg.
select cc.DepartmentID , cc.Name from HumanResources.Department cc
When I run this query I am getting output like this.
ID Name
12 Document Control
1 Engineering
16 Executive
14 Facilities and Maintenance
10 Finance
9 Human Resources
I want to remove ID and Name (Column Header) from the output in SQL Server.
I will run this query by script to generate csv file.
Edit:
When i run the query by script i got the csv file as output and it look like this.
#TYPE System.Data.DataRow
ID Name
Update:
I am putting powershell script.
$Database = "temp"
$Server = "localhost"
$AttachmentPath = "output.csv"
# Connect to SQL and query data, extract data to SQL Adapter
$SqlQuery = "select cc.DepartmentID , cc.Name from HumanResources.Department cc"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Data Source=$Server;Initial Catalog=$Database;Integrated Security = True"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
$DataSet = New-Object System.Data.DataSet
$nRecs = $SqlAdapter.Fill($DataSet)
$nRecs | Out-Null
#Populate Hash Table
$objTable = $DataSet.Tables[0]
#Export Hash Table to CSV File
$objTable | Export-CSV $AttachmentPath
I want to remove column header from output.
In SSMS Under Tools/Options/Query Results/SQL Server/Results to Text there is a checkbox to 'Include column headers in the result set'. Similar for results to grid.
If you are using sqlcmd via powershell you can use /h-1 to disable the headers.
This setting corresponds to the environment variable SQLCMDHEADERS.
Tips and Tricks
Use a value of -1 to specify that no headers should be
printed. If -1 is supplied, there must be no space between the
parameter and the setting, that is, -h-1. Otherwise, SQLCMD interprets
it as a separate option and fails.
Example (modified from [TechNet])1:
sqlcmd -q /h-1 "SELECT * FROM AdventureWorks2012.Person.Person"
will also work with -h-1
In management studio at query window right click and select Query options. Look for Result>Text at a tree in the left and check out Include column headers in result set option. I think Hamlet Hakobyan is right, it is client that add column headers.
Replace your last line $objTable | Export-CSV $AttachmentPath with
$objTable | ConvertTo-Csv -NoTypeInformation | select -Skip 1 | out-file $AttachmentPath
Using the Save As option would not include the attribute (column) names.
This work correctly and column header not exists in out-file:
$workpath = "C:\myworkdir"
$InvSQLParams = #{
ServerInstance = "SQL2016"
Database = "testdb"
InputFile = "$($workpath)\selectclause.sql"
}
Invoke-Sqlcmd #InvSQLParams | ConvertTo-Csv -NoTypeInformation | select -Skip 1 | out-file "$($workpath)\result.csv"
in your script, pipe (|) the output to the "tail +3" command.
this will skip the first 2 lines of output from the SQL.
set this after connecting to database
SET HEADING OFF

Powershell: Retrieve xml data from column in SQL Server database

Following this tutorial I tried to use PowerShell to retrieve xml data from SQL Server, but I only get one element back.
Here is a query to show the actual data:
But running this script I only get one element back:
$SQLServer = 'MYSERVER,1433'
$SQLDBName = "test"
$Query =
#'
USE test
SELECT EventLogXML FROM ForwardedEvents
'#
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $Database; Integrated Security = True"
$SqlConnection.open()
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $Query
$SqlCmd.Connection = $SqlConnection
$xr = $SqlCmd.ExecuteXmlReader()
$xd = New-Object System.Xml.XmlDataDocument
$xd.Load($xr)
$xr.Close()
$SQLConnection.Close()
$xd
$xd only has one element. What am I doing wrong?
---edit
I can confirm its only one xml doc by doing $xd.outerxml which reveals the complete doc. It is only one of the thousand or so event xml docs I'm storing in the EventLogXML column.
I think that XmlDataDocument is mainly for returning a single xml. Basically if you do in sql select * from bla for xml, auto you then can read it with the ExecuteXmlReader and XmlDataDocument. This is not what you want.
Modifying the example you linked to your needs we'll get somethign like:
$con = New-Object System.Data.SqlClient.SqlConnection
$con.ConnectionString = "Server=.; Database=AdventureWorks2012;Integrated Security=true"
$con.open()
$cmd = New-Object System.Data.SqlClient.SqlCommand
$cmd.CommandText = "SELECT Instructions FROM Production.ProductModel WHERE Instructions is not null"
$cmd.Connection = $con
$as = New-Object System.Data.SqlClient.SqlDataAdapter
$ds = New-Object System.Data.DataSet
$as.SelectCommand = $cmd
$as.Fill($ds);
$xmlDocs = $ds.Tables[0] | %{ [xml]$_.Instructions }
Now xmlDocs will contain a list of xml documents, one document per row.
Powershell wraps XML stuff into handy little objects, which you can explore using .Property syntax. If you just look at $xd, powershell by default will only show you the root node.
I don't know the structure of your XML column, but if the root node is called MyRoot, followed by common subnodes called MySub, try something like this:
$xd.MyRoot.MySub
This is just as the linked example shows the need to use $xd.root.Location
Edit
Ok so that is not the problem. Looks like it is by-design to return back only the first row when calling ExecuteXmlReader with a normal select statement (doc here):
if more than one row is returned, the ExecuteXmlReader method attaches
the XmlReader to the value on the first row, and discards the rest of
the result set
From some basic searching around, this blog post seems to explain the issue the best, and provides a workaround. See also here.
I may be out to lunch, but couldn't it be because you are declaring the database as $SQLDBName and then trying to connect to $Database in your connectionstring?

Resources