How to import data from .csv in SQL Server using PowerShell? - sql-server

I'm using PowerShell and have to import data from a .csv file into a already created table on a SQL Server Database. So I don't need the header line from the csv, just write the data.
Here is what I have done so far:
#Setup for SQL Server connection
#SQL Server Name
$SQLServer = "APPLIK02\SQLEXPRESS"
#Database Name
$SQLDBName = "code-test"
#Create the SQL Connection Object
$SQLConn = New-Object System.Data.SQLClient.SQLConnection
#Create the SQL Command Object, to work with the Database
$SQLCmd = New-Object System.Data.SQLClient.SQLCommand
#Set the connection string one the SQL Connection Object
$SQLConn.ConnectionString = "Server=$SQLServer;Database=$SQLDBName; Integrated Security=SSPI"
#Open the connection
$SQLConn.Open()
#Handle the query with SQLCommand Object
$SQLCmd.CommandText = $query
#Provide the open connection to the Command Object as a property
$SQLCmd.Connection = $SQLConn
#Execute
$SQLReturn=$SQLCmd.ExecuteReader()
Import-module sqlps
$tablename = "dbo."+$name
Import-CSV .\$csvFile | ForEach-Object Invoke-Sqlcmd
-Database $SQLDBName -ServerInstance $SQLServer
#-Query "insert into $tablename VALUES ('$_.Column1','$_.Column2')"
#Close
$SQLReturn.Close()
$SQLConn.Close()

I wrote a blog post about using SQL with PowerShell, so you can read more about it here.
We can do this easily if you have the SQL-PS module available. Simply provide values for your database name, server name, and table, then run the following:
$database = 'foxdeploy'
$server = '.'
$table = 'dbo.powershell_test'
Import-CSV .\yourcsv.csv | ForEach-Object {Invoke-Sqlcmd `
-Database $database -ServerInstance $server `
-Query "insert into $table VALUES ('$($_.Column1)','$($_.Column2)')"
}
To be clear, replace Column1, Column2 with the names of the columns in your CSV.
Be sure that your CSV has the values in the same format as your SQL DB though, or you can run into errors.
When this is run, you will not see any output to the console. I would recommend querying afterwards to be certain that your values are accepted.

Related

How to delete record from database if no longer existed

If I have a new data/record in my CSV file then I'm importing it to my SQL Server database using PowerShell like this.
$CSVImport = Import-CSV $Global:ErrorReport
ForEach ($CSVLine1 in $CSVImport) {
$CSVHold1 = $CSVLine1.Hold
$CSVGSID1 = $CSVLine1.GSID
$CSVSource1 = $CSVLine1.Source
$CSVTYPE1 = $CSVLine1.TYPE
$CSVMessage1 = $CSVLine1.Message
$CSVCreatedDate1 = $CSVLine1.Time
$query = "USE $Global:Database
IF NOT EXISTS (SELECT * FROM Lit_Hold_Err
WHERE GSID = '$CSVGSID1'
AND Source = '$($CSVSource1 -replace "'","''")')
BEGIN
INSERT INTO Lit_Hold_Err (Hold, GSID, Source, Error, Type,Date)
VALUES('$CSVHold1', '$CSVGSID1', '$($CSVSource1 -replace "'","''")','$CSVMessage1','$CSVTYPE1', '$CSVCreatedDate1')
END"
Invoke-Sqlcmd -Query $query -ServerInstance $Global:Server -Database $Global:Database
I'm just wondering how should I write another query that compare my CSV file with my database and delete a particular/s record from my database if it's no longer existed in my CSV file?.
I know I'll need to use DELETE Statement but I'm kinda stuck on WHERE Clause so Any help or suggestion would be really appreciated.
A suggestion would be using DBATool's Import-DbaCsv to read the CSV file and bulk load it to a table on tempdb, then use a merge instruction to make the inserts/updates/deletes.
The import command would be something like:
$Credential = Get-Credential
Import-DbaCsv -Path .\test.csv -AutoCreateTable -SqlInstance my-instance -Database tempdb -SqlCredential $Credential -SupportsMultiline

Connecting powershell to SQL Server

I'm quite new to both SQL and Power shell, but I want to run a script which pulls data from a server within SQL - Any advice on where to start?
I tried using this as a starting point, but got no luck as it doesn't like my credentials even though they are correct
Test:
SqlConnection -ServerName 'END-HDSQ02\DEV4' -DatabaseName 'tbl_cert_expiry' -Credential (Get-Credential)
EDIT: Since Powershell V2, you are required to manually load the necessary Snap-Ins;
Powershell - Invoke-Sqlcmd unable to run
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
This is what we use to query a SQL View;
[string] $Server= "ServerName"
[string] $Database = "DatabaseName"
[string] $SQLQuery= $("SELECT * FROM schema.TableView order by column")
$data = Invoke-Sqlcmd -ServerInstance $server -Database $database -Username $username -Password $value1 -Query $SQLQuery
Obviously you'll need to pass the connecting user and password as well, but I've omitted them from my example.

Import CSV from PowerShell to SQL Server

I was trying to import a CSV file from PowerShell over to my SQL Server database. I've already created the database and tables with columns. I got the data on a CSV file that I need to import.
I tried to research and found a bit of code I modified so it should be working, but when I run the code I get the error:
Import-CsvToSql : exeption calling "writetoserver" with "1"
argument(s): "The transaction is iether not associated with the
connection or has been completed"
But I hasn't imported the data to the table, so I don't know what's wrong.
Here is the code I've got so far:
Import-Module csvsqlimport
Import-CsvToSql -Csv C:\Users\Tim\Desktop\POWERSHELL\AutoParts.csv `
-SqlServer DHCP_SERVER -Database FeilAuto4 `
-Table dbo.Reservedele -FirstRowColumns -Delimiter ";" -Truncate
You may want to try the "SqlServer" module as it is being kept up to date by Microsoft and has multiple SQL cmdlets. The only downside is that you will have to separate the script into multiple commands based on their cmdlets.
## Install module if not installed, this is a one time install.
Install-Module SqlServer
## Input Variables
$csvPath = "C:\Users\Tim\Desktop\POWERSHELL\AutoParts.csv"
$csvDelimiter = ";"
$serverName = "DHCP_SERVER"
$databaseName = "FeilAuto4"
$tableSchema = "dbo"
$tableName = "Reservedele"
## Truncate Table
Invoke-Sqlcmd -ServerInstance $serverName -Database $databaseName -Query "TRUNCATE TABLE $tableSchema.$tableName"
## Import CSV into SQL
Import-Csv -Path $csvPath -Delimiter $csvDelimiter | Write-SqlTableData -ServerInstance $serverName -DatabaseName $databaseName -SchemaName $tableSchema -TableName $tableName -Force

ADO.NET query not showing in XEvents

I sent a query to SQL Server using an ADO.Net SqlAdapter from PowerShell. The query returns the correct result.
When I run an XEvent session with the sqlserver.sql_statement_starting event, the query from ADO.NET does not show up. Queries I sent from SSMS are shown immediately.
Is this a bug, or why do I not see the ADO.NET queries?
The code I am using is
$serverName = 'localhost'
$databaseName = 'Contoso Retail DW'
$schemaName = 'dbo'
$tableName = 'FactSalesFMCG'
$connString = Get-ConnectionString -IntegratedSecurity -Server $serverName -Database $databaseName
$sqlConn = [System.Data.SqlClient.SqlConnection]::new($connString)
$sqlConn.Open()
$columnMetadataAdapter = Get-ColumnMetadataAdapter -Conn $sqlConn -SchemaName $schemaName -TableName $tableName
$table = [System.Data.DataTable]::new()
$columnMetadataAdapter.Fill($table)
$sqlConn.Close()
Get-ConnectionString and Get-ColumnMetadataAdapter are PowerShell functions that assist in creating the needed ADO.NET objects. The table gets filled with the column metadata I wanted, but the SELECT statement is not shown in XEvents.
Can't repro.
CREATE EVENT SESSION [trc] ON SERVER
ADD EVENT sqlserver.rpc_completed,
ADD EVENT sqlserver.sp_statement_completed,
ADD EVENT sqlserver.sql_batch_completed,
ADD EVENT sqlserver.sql_statement_completed
GO
Start the session and watch live events in SSMS.
Then
PS C:\Users\dbrowne> $da = new-object system.data.sqlclient.sqldataadapter
PS C:\Users\dbrowne> $con = new-object system.data.sqlclient.sqlconnection "server=.;database=tempdb;integrated security=true"
PS C:\Users\dbrowne> $con.open()
PS C:\Users\dbrowne> $cmd = $con.createcommand()
PS C:\Users\dbrowne> $cmd.commandtext = "select * from sys.objects"
PS C:\Users\dbrowne> $da.selectcommand = $cmd
PS C:\Users\dbrowne> $dt = new-object system.data.datatable
PS C:\Users\dbrowne> $da.fill($dt)
106
and see both the sql_statement_completed and the sql_batch_completed (note with different code you might get an rpc_completed instead of a sql_batch_completed).
If you bind parameters into the SqlCommand it will be sent as an RPC call instead of a batch call, and the events will be a bit different rpc/sp instead of batch/sql.
rpc_completed/sp_statement_completed
instead of
sql_batch_completed/sql_statement_completed

How do I execute a SELECT query against a SQLServer database and iterate results using PowerShell

Say I have a table with 3 columns - "Column1", "Column2", and "Column3" - datatype is varchar(100) for all 3.
Using PowerShell, how do I connect to SQL Server and use SqlDataReader and ForEach operator to view the contents of "Column2"?
Here's roughly how I'm doing it:
$SqlServer = 'sql.example.com';
$SqlDatabase = 'MyDB';
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI' -f $SqlServer, $SqlDatabase;
$SqlQuery = "SELECT Name FROM dbo.Person ORDER BY Name;";
$SqlConnection = New-Object -TypeName System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;
$SqlCommand = $SqlConnection.CreateCommand();
$SqlCommand.CommandText = $SqlQuery;
$SqlConnection.Open();
$SqlDataReader = $SqlCommand.ExecuteReader();
#Fetch data and write out to files
while ($SqlDataReader.Read()) {
Write-Output $SqlDataReader['Name'];
}
$SqlConnection.Close();
$SqlConnection.Dispose();
If I remember right, I basically refactored the code from the MSDN example.
For those wondering why I'm using SqlDataReader: Most of my scripts use SqlDataAdapter, but this one retrieves about 8,000 PDFs from a database so I wasn't really interested in calling SqlDataAdapter.Fill(). In exchange for holding shared locks on the table much longer than SqlDataAdapter.Fill() would, SqlDataReader.Read() keeps memory usage down to a manageable level for the client by fetching one record at a time.

Resources