How to use PowerShell to batch call Update-Database - database

We use an Azure Elastic Pool resulting in multiple client databases and one master database with references to the client database.
We already have multiple databases and are working on a new version of the code. We use EF6 Code-First.
When we make a change to our model (add a property) we create the migration file and need to call Update-Database for all existing client databases.
This is monkey work we want to skip.
I already have a Powershell script to connect to the master database and execute a query on a table. This returns the names of the child databases.
With it I can change the Web.config and replace the Template database name with the proper name of the child database.
Now I need to call Update-Database to execute the migration scripts. With this last part I'm struggling because I'm running the ps1-script outside Visual Studio and thus the command Update-database is unknown. I tried using migrate.exe but then I get lots of errors.
I think the easiest solution is to run my script within the Package manager console but I can't figure out how to do that.

I managed to get it working. After I placed the ps1-file in the root of my code folder I could run it in the Package Manager Console using .\UpdateDatabases.ps1.
For completeness here's the script I created. I'm new to PowerShell so some optimizations might be possible.
cls
$currentPath = (Get-Item -Path ".\" -Verbose).FullName
#Read Web.config
$webConfig = $currentPath + "\<your project>\Web.config"
$doc = (Get-Content $webConfig) -as [Xml]
$DatabaseNamePrefix = $doc.configuration.appSettings.add | where {$_.Key -eq 'DatabaseNamePrefix'}
#Get Master connectionstring
$root = $doc.get_DocumentElement();
foreach($connString in $root.connectionStrings.add | where {$_.Name -eq "Master"})
{
$masterConn = $connString.connectionString
}
#Connect to master database
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = $masterConn
#Query Client table for the child database names
$SqlQuery = "select Code from Clients"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
#Put query result in dataset
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
foreach ($row in $DataSet.Tables[0].Rows)
{
$clientDbName = $row[0].ToString().Trim()
#Change Web.Config
foreach($connString in $root.connectionStrings.add | where {$_.Name -eq "DevelopmentDb"})
{
$newDatabaseName = "Database=" + $DatabaseNamePrefix.value + $clientDbName + ";";
$newConn = $connString.connectionString -replace "(Database=.*?;)",$newDatabaseName
$connString.connectionString = $newConn;
}
$doc.Save($webConfig)
#Update database
Update-Database -ConfigurationTypeName Application
}
"Finished"

You may want to take a look at Azure Elastic Database Jobs. Which is designed to work with the elastic database pools.
The Elastic Database Jobs SDK includes also PowerShell components.

Related

How to create a copy of existing SQL DB on same server

I to perform an operation a part of which has me looking for a way to create a copy of SQL DB on same server. I tried the suggestion given at Copy SQL Server database with PowerShell script . However the resulting copy is about a quarter size of the actual DB.
Ideas anyone?
Thanks
If your PowerShell solution is working except you are noticing a file-size discrepancy with the newly copied database compared to the source database, it may not be an actual problem.
SQL Server database and log sizes are variable and are typically not an exact indication of the amount of data they contain. The copied database may be "optimized" in terms of its disk file usage in a way that the source database is currently not.
There are three things you can do to convince yourself you have a working solution.
Run the Shrink command on both of the databases to free space and see if the resulting disk files are more similar in terms of size https://learn.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-shrinkdatabase-transact-sql?view=sql-server-ver15
Write a benchmark script that compares record counts on all the tables. If you see a discrepancy in the record count, you know you have a problem.
Example benchmark script:
declare #sourceCount int;
declare #copyCount int;
set #sourceCount = (select count(*) from SourceDb.dbo.SomeTable);
set #copyCount = (select count(*) from CopyDb.dbo.SomeTable);
if #sourceCount <> #copyCount
begin
select 'PROBLEM!'
end
-- Now repeat for all the other tables
Use a SQL data comparison tool to automate the method in step 2. above. There are many such tools including the one built-in to Visual Studio 2019 (Tools menu) or otherwise: https://learn.microsoft.com/en-us/sql/ssdt/how-to-compare-and-synchronize-the-data-of-two-databases?view=sql-server-ver15
However, all these methods will only work reliably if you can ensure that either database isn't receiving updates during the copy/benchmarking process. If someone is accessing one or the other of the databases independently while you are measuring, and they alter the data independently while you are measuring, your results would be invalidated, possibly without your knowing.
EDIT
I managed to make it work as soon as I have started using the SqlServer module instead of the SQLPS module, because the latter had long been deprecated. I edited the answer I am referring to in my initial post, below.
I was having a similar error implementing this. Tried literally everything, it just wouldn't work. What did work for me, was generating a script through the ScriptTransfer method, create the new database and then apply the script to the new database through Invoke-SqlCmd. I have posted a detailed explanation and code in this answer.
Okay. I managed to implement this. And anybody who needs to do this in future, please try this:
Import-Module SQLPS -DisableNameChecking
$SQLInstanceName = "$env:COMPUTERNAME\sqlexpress"
$SourceDBName = "xxx"
$CopyDBName = "${SourceDBName}_copy"
$Server = New-Object -TypeName 'Microsoft.SqlServer.Management.Smo.Server' -ArgumentList $SQLInstanceName
$SourceDB = $Server.Databases[$SourceDBName]
$CopyDB = New-Object -TypeName 'Microsoft.SqlServer.Management.SMO.Database' -ArgumentList $Server , $CopyDBName
# Delete any existing copy
Try
{
Invoke-Sqlcmd -ServerInstance "$SQLInstanceName" -Query "Drop database $CopyDBName;" -Username "***" -Password "****" -Verbose
}
Catch
{
Write-Output 'Failed to delete database'
}
$CopyDB.create()
$ObjTransfer = New-Object -TypeName Microsoft.SqlServer.Management.SMO.Transfer -ArgumentList $SourceDB
$ObjTransfer.DestinationDatabase = $CopyDBName
$ObjTransfer.DestinationServer = $Server.Name
$ObjTransfer.DestinationLoginSecure = $true
$ObjTransfer.CopyData = $true
$ObjTransfer.CopyAllObjects = $false
$ObjTransfer.CopyAllDatabaseTriggers = $true
$ObjTransfer.CopyAllDefaults = $true
$ObjTransfer.CopyAllRoles = $true
$ObjTransfer.CopyAllRules = $true
$ObjTransfer.CopyAllSchemas = $true
$ObjTransfer.CopyAllSequences = $true
$ObjTransfer.CopyAllSqlAssemblies = $true
$ObjTransfer.CopyAllSynonyms = $true
$ObjTransfer.CopyAllTables = $true
$ObjTransfer.CopyAllViews = $true
$ObjTransfer.CopyAllStoredProcedures = $true
$ObjTransfer.CopyAllUserDefinedAggregates = $true
$ObjTransfer.CopyAllUserDefinedDataTypes = $true
$ObjTransfer.CopyAllUserDefinedTableTypes = $true
$ObjTransfer.CopyAllUserDefinedTypes = $true
$ObjTransfer.CopyAllUserDefinedFunctions = $true
$ObjTransfer.CopyAllUsers = $true
$ObjTransfer.PreserveDbo = $true
$ObjTransfer.Options.AllowSystemObjects = $false
$ObjTransfer.Options.ContinueScriptingOnError = $true
$ObjTransfer.Options.IncludeDatabaseRoleMemberships = $true
$ObjTransfer.Options.Indexes = $true
$ObjTransfer.Options.Permissions = $true
$ObjTransfer.Options.WithDependencies = $true
$ObjTransfer.TransferData()

ADO.NET query not showing in XEvents

I sent a query to SQL Server using an ADO.Net SqlAdapter from PowerShell. The query returns the correct result.
When I run an XEvent session with the sqlserver.sql_statement_starting event, the query from ADO.NET does not show up. Queries I sent from SSMS are shown immediately.
Is this a bug, or why do I not see the ADO.NET queries?
The code I am using is
$serverName = 'localhost'
$databaseName = 'Contoso Retail DW'
$schemaName = 'dbo'
$tableName = 'FactSalesFMCG'
$connString = Get-ConnectionString -IntegratedSecurity -Server $serverName -Database $databaseName
$sqlConn = [System.Data.SqlClient.SqlConnection]::new($connString)
$sqlConn.Open()
$columnMetadataAdapter = Get-ColumnMetadataAdapter -Conn $sqlConn -SchemaName $schemaName -TableName $tableName
$table = [System.Data.DataTable]::new()
$columnMetadataAdapter.Fill($table)
$sqlConn.Close()
Get-ConnectionString and Get-ColumnMetadataAdapter are PowerShell functions that assist in creating the needed ADO.NET objects. The table gets filled with the column metadata I wanted, but the SELECT statement is not shown in XEvents.
Can't repro.
CREATE EVENT SESSION [trc] ON SERVER
ADD EVENT sqlserver.rpc_completed,
ADD EVENT sqlserver.sp_statement_completed,
ADD EVENT sqlserver.sql_batch_completed,
ADD EVENT sqlserver.sql_statement_completed
GO
Start the session and watch live events in SSMS.
Then
PS C:\Users\dbrowne> $da = new-object system.data.sqlclient.sqldataadapter
PS C:\Users\dbrowne> $con = new-object system.data.sqlclient.sqlconnection "server=.;database=tempdb;integrated security=true"
PS C:\Users\dbrowne> $con.open()
PS C:\Users\dbrowne> $cmd = $con.createcommand()
PS C:\Users\dbrowne> $cmd.commandtext = "select * from sys.objects"
PS C:\Users\dbrowne> $da.selectcommand = $cmd
PS C:\Users\dbrowne> $dt = new-object system.data.datatable
PS C:\Users\dbrowne> $da.fill($dt)
106
and see both the sql_statement_completed and the sql_batch_completed (note with different code you might get an rpc_completed instead of a sql_batch_completed).
If you bind parameters into the SqlCommand it will be sent as an RPC call instead of a batch call, and the events will be a bit different rpc/sp instead of batch/sql.
rpc_completed/sp_statement_completed
instead of
sql_batch_completed/sql_statement_completed

How to get the compatibility level of a database from analysis server?

I searched around and found there is a way to get compatibility level of a database in powershell through something like:
Import-Module SqlServer
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=$Server;Initial Catalog=$DB;Integrated Security=SSPI"
$conn = new-object Microsoft.SqlServer.Management.Common.ServerConnection($sqlConnection)
$srv = new-object Microsoft.SqlServer.Management.Smo.Server($conn)
$db = New-Object Microsoft.SqlServer.Management.Smo.Database
$db = $srv.Databases.Item("$DB")
$comp_lvl = New-Object Microsoft.SqlServer.Management.Smo.CompatibilityLevel
$comp_lvl = $db.CompatiblityLevel
Write-Host "Compatibility level =" $db.CompatibilityLevel
However, I am getting errors trying to get compatibility level of a database that is on an analysis server
Exception getting "Item": "Failed to connect to server ..."
I realized it is probably working for a regular database engine, but something else may be used for an analysis server. I looked around on MS Docs and didnt really find anything helpful.
SO is this even possible?
UPDATE:
I was able to find something on this page: https://technet.microsoft.com/en-us/hh213141(v=sql.100)
Import-Module SqlServer
$as = New-Object Microsoft.AnalysisServices.Server
$as.connect("$Server")
$as.databases
Write-Host "Compatibility level ="$as.DefaultCompatibilityLevel
but this returns ALL databases back...
I want to specify just one database to get the compatibility level of...
I tried this,
$as.databases["$Database"]
but it seems not to return the proper level, because the DB i am passing has a lvl of 1103, not 1200...
I figured it out!
Import-Module SqlServer
$as = New-Object Microsoft.AnalysisServices.Server
$as.connect("$Server")
$c = $as.Databases | Where-Object { $_.ID -eq $Database }
Write-Host "Compatibility level =" $c.CompatibilityLevel

How do I execute a SELECT query against a SQLServer database and iterate results using PowerShell

Say I have a table with 3 columns - "Column1", "Column2", and "Column3" - datatype is varchar(100) for all 3.
Using PowerShell, how do I connect to SQL Server and use SqlDataReader and ForEach operator to view the contents of "Column2"?
Here's roughly how I'm doing it:
$SqlServer = 'sql.example.com';
$SqlDatabase = 'MyDB';
$SqlConnectionString = 'Data Source={0};Initial Catalog={1};Integrated Security=SSPI' -f $SqlServer, $SqlDatabase;
$SqlQuery = "SELECT Name FROM dbo.Person ORDER BY Name;";
$SqlConnection = New-Object -TypeName System.Data.SqlClient.SqlConnection -ArgumentList $SqlConnectionString;
$SqlCommand = $SqlConnection.CreateCommand();
$SqlCommand.CommandText = $SqlQuery;
$SqlConnection.Open();
$SqlDataReader = $SqlCommand.ExecuteReader();
#Fetch data and write out to files
while ($SqlDataReader.Read()) {
Write-Output $SqlDataReader['Name'];
}
$SqlConnection.Close();
$SqlConnection.Dispose();
If I remember right, I basically refactored the code from the MSDN example.
For those wondering why I'm using SqlDataReader: Most of my scripts use SqlDataAdapter, but this one retrieves about 8,000 PDFs from a database so I wasn't really interested in calling SqlDataAdapter.Fill(). In exchange for holding shared locks on the table much longer than SqlDataAdapter.Fill() would, SqlDataReader.Read() keeps memory usage down to a manageable level for the client by fetching one record at a time.

How to copy tables plus data for testing purposes

How can I copy a number of tables, plus the data they contain, from a live SQL Server database?
I want to get some data for basic testing on my machine.
I was originally going to do a simple backup of the database, until I noticed that it was over 100GB in size. However the tables I'm after are only a number of the smaller ones. I then tried export to Excel, but hit the 65K limit (and we don't have Excel 2007)
You can try Exporting Data by Using the SQL Server Import and Export Wizard
Here is MSDN video
you can export it as Flat file
In Management Studio, select the database, right-click and select Tasks->Export Data. There you will see options to export to different kinds of formats including CSV.
You can also run your query from the Query window and save the results to CSV.
Can't you use the Export Data wizard from your live server to your testing machine? Or use bcp? Or even use a simple PowerShell script?
$Server = "MyServer"
$ServerInstance = "$Server\MyInstance"
$database = "MyDatabase"
$BackupFile = "c:\MyBackupFile.sql"
$tables = #('TableBlah','TableBluh','TableBloh')
$server = New-Object (
'Microsoft.SqlServer.Management.Smo.Server') $ServerInstance
$scripter = New-Object ('Microsoft.SqlServer.Management.Smo.Scripter') $server
$scripter.Options.SchemaQualify = $false
$scripter.Options.ScriptSchema = $false
$scripter.Options.ScriptData = $true
$scripter.Options.NoCommandTerminator = $true
$scripter.Options.ToFileOnly = $true
$scripter.Options.FileName = $BackupFile
$ServerUrn=$server.Urn
$UrnsToScript = New-Object Microsoft.SqlServer.Management.Smo.UrnCollection
foreach ($t in $tables)
{
# Could use bcp here for dumping big tables (like archives)
# $ret = (bcp.exe "$database..$t" out `"$ConfigBackupDir\$t.bcp`"
# -S $ServerInstance -U sa -P $SAPWD -n)
$Urn = "$ServerUrn/Database[#Name='" +
$database + "']/Table[#Name='" + $t + "' and #Schema='dbo']"
$UrnsToScript.Add($Urn)
}
$scripter.EnumScript($UrnsToScript)

Resources