I to perform an operation a part of which has me looking for a way to create a copy of SQL DB on same server. I tried the suggestion given at Copy SQL Server database with PowerShell script . However the resulting copy is about a quarter size of the actual DB.
Ideas anyone?
Thanks
If your PowerShell solution is working except you are noticing a file-size discrepancy with the newly copied database compared to the source database, it may not be an actual problem.
SQL Server database and log sizes are variable and are typically not an exact indication of the amount of data they contain. The copied database may be "optimized" in terms of its disk file usage in a way that the source database is currently not.
There are three things you can do to convince yourself you have a working solution.
Run the Shrink command on both of the databases to free space and see if the resulting disk files are more similar in terms of size https://learn.microsoft.com/en-us/sql/t-sql/database-console-commands/dbcc-shrinkdatabase-transact-sql?view=sql-server-ver15
Write a benchmark script that compares record counts on all the tables. If you see a discrepancy in the record count, you know you have a problem.
Example benchmark script:
declare #sourceCount int;
declare #copyCount int;
set #sourceCount = (select count(*) from SourceDb.dbo.SomeTable);
set #copyCount = (select count(*) from CopyDb.dbo.SomeTable);
if #sourceCount <> #copyCount
begin
select 'PROBLEM!'
end
-- Now repeat for all the other tables
Use a SQL data comparison tool to automate the method in step 2. above. There are many such tools including the one built-in to Visual Studio 2019 (Tools menu) or otherwise: https://learn.microsoft.com/en-us/sql/ssdt/how-to-compare-and-synchronize-the-data-of-two-databases?view=sql-server-ver15
However, all these methods will only work reliably if you can ensure that either database isn't receiving updates during the copy/benchmarking process. If someone is accessing one or the other of the databases independently while you are measuring, and they alter the data independently while you are measuring, your results would be invalidated, possibly without your knowing.
EDIT
I managed to make it work as soon as I have started using the SqlServer module instead of the SQLPS module, because the latter had long been deprecated. I edited the answer I am referring to in my initial post, below.
I was having a similar error implementing this. Tried literally everything, it just wouldn't work. What did work for me, was generating a script through the ScriptTransfer method, create the new database and then apply the script to the new database through Invoke-SqlCmd. I have posted a detailed explanation and code in this answer.
Okay. I managed to implement this. And anybody who needs to do this in future, please try this:
Import-Module SQLPS -DisableNameChecking
$SQLInstanceName = "$env:COMPUTERNAME\sqlexpress"
$SourceDBName = "xxx"
$CopyDBName = "${SourceDBName}_copy"
$Server = New-Object -TypeName 'Microsoft.SqlServer.Management.Smo.Server' -ArgumentList $SQLInstanceName
$SourceDB = $Server.Databases[$SourceDBName]
$CopyDB = New-Object -TypeName 'Microsoft.SqlServer.Management.SMO.Database' -ArgumentList $Server , $CopyDBName
# Delete any existing copy
Try
{
Invoke-Sqlcmd -ServerInstance "$SQLInstanceName" -Query "Drop database $CopyDBName;" -Username "***" -Password "****" -Verbose
}
Catch
{
Write-Output 'Failed to delete database'
}
$CopyDB.create()
$ObjTransfer = New-Object -TypeName Microsoft.SqlServer.Management.SMO.Transfer -ArgumentList $SourceDB
$ObjTransfer.DestinationDatabase = $CopyDBName
$ObjTransfer.DestinationServer = $Server.Name
$ObjTransfer.DestinationLoginSecure = $true
$ObjTransfer.CopyData = $true
$ObjTransfer.CopyAllObjects = $false
$ObjTransfer.CopyAllDatabaseTriggers = $true
$ObjTransfer.CopyAllDefaults = $true
$ObjTransfer.CopyAllRoles = $true
$ObjTransfer.CopyAllRules = $true
$ObjTransfer.CopyAllSchemas = $true
$ObjTransfer.CopyAllSequences = $true
$ObjTransfer.CopyAllSqlAssemblies = $true
$ObjTransfer.CopyAllSynonyms = $true
$ObjTransfer.CopyAllTables = $true
$ObjTransfer.CopyAllViews = $true
$ObjTransfer.CopyAllStoredProcedures = $true
$ObjTransfer.CopyAllUserDefinedAggregates = $true
$ObjTransfer.CopyAllUserDefinedDataTypes = $true
$ObjTransfer.CopyAllUserDefinedTableTypes = $true
$ObjTransfer.CopyAllUserDefinedTypes = $true
$ObjTransfer.CopyAllUserDefinedFunctions = $true
$ObjTransfer.CopyAllUsers = $true
$ObjTransfer.PreserveDbo = $true
$ObjTransfer.Options.AllowSystemObjects = $false
$ObjTransfer.Options.ContinueScriptingOnError = $true
$ObjTransfer.Options.IncludeDatabaseRoleMemberships = $true
$ObjTransfer.Options.Indexes = $true
$ObjTransfer.Options.Permissions = $true
$ObjTransfer.Options.WithDependencies = $true
$ObjTransfer.TransferData()
Related
I searched around and found there is a way to get compatibility level of a database in powershell through something like:
Import-Module SqlServer
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server=$Server;Initial Catalog=$DB;Integrated Security=SSPI"
$conn = new-object Microsoft.SqlServer.Management.Common.ServerConnection($sqlConnection)
$srv = new-object Microsoft.SqlServer.Management.Smo.Server($conn)
$db = New-Object Microsoft.SqlServer.Management.Smo.Database
$db = $srv.Databases.Item("$DB")
$comp_lvl = New-Object Microsoft.SqlServer.Management.Smo.CompatibilityLevel
$comp_lvl = $db.CompatiblityLevel
Write-Host "Compatibility level =" $db.CompatibilityLevel
However, I am getting errors trying to get compatibility level of a database that is on an analysis server
Exception getting "Item": "Failed to connect to server ..."
I realized it is probably working for a regular database engine, but something else may be used for an analysis server. I looked around on MS Docs and didnt really find anything helpful.
SO is this even possible?
UPDATE:
I was able to find something on this page: https://technet.microsoft.com/en-us/hh213141(v=sql.100)
Import-Module SqlServer
$as = New-Object Microsoft.AnalysisServices.Server
$as.connect("$Server")
$as.databases
Write-Host "Compatibility level ="$as.DefaultCompatibilityLevel
but this returns ALL databases back...
I want to specify just one database to get the compatibility level of...
I tried this,
$as.databases["$Database"]
but it seems not to return the proper level, because the DB i am passing has a lvl of 1103, not 1200...
I figured it out!
Import-Module SqlServer
$as = New-Object Microsoft.AnalysisServices.Server
$as.connect("$Server")
$c = $as.Databases | Where-Object { $_.ID -eq $Database }
Write-Host "Compatibility level =" $c.CompatibilityLevel
We use an Azure Elastic Pool resulting in multiple client databases and one master database with references to the client database.
We already have multiple databases and are working on a new version of the code. We use EF6 Code-First.
When we make a change to our model (add a property) we create the migration file and need to call Update-Database for all existing client databases.
This is monkey work we want to skip.
I already have a Powershell script to connect to the master database and execute a query on a table. This returns the names of the child databases.
With it I can change the Web.config and replace the Template database name with the proper name of the child database.
Now I need to call Update-Database to execute the migration scripts. With this last part I'm struggling because I'm running the ps1-script outside Visual Studio and thus the command Update-database is unknown. I tried using migrate.exe but then I get lots of errors.
I think the easiest solution is to run my script within the Package manager console but I can't figure out how to do that.
I managed to get it working. After I placed the ps1-file in the root of my code folder I could run it in the Package Manager Console using .\UpdateDatabases.ps1.
For completeness here's the script I created. I'm new to PowerShell so some optimizations might be possible.
cls
$currentPath = (Get-Item -Path ".\" -Verbose).FullName
#Read Web.config
$webConfig = $currentPath + "\<your project>\Web.config"
$doc = (Get-Content $webConfig) -as [Xml]
$DatabaseNamePrefix = $doc.configuration.appSettings.add | where {$_.Key -eq 'DatabaseNamePrefix'}
#Get Master connectionstring
$root = $doc.get_DocumentElement();
foreach($connString in $root.connectionStrings.add | where {$_.Name -eq "Master"})
{
$masterConn = $connString.connectionString
}
#Connect to master database
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = $masterConn
#Query Client table for the child database names
$SqlQuery = "select Code from Clients"
$SqlCmd = New-Object System.Data.SqlClient.SqlCommand
$SqlCmd.CommandText = $SqlQuery
$SqlCmd.Connection = $SqlConnection
$SqlAdapter = New-Object System.Data.SqlClient.SqlDataAdapter
$SqlAdapter.SelectCommand = $SqlCmd
#Put query result in dataset
$DataSet = New-Object System.Data.DataSet
$SqlAdapter.Fill($DataSet)
$SqlConnection.Close()
foreach ($row in $DataSet.Tables[0].Rows)
{
$clientDbName = $row[0].ToString().Trim()
#Change Web.Config
foreach($connString in $root.connectionStrings.add | where {$_.Name -eq "DevelopmentDb"})
{
$newDatabaseName = "Database=" + $DatabaseNamePrefix.value + $clientDbName + ";";
$newConn = $connString.connectionString -replace "(Database=.*?;)",$newDatabaseName
$connString.connectionString = $newConn;
}
$doc.Save($webConfig)
#Update database
Update-Database -ConfigurationTypeName Application
}
"Finished"
You may want to take a look at Azure Elastic Database Jobs. Which is designed to work with the elastic database pools.
The Elastic Database Jobs SDK includes also PowerShell components.
I have a script that I created a powershell script based on a script I found, Start-Migration. It is a great script and gave me a lot of really good ideas; however I am running into some issues when attempting to restore large databases or databases that take longer than 10 minutes. I have attempted to use both them invoke-sqlcmd2 function I found and the class for the restore for the microsoft.sqlserver.management.smo namespace. both of which are timing out after 10 minutes. I have also tried increasing the connection timeout even setting the connection to 1200. any suggestions would be welcomed.
Function Restore-SQLDatabase {
<#
.SYNOPSIS
Restores .bak file to SQL database. Creates db if it doesn't exist. $filestructure is
a custom object that contains logical and physical file locations.
.EXAMPLE
$filestructure = Get-SQLFileStructures $sourceserver $destserver $ReuseFolderstructure
Restore-SQLDatabase $destserver $dbname $backupfile $filestructure
.OUTPUTS
$true if success
$true if failure
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[object]$server,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string]$dbname,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[string]$backupfile,
[Parameter(Mandatory = $true)]
[ValidateNotNullOrEmpty()]
[object]$filestructure
)
$servername = $server.Name
$server.ConnectionContext.StatementTimeout = 0
$restore = New-Object "Microsoft.SqlServer.Management.Smo.Restore"
foreach ($file in $filestructure.databases[$dbname].destination.values) {
$movefile = New-Object "Microsoft.SqlServer.Management.Smo.RelocateFile"
$movefile.LogicalFileName = $file.logical
$movefile.PhysicalFileName = $file.physical
$null = $restore.RelocateFiles.Add($movefile)
}
Write-Host "Restoring $dbname to $servername" -ForegroundColor Yellow
try{
$Percent = [Microsoft.SqlServer.Management.Smo.PercentCompleteEventHandler]{
Write-Progress -Id 1 -Activity "Restoring $dbname to $ServerName" -PercentComplete $_.Percent -Status ([System.String]::Format("Progress: {0}%",$_.Percent))
}
$restore.add_PercentComplete($Percent)
$restore.PercentCompleteNotification = 1
$restore.add_Complete($Complete)
$restore.ReplaceDatabase = $true
$restore.Database = $dbname
$restore.Action = "Database"
$restore.NoRecovery = $false
$device = New-Object -TypeName Microsoft.SqlServer.Management.Smo.BackupDeviceItem
$device.Name = $backupfile
$device.DeviceType = "File"
$restore.Devices.Add($device)
Write-Progress -Id 1 -Activity "Restoring $dbname to $servername" -PercentComplete 0 -Status([System.String]::Format("Progress: {0}%",0))
$restore.SqlRestore($servername)
# $query = $restore.Script($ServerName)
# Write-Host $query
# Invoke-Sqlcmd2 -ServerInstance $servername -Database master -Query $query -ConnectionTimeout 1200
# Write-Host "Restoring $dbname to $servername from " $restore.Devices.ToString() -ForegroundColor Magenta
Write-Progress -Id 1 -Activity "Restore $dbname to $servername" -Status "Complete" -Completed
return $true
}
catch{
Write-Error $_.Exception.ToString()
Write-Warning "Restore failed: $($_.Exception.InnerException.Message)"
return $false
}
when the restore process takes place ,$restore.SqlRestore($ServerName), on my larger databases it returns saying that the script timed out. I am trying to figure out how to correct this. I have tried increasing the statementtimeout = 1200 and it still stops after 10 minutes. i even attempted to us an invoke-sqlcmd As you can see I commented it out when trying different options. I am at wits end right now.
If you have to use the script you found
I think that this line `$server.ConnectionContext.StatementTimeout = 0' is not actually doing anything or changing the timeout threshold.
Try changing the line to this, instead
$server = New-Object("Microsoft.SqlServer.Management.Smo.Server") $server
$server.ConnectionContext.StatementTimeout = 0
$server.ConnectionContext.Connect()
However, I would recommend avoiding this approach entirely, because you can use the much much easier SQLPS Cmdlets. The approach you're using comes from a long time ago, in which we didn't have actual PowerShell cmdlets to work with SQL.
Nowadays, we have the Restore-SQLDatabase cmdlet, which lets you do with one line what your code now is doing in ~30 lines!!
How to Use Restore-SQLDatabase instead
This is so much easier, and you'll really thank me, I think.
Restore-SqlDatabase -ServerInstance "$server\InstanceName" `
-Database $dbname `-BackupFile $BackupFile
and...that's it. One single line! And you can also specify the timeout easily here, unlike before. To specify the maximum timeout:
Restore-SqlDatabase -ServerInstance "$server\InstanceName" `
-Database $dbname -BackupFile $BackupFile -ConnectionTimeout 0
That should get you started. Trust me, using the Cmdlets is so much easier than using some random script you find online.
Okay I got it to work. I am not completely sure why it worked but here is what I did. I created a parameter $Server as follows:
$Server = New-Object Microsoft.SqlServer.Management.Smo.Server $Source
$Server.ConnectionContext.ConnectTimeout = 0
Now when I call the function that actually does the restore, I was doing something that I didn't need to do:
$ServerName = $server.Name
instead of using the $ServerName, I used the object $server:
$restore.SqlRestore($servername) # Original Script
$restore.SqlRestore($server) # Change
and now it stays open long enough to complete the restore process.
I am trying to call a SP (Ola's maintenance script!) on a remote server (that part of the code works), and the output of the SP is the results of DBCC CHECKDB (so it's in the Message tab).
I tried to put together some code to capture this Message output into a file on the remote server, but the file is not being created, though the SP completes fine.
$OutputFile = "\\XXX\E$\SQLAdmin\DatabaseCheckDB\ScriptOutput\ScriptOutput.txt"
$handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] {param($sender, $event) Out-File -filepath $OutputFile -inputobject $event.Message };
$SqlConnection.add_InfoMessage($handler);
$SqlConnection.FireInfoMessageEventOnUserErrors = $true;
$SqlConnection = new-Object System.Data.SqlClient.SqlConnection("Server=XXX;DataBase=master;Integrated Security=SSPI")
$SqlConnection.Open() | Out-Null
$cmd = new-Object System.Data.SqlClient.SqlCommand("dbo.DatabaseIntegrityCheck", $SqlConnection)
$cmd.CommandType = [System.Data.CommandType]'StoredProcedure'
$cmd.Parameters.Add("#Databases","ALL_DATABASES") | Out-Null
$cmd.ExecuteNonQuery() | Out-Null
$SqlConnection.Close()
Can anyone see what i'm doing wrong here? THanks in advance!
Do you have the SQL Powershell module installed (sqlps)? If so, then you can use this and pipe the output from the Verbose stream (which contains printed messages from SQL), to your file.
Invoke-Sqlcmd -Query 'DBCC CHECKDB' `
-ServerInstance '(local)' `
-Database 'tempdb' `
-Verbose 4>&1 |
Out-File c:\temp\test.txt
If that isn't an option, then I think I have spotted the problem in your original code - you wire up the InfoMessage event, but you then proceed to create a brand new SqlConnection. This new SqlConnection doesn't have the event handler on it, and so won't respond to any of the printed messages.
Try replacing
$SqlConnection.add_InfoMessage($handler);
$SqlConnection.FireInfoMessageEventOnUserErrors = $true;
$SqlConnection = new-Object System.Data.SqlClient.SqlConnection("Server=XXX;DataBase=master;Integrated Security=SSPI")
with
$SqlConnection = new-Object System.Data.SqlClient.SqlConnection("Server=XXX;DataBase=master;Integrated Security=SSPI")
$SqlConnection.add_InfoMessage($handler);
$SqlConnection.FireInfoMessageEventOnUserErrors = $true;
How can I copy a number of tables, plus the data they contain, from a live SQL Server database?
I want to get some data for basic testing on my machine.
I was originally going to do a simple backup of the database, until I noticed that it was over 100GB in size. However the tables I'm after are only a number of the smaller ones. I then tried export to Excel, but hit the 65K limit (and we don't have Excel 2007)
You can try Exporting Data by Using the SQL Server Import and Export Wizard
Here is MSDN video
you can export it as Flat file
In Management Studio, select the database, right-click and select Tasks->Export Data. There you will see options to export to different kinds of formats including CSV.
You can also run your query from the Query window and save the results to CSV.
Can't you use the Export Data wizard from your live server to your testing machine? Or use bcp? Or even use a simple PowerShell script?
$Server = "MyServer"
$ServerInstance = "$Server\MyInstance"
$database = "MyDatabase"
$BackupFile = "c:\MyBackupFile.sql"
$tables = #('TableBlah','TableBluh','TableBloh')
$server = New-Object (
'Microsoft.SqlServer.Management.Smo.Server') $ServerInstance
$scripter = New-Object ('Microsoft.SqlServer.Management.Smo.Scripter') $server
$scripter.Options.SchemaQualify = $false
$scripter.Options.ScriptSchema = $false
$scripter.Options.ScriptData = $true
$scripter.Options.NoCommandTerminator = $true
$scripter.Options.ToFileOnly = $true
$scripter.Options.FileName = $BackupFile
$ServerUrn=$server.Urn
$UrnsToScript = New-Object Microsoft.SqlServer.Management.Smo.UrnCollection
foreach ($t in $tables)
{
# Could use bcp here for dumping big tables (like archives)
# $ret = (bcp.exe "$database..$t" out `"$ConfigBackupDir\$t.bcp`"
# -S $ServerInstance -U sa -P $SAPWD -n)
$Urn = "$ServerUrn/Database[#Name='" +
$database + "']/Table[#Name='" + $t + "' and #Schema='dbo']"
$UrnsToScript.Add($Urn)
}
$scripter.EnumScript($UrnsToScript)