What is the best way to copy a production database to multiple test databases, while maintaining the integrity of fully qualified names?
Currently to refresh a test environment we restore the test database from the production database. Then, we script all of the stored procedures/views/etc. and do a find/replace on all of the database references to point at the test objects. After we have all of the references correct, we alter them.
For example, after the database is copied from production, we'll have a stored procedure like so:
alter procedure dbo.SomeProcedure
as
select SomeColumn
from DB.dbo.SomeTable
join Validation.dbo.AnotherTable on SomId = AnoId
For the test database, it needs to be:
alter procedure dbo.SomeProcedure
as
select SomeColumn
from DBQA1.dbo.SomeTable
join ValidationQA1.dbo.AnotherTable on SomId = AnoId
Each test database has views/stored procedures/functions can reference up to 30 different other test databases, so the "find/replace" process is very time consuming and is prone to a lot of errors.
What is the best way to restore these test environments?
We are using SQL Server 2008R2.
Assuming that the different environments are on different SQL Servers (or at least on different Instances), then would recommend that you keep the database names on all environments exactly the same. Use permissions (e.g. Integrated security) to ensure that only the correct environment systems and users access the appropriate environment databases.
However, if you do need to keep different database names for different environments (e.g. all environments on the same SQL instance), you could look at using sqlcmd with the -v switch to parameterize the database name.
Your change scripts would then need to be rewritten like so:
alter procedure dbo.SomeProcedure
as
select SomeColumn
from [$(InternetSecurity)].dbo.SomeTable
join [$(Validation)].dbo.AnotherTable on SomId = AnoId
And then you could write batch files to pass the correct parameter values to sqlcmd.
Alternatively, you could use a .dbproj project in Visual Studio to setup multiple configurations to provide different values for each environment, and generate scripts / publish from Visual Studio.
Also, AFAIK SQL Synonyms are't really going to help here. You would need to replace the 3 part table names in all procs and functions with synonyms, which could confuse the issue as it doesn't make it clear whether the table is local or external.
As far as I know,There is no other simpler way than replacing the database names in the script here.
Related
I’m seriously confused about how flyway generally works to maintain the db as a code. Suppose I have the following V0 script:
Create table student(
Name varchar(25)
)
That would be my initial db. Now, suppose I want to add a new column, why am I being forced to do a V1 script like this one?
Alter table student add column surname varchar(25)
What I’d like to do would be to simply update the v0 script like this:
Create table student(
Name varchar(25),
Surname varchar(25)
)
Then the tool, by comparing the actual db, should be able to understand that a new column should be created!
This is as other code (java, javasctipt,..) tools work and the same I would it like to be for db as a code tools.
So my question is: is there a way to achieve this behavior without dropping/recreating the db?
I tagged this question with flyway and liquibase tools but feel free to suggest other tools that would fit my needs.
Whatever way you develop the database,there is no way to achieve this behavior without dropping/recreating the db, because the CREATE TABLE statement assumes that the table that you specify isn't already there. You can't use a CREATE OR ALTER statement because these aren't supported for tables even where the RDBMS that you use supports that syntax.
In the early stages of a database project, you can work very much quicker with a build script that you use to create a database with tables, views and so on. You can then insert some data, try it out, run a few tests, maybe and then tear it down. Flyway community supports this: you just have a single migration script starting from an empty database that you repeatedly 'clean' and 'migrate', until you reach your first version. Flyway takes care of the 'Tear-down' process. and give you a fresh start, by wiping your configured schemas completely clean.
Flyway Teams supports a special type of migration, the 'repeatable' that allows you to use for migrations SQL files that you can alter. However, you would need to add logic that deletes the table if it already exists before it executes your CREATE TABLE statement. It avoids having to 'Flyway clean', but it is a lot of extra work. It also means that you lose the whole advantage of a version representing an exact state of a database.
At some point, you are going to use migrations because you're likely to have copies of the database to keep up-to-date. Whatever tool you use to update a development or producton database, you are going to need to use a migration for this because of the existing data in tables.
Flyway Enterprise supports the automatic generation of a migration, if you are using Oracle or SQL Server. SQL Compare is provided to compare two versions of a database and produce a migration script from one version to the next. This allows you to use a build script as you suggest, compare it with the current version of the database, and generate a migration script to get from the one to the other.
I have a requirement that a user can have multiple environments to make experiments and test how good their modifications are; after the users are satisfied with the modifications they've done to the data in the working environment, these modifications can be (partially or completely) copied to another environment; these environments can be created as empty or as copies of other environments; right now we are using SQL Azure and our current (not implemented) approach is creating each environment as a different SQL schema in the same database using the statement
CREATE SCHEMA
till now in POCs this is working really good for us. But what i don't like of this approach is that creating a new schema involves executing several scripts to create the tables and the SPs in the new schema, so as we create or update the default schema objects, we also need to update the scripts that create the schema, also that when the schema is created we need to bulk copy the data from the original schema using another script, so considering the size of the client's data this process sometimes it cannot be not as fast as I would like, and also maintaining the SQL code to create environments is not that good for the team.
so my question is, is there any way to duplicate an entire dbo schema with a different name using T-SQL Statements?, i know this can be done manually using SQL Server Management Studio and the generate scripts option, but this must happen automatically because the users can create a new environment at any time, i already checked the documentation for
ALTER SCHEMA TargetSchema
TRANSFER SourceSchema.TableName;
but this just changes the database object schema, it does not create an actual copy of the object.
EDIT:
I am not trying to create different databases for dev, qa and production, I already have them; what I want to achieve is create a web app with multiple environments, each environment is a sandbox for the final user to make experiments, imagine it is like creating a draft before making this data available for the general public, so when the users are satisfied with the modifications they can move this data to the public environment and then when it is moved, it is available for others to see it
You can use the CREATE DATABASE ... AS COPY OF transact-sql statement to create copies of your Production database that can be used as QA, testing and development databases.
CREATE DATABASE db_copy
AS COPY OF ozabzw7545.db_original ( SERVICE_OBJECTIVE = 'P2') ;
Here ozabzw7545 is the name of Azure SQL Database server.
The following is the full syntax + additional information that is specific to Azure SQL database: CREATE DATABASE (Azure SQL Database)
Additional Information for Copy an Azure SQL database
You also can use PowerShell:
New-AzureRmSqlDatabaseCopy -ResourceGroupName "myResourceGroup" `
-ServerName $sourceserver `
-DatabaseName "MySampleDatabase" `
-CopyResourceGroupName "myResourceGroup" `
-CopyServerName $targetserver `
-CopyDatabaseName "CopyOfMySampleDatabase"
in my company, we have several environments with MS SQL database servers (SQL 2008 R2, SQL 2014). For the sake of simplicity, let us consider just a TEST environment and a PROD environment and two sql servers in each. Let the servers be called srTest1, srTest2, srProd1, srProd2 and each be running a default MS SQL Server instance. We work with multiple databases, say DataDb, ReportDb, DWHDb.
We want to keep the same source code in T-SQL for both TEST and PROD, but the problem is the architecture or distribution of the above mentioned databases in each environment:
TEST:
srTest1 - DataDb
srTest2 - DWHDb, ReportDb
PROD:
srProd1 - DataDb, ReportDb
srProd2 - DWHDb
Now, say, in ReportDb, we write stored procedures with many SELECTs referencing tables and other objects in DataDb and DWHDb. In order to have source code as universal as possible, we decided to create linked servers for each database on each db server in each environment and name them with respect to the database they're created for. Therefore, there'll be these linked servers:
lnkDataDb, lnkReportDb and lnkDWHDb on srTest1,
lnkDataDb, lnkReportDb and lnkDWHDb on srTest2,
lnkDataDb, lnkReportDb and lnkDWHDb on srProd1,
lnkDataDb, lnkReportDb and lnkDWHDb on srProd2.
And we'll adjust the source in the stored procs accordingly. For instance:
Instead of
SELECT * FROM DataDb.dbo.Contact
We'll write
SELECT * FROM lnkDataDb.DataDb.dbo.Contact
The example above is reasonable for a situation where the database from which you execute the query (ReportDb) lies on a different server than that with the referenced table (DataDb). Which is the case for the TEST environment. But not so in PROD. It is performance I'm here concerned about. The SQL Server will treat that SELECT as a "remote query" no matter whether, in fact, it is a reference to a local object or not.
Now, it comes the most important part:
If you check these 3 queries for their actual execution plans, you'll see an interesting thing:
(1) SELECT * FROM DataDb.dbo.Contact
(2) SELECT * FROM srProd1.DataDb.dbo.Contact
(3) SELECT * FROM lnkDataDb.DataDb.dbo.Contact
The first two (query #1 and #2) have the same execution plan (the fastest possible) even if you use the four-part name manner of referencing the table Contact in #2.
The last query has a different plan (remote query, thus slower).
The question is:
Can you somehow create a linked server to self (the same sql server instance, the default instance actually) as an "alias" to the name of the host (srProd1) in order for the SQL server to be forced to understand it as local and not issue "remote execution" plans?
Thanks a lot for any hints
Pavel
Recently I found a workaround which seems to solve this kind of issues more efficiently and more elegantly than the solution with self-pointing linked servers.
If you work (making reports, for example) with multiple databases on multiple SQL servers and the physical distribution of the databases on the servers is a challenge since it may differ from one environment to another (e.g. TEST vs PROD), I suggest this:
Use three-part db object names whenever possible. If the objects are local, then execution plans are also local, and thus effective.
Example:
SELECT * FROM DataDb.dbo.Contact
If you happen to run the above query from within a different SQL server instance (residing on a different physical machine, for example, but this not necessarily, the other SQL server instance could be installed even on the same machine), briefly if you're about to use a four-part name:
SELECT * FROM lnkDataDb.DataDb.dbo.Contact
Then you can circumvent that using the following trick:
Let's assume lnkDataDb points to srTest2 and you're executing your queries from srTest1. Now, you'll create a "fake" database DataDb on your local server (srTest1). This fake DataDb shall contain no real db objects (no tables, no views, no stored procedures, no UDFs etc.). There shall only be synonyms defined in it. (And there also shall be the same schemas in it as those in the real DataDb on srTest2). These synonyms shall be named exactly the same way as their real db-object counterparts in DataDb on srTest2. Example:
-- To be executed on srTest1.
EXEC sp_addlinkedserver
#server = N'lnkDataDb',
#srvproduct = N'',
#provider = N'SQLNCLI',
#datasrc = N'srTest2'
;
GO
CREATE DATABASE [DataDb];
GO
USE [DataDb];
GO
CREATE SYNONYM dbo.Contact FOR lnkDataDb.DataDb.dbo.Contact;
GO
Now, if you want to SELECT rows from the table dbo.Contact residing in the database DataDb on srTest2 and you're executing your query from srTest1, you'll use a simple three-part table name:
SELECT * FROM DataDb.dbo.Contact
Of course, on srTest1, this is not a table, that's just a synonym referencing the same-named table on srTest2. However, that's the trick, you use the same query syntax as if you were executing it on srTest2 where the real db object resides.
There are disadvantages of this approach:
On the local server, at the beginning, there must not be a database
with the same name as the remote one. Because you're about to create
a "fake" database with that name to reflect the names of remote
db objects.
You're creating one database that is almost empty, thus
increasing the mess of various databases residing on your local
SQL server. This might provoke reluctance of your database admin
if they prefer having as few databases as possible.
If you're developing your T-SQL scripts in SQL Server Management
Studio, for example, using synonyms cuts you off from the convenience
of the IntelliSense feature.
Advantages outweigh the above-mentioned disadvantages, though:
Your scripts work in any environment (DEV, TEST, PROD) without
the need to change any part of the source code.
If the other database you're querying data from resides on the same
SQL server instance as your script, you also use the three-part name
convention and your SQL server evaluates the query in execution plan
as local which is OK. (This is what the original question of this
post was searching to solve.)
If the other database you're querying data from resides on another
SQL server instance, you still use a "local syntax manner" of a SQL
query (with the synonym) which, only at runtime, evaluates in
a remote execution plan. Which is also fine because the db object
actually is remote.
To summarize
The query executes as local if the referenced object is local, the query executes as remote if the referenced object is remote, but the T-SQL script is always the same. You don't have to change a letter in it.
I have a database from a 3rd party. They supply a tool to update the database data weekly. The tool is pretty old and uses ODBC. Updates can either be incremental or can delete all database data then recreate the data. The update can take several hours. In order to have high availability, it was suggested to have 2 SQL databases, and store a "active database" setting in another database to determine which of the two databases applications should use (while the other could be being updated).
One issue we are running into is: How to do reference the active database in stored procedures in other databases?
Is this the right approach? Is there a simple, perhaps-infrastructure-based approach? (Should this be posted on ServerFault?)
Note: Databases are read-only besides the update tool.
If the databases are on different servers, you can create an alias for the server which will redirect to the other server in SQL Server Configuration Manager. Under SQLNative Client 10.0 Configuration (or 9.0 if you're in SQL Server 2005) you can add a new alias.
Otherwise, you can always rename the databases using sp_dbrename so thata your client applications are always using database1 while you are updating database2.
If you want to use different databases inside a stored procedure you either need to:
Duplicate all the calls. Ugly. You would end with a lot of:
if #firstDatabase=1
select * from database1..ExampleTable where ...
else
select * from database2..ExampleTable where ...
Use dynamic queries. Less ugly:
set #sqlQuery='select * from '+#currentDatabase+'..ExampleTable where...'
exec sp_executesql #sqlQuery
I admit that neither solution is perfect...
I'd take the approach of having the stored procedures in both databases with some sort of automatic trigger to update the stored procedures in the other database if a stored procedure is changed.
how to link two different database in same SQL Server instance
and send queries between them
use like below.
DB1.dbo.TableFromDB1
DB2.dbo.TableFromDB2
DB - database
Look at using synonyms "CREATE SYNONYM".
You can access the databases directly with a full path. But, that code will break if the database is ever renamed or changed.
Using a synonym, the code can remain unchanged; when the database moves, just update the synonym.
This can be useful when you have a test and production environment. The code does not have to change just because you move it from test to production and the database names do not have to be identical.