SQL Server Bulk Copy dynamic script - sql-server

I've quite a bunch of CSV files I would like to import into a SQL Server database. Each of these files contain over 50 million rows which I need to import into a single table called ControlExperimentTable.
A total of 500 of these files will be sent to me over the next few weeks to import into the database, so I wrote a script to automate the process it is obviously long and cumbersome.
In a nutshell, the process works as follows -- CSV files are copied into a repository folder for processing. All files have the same prefix and a unique suffix e.g ExportData0001.txt, ExportData0002.txt, ExportData0003.txt, ExportData0004.txt, etc.
The script sequentially processes each file, importing its contents into the SQL Server database. After all rows have been imported, the processed file is then moved to an archive folder, a full database backup is taken and the process moves on to the next file.
The following is the code I'm using to accomplish the task:
--Int variable declaration section.
DECLARE #totalFilesToProcess INT = 500 /*The number of files in the repository.*/
DECLARE #count INT = 0
--VarChar variable declaration section.
DECLARE #sqlBulkInsertCommand VARCHAR(255)
DECLARE #sqlMoveCommand VARCHAR(255)
DECLARE #sourceFilename VARCHAR(255)
DECLARE #errorFilename VARCHAR(255)
DECLARE #suffix VARCHAR(4)
--Ensure that the xp_cmdshell server configuration option is enabled.
EXEC master.dbo.sp_configure 'show advanced options', 1
RECONFIGURE
EXEC master.dbo.sp_configure 'xp_cmdshell', 1
RECONFIGURE
--Change to the target database instance.
USE [DataCollectionTable]
--Set the script to loop the number of times as there are as may files to import
WHILE (#count <= #totalFilesToProcess)
BEGIN
--Set variables.
SET #count = #count + 1
SET #suffix = RIGHT('0000' + CAST(#count AS VARCHAR(4)), 4)
SET #errorFilename = FORMATMESSAGE('E:\SharedDocs\ControlExportData%s.csv', #suffix)
SET #sourceFilename = FORMATMESSAGE('E:\SharedDocs\ControlExportData%s.txt', #suffix)
--COMMAND CONSTRUCT: Insert data from flat file into the ControlExperimentTable table.
SET #sqlBulkInsertCommand = FORMATMESSAGE('BULK INSERT ControlExperimentTable FROM ''%s'' WITH (FIRSTROW = 2, FIELDTERMINATOR = '','', ROWTERMINATOR = ''\n'', ERRORFILE = ''%s'', TABLOCK)', #sourceFilename, #errorFilename)
--COMMAND CONSTRUCT: Move the source file to the archive folder.
SET #sqlMoveCommand = FORMATMESSAGE('MOVE E:\SharedDocs\ControlExportData%s.txt E:\SharedDocs\Archive\ControlExportData%s.txt', #suffix, #suffix)
--Display record count before every update.
SELECT sys.sysindexes.rows FROM sys.sysindexes INNER JOIN sys.sysobjects
ON sys.sysobjects.id=sys.sysindexes.id
WHERE sys.sysindexes.first IS NOT NULL AND sys.sysobjects.name = 'ControlExperimentTable'
--Execute the BULK INSERT command.
EXEC #sqlBulkInsertCommand
--Execute the source file MOVE command.
EXEC master.dbo.xp_cmdshell #sqlMoveCommand
--Backup database after each commitment, maintaining the current and previous copies.
IF (#count % 2) = 0
BEGIN
BACKUP DATABASE [ExperimentDataCollection] TO DISK = N'E:\Backups\ExperimentDataCollectionBackup-01.bak' WITH NOFORMAT, INIT, NAME = N'ExperimentDataCollection-Full Database Backup', SKIP, NOREWIND, NOUNLOAD, STATS = 10
END ELSE
BEGIN
BACKUP DATABASE [ExperimentDataCollection] TO DISK = N'E:\Backups\ExperimentDataCollectionBackup-02.bak' WITH NOFORMAT, INIT, NAME = N'ExperimentDataCollection-Full Database Backup', SKIP, NOREWIND, NOUNLOAD, STATS = 10
END
END
After running this code, I'm getting the following error:
Configuration option 'show advanced options' changed from 1 to 1. Run the RECONFIGURE statement to install.
Configuration option 'xp_cmdshell' changed from 1 to 1. Run the RECONFIGURE statement to install.
Msg 911, Level 16, State 4, Line 48
Database 'BULK INSERT ControlExperimentTable FROM 'E:\SharedDocs\ExportData0001' does not exist. Make sure that the name is entered correctly.
Completion time: 2021-02-18T18:06:15.0003210+02:00
Mind you, everything is where they are supposed to be and correctly specified in the code. So much so that manually running the following code, which is exactly what I'm trying to parse works without a hassle.
USE [ExperimentDataCollection]
BULK INSERT [ExperimentDataCollection].[dbo].[ControlExperimentTable]
FROM 'E:\SharedDocs\ControlExportData0001.txt'
WITH (FIRSTROW = 2,
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n',
ERRORFILE = 'E:\SharedDocs\ControlExportData0001.csv',
TABLOCK)
Where is it I'm going wrong?

When executing dynamic sql, you need to wrap the variable in parentheses. So change this:
EXEC #sqlBulkInsertCommand
to this:
EXEC(#sqlBulkInsertCommand)

Related

Create a single table by importing all CSV files in a folder?

I have around 30-40 CSV files in a folder. For example, suppose folder 'Florida' has customer information from different cities of state Florida. each CSV file has customer information of one city. Now I want to create a table in SQL Server by importing all the CSV files from that folder to create a table for all customers in Florida. I wanted to know if there is any way I could perform this action for all CSV files at once. I am using SQL Server Management Studio (SSMS).
All the CSV files have same column names.
I am doing the following for one CSV file:
CREATE TABLE sales.cust (
Full_name VARCHAR (100) NOT NULL,
phone VARCHAR(50),
city VARCHAR (50) NOT NULL,
state VARCHAR (50) NOT NULL,
);
BULK INSERT sales.cust
FROM 'C:\Users..............\cust1.csv'
WITH
(
FIRSTROW = 2,
FIELDTERMINATOR = ',', --CSV field delimiter
ROWTERMINATOR = '\n', --Use to shift the control to next row
ERRORFILE = 'C:\Users\..............\cust1ErrorRows.csv',
TABLOCK
)
Suggestion to use command prompt only because of limited tools.
I thought of another solution you can use that could help you out and make it so you only have to import one file.
Create your table:
CREATE TABLE sales.cust (
Full_name VARCHAR (100) NOT NULL,
phone VARCHAR(50),
city VARCHAR (50) NOT NULL,
state VARCHAR (50) NOT NULL,
);
Using command Prompt do the following:
a. Navigate to your directory using cd "C:\Users..............\"
b. Copy the files into one giant file using:
copy *.csv combined.csv
Import that file using GUI in SSMS
Deal with the headers
delete from sales.cust where full_name = 'Full_name' and phone = 'phone'
You can only do this because all columns are varchar.
Here is one route to get all the files into a table---
-- from Rigel and froadie # https://stackoverflow.com/questions/26096057/how-can-i-loop-through-all-the-files-in-a-folder-using-tsql
-- 1.Allow for SQL to use cmd shell
EXEC sp_configure 'show advanced options', 1 -- To allow advanced options to be changed.
RECONFIGURE -- To update the currently configured value for advanced options.
EXEC sp_configure 'xp_cmdshell', 1 -- To enable the feature.
RECONFIGURE -- To update the currently configured value for this feature.
-- 2.Get all FileNames into a temp table
--for repeatability when testing in SMSS, delete any prior table
IF OBJECT_ID('tempdb..#tmp') IS NOT NULL DROP TABLE #tmp
GO
CREATE TABLE #tmp(csvFileName VARCHAR(100));
INSERT INTO #tmp
EXEC xp_cmdshell 'dir /B "C:\\Users..............\\*.csv"';
-- from Chompy #https://bytes.com/topic/sql-server/answers/777399-bulk-insert-dynamic-errorfile-filename
-- 3.Create sql prototype of the Dynamic sql
---- with CSV field delimiter=',' and CSV shift the control to next row='\n'
DECLARE #sqlPrototype nvarchar(500)
SET #sqlPrototype = N'BULK INSERT sales.cust
FROM ''C:\\Users..............\\xxxx''
WITH ( FIRSTROW = 2,
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''\n'',
ERRORFILE = ''C:\\Users..............\\xxxx_ErrorRows.txt'',
TABLOCK)'
-- 4.Loop through all of the files
Declare #fileName varchar(100)
While (Select Count(*) From #tmp where csvFileName is not null) > 0
Begin
Select Top 1 #fileName = csvFileName From #tmp
-- 5.Replace real filename into prototype
PRINT(#filename)
DECLARE #sqlstmt nvarchar(500)
Set #sqlstmt = replace(#sqlPrototype, 'xxxx', #filename)
--print(#sqlstmt)
-- 6.Execute the resulting sql
EXEC sp_executesql #sqlstmt;
-- 4A.Remove FileName that was just processed
Delete from #tmp Where csvFileName = #FileName
End
Caution--If ErrorFile exists, then BulkInsert will fail.

Schedule importing flat files with different names into SQL server 2014

As I am a beginner in SQL Server and my scripting is not very polished yet. I need suggestions on the below issue.
I receive files from a remote server to my machine (around 700/day) as follows :
ABCD.100.1601310200
ABCD.101.1601310210
ABCD.102.1601310215
Naming Convention:
Here the first part 'ABCD' remains the same, middle part is a sequence id which is in incremental order for every file. The last part is time stamp.
File structure
The file does not have any specific extension but can be opened with notepad/excel. Therefore can be called as flat file. Each files consist of 95 columns and 20000 rows fixed with some garbage value on top 4 and bottom 4 rows of column 1.
Now, I need to make a database in SQL server where I can import data from these flat files using a scheduler. Suggestion needed.
There are probably other ways of doing this, but this is one way:
Create a format file for your tables. You only need to create it once. Use this file in the import script in step 2.
Create an import script based on OPENROWSET(BULK '<file_name>', FORMATFILE='<format_file>'
Schedule the script from step 2 in SQL Server to run against the database you want the data imported in
Create the format file
This creates a format file to be used in the next step. The following script creates a format file in C:\Temp\imp.fmt based on an existing table (replace TEST_TT with the database you are importing to). This creates such a format file with a , as field seperator. If the files have tab as seperator, remove the -t, switch.
DECLARE #cmd VARCHAR(8000);
SET #cmd='BCP TEST_TT.dbo.[ABCD.100.1601310200] format nul -f "C:\Temp\imp.fmt" -c -t, -T -S ' + (SELECT ##SERVERNAME);
EXEC master..xp_cmdshell #cmd;
Before executing this you will to reconfigure SQL Server to allow the xp_cmdshell stored procedure. You only need to do this once.
EXEC sp_configure 'show advanced options', 1
GO
RECONFIGURE
GO
EXEC sp_configure 'xp_cmdshell', 1
GO
RECONFIGURE
GO
import script
This script assumes:
The files need to be imported to separate tables
The files are located in C:\Temp
The format file is C:\Temp\imp.fmt (generated in the previous step)
SET NOCOUNT ON;
DECLARE #store_path VARCHAR(256)='C:\Temp';
DECLARE #files TABLE(fn NVARCHAR(256));
DECLARE #list_cmd VARCHAR(256)='DIR ' + #store_path + '\ABCD.* /B';
INSERT INTO #files EXEC master..xp_cmdshell #list_cmd;
DECLARE #fullcmd NVARCHAR(MAX);
SET #fullcmd=(
SELECT
'IF OBJECT_ID('''+QUOTENAME(fn)+''',''U'') IS NOT NULL DROP TABLE '+QUOTENAME(fn)+';'+
'SELECT * INTO '+QUOTENAME(fn)+' '+
'FROM OPENROWSET(BULK '''+#store_path+'\'+fn+''',FORMATFILE=''C:\Temp\imp.fmt'') AS tt;'
FROM
#files
WHERE
fn IS NOT NULL
FOR XML PATH('')
);
EXEC sp_executesql #fullcmd;

Switching from one database to another within the same script

I would like to know how I can switch from one database to another within the same script. I have a script that reads the header information from a SQL Server .BAK file and loads the information into a test database. Once the information is in the temp table (Test database) I run the following script to get the database name.
This part works fine.
INSERT INTO #HeaderInfo EXEC('RESTORE HEADERONLY
FROM DISK = N''I:\TEST\database.bak''
WITH NOUNLOAD')
DECLARE #databasename varchar(128);
SET #databasename = (SELECT DatabaseName FROM #HeaderInfo);
The problem is when I try to run the following script nothing happens. The new database is never selected and the script is still on the test database.
EXEC ('USE '+ #databasename)
The goal is switch to the new database (USE NewDatabase) so that the other part of my script (DBCC CHECKDB) can run. This script checks the integrity of the database and saves the results to a temp table.
What am I doing wrong?
You can't expect a use statement to work in this fashion using dynamic SQL. Dynamic SQL is run in its own context, so as soon as it has executed, you're back to your original context. This means that you'd have to include your SQL statements in the same dynamic SQL execution, such as:
declare #db sysname = 'tempdb';
exec ('use ' + #db + '; dbcc checkdb;')
You can alternatively use fully qualified names for your DB objects and specify the database name in your dbcc command, even with a variable, as in:
declare #db sysname = 'tempdb';
dbcc checkdb (#db);
You can't do this because Exec scope is limited to dynamic query. When exec ends context is returned to original state. But context changes in Exec itself. So you should do your thing in one big dynamic statement like:
DECLARE #str NVARCHAR(MAX)
SET #str = 'select * from table1
USE DatabaseName
select * from table2'
EXEC (#str)

How to rename the Physical Database Files

I have used tsql to detach a database like this:
EXEC sp_detach_db #dbname = 'my_db'
I then made use of PHP to rename the physical files. I was able to rename the mdf file but not the ldf file! I even tried a dos command REN but that didn't work for the ldf file either!
I wanted to ask, is there something special about the physical log files that allow it not to be renamed?
Is there a better way of doing this?
Thanks all
Detach the Database, Rename the files, Attach it again.
Backup the original database
Drop the original database
Restore the original database from the backup, but with different name; the files of the restored database will be also automatically named taking into account new database name.
The "ALTER DATABASE (your database) MODIFY FILE" command will only rename the logical names. This post shows how to use xp_cmdshell to also rename the physical files: http://www.mssqltips.com/sqlservertip/1891/best-practice-for-renaming-a-sql-server-database/
Please note the following:
1. xp_cmdshell will be executed under the user which the SQL Server process runs as, and might not have the file system permissions required to rename the database files
2. For security reasons, remember to disable xp_xmdshell
The following is an example of how the renaming can be done based on the mentioned blog post. It will replace the database MyDB with the database NewMyDB. The original MyDB (renamed to MyDB_OLD) will be left detached.
-- Enable xp_cmdshell:
sp_configure 'show advanced options', 1
RECONFIGURE WITH OVERRIDE
GO
sp_configure 'xp_cmdshell', 1
RECONFIGURE WITH OVERRIDE
GO
-- Get physical file names:
declare #MyDBOriginalFileName nvarchar(300) = (select physical_name FROM sys.master_files where name = 'MyDB')
declare #MyDBLogOriginalFileName nvarchar(300) = (select physical_name FROM sys.master_files where name = 'MyDB_log')
declare #NewMyDBOriginalFileName nvarchar(300) = (select physical_name FROM sys.master_files where name = 'NewMyDB')
declare #NewMyDBLogOriginalFileName nvarchar(300) = (select physical_name FROM sys.master_files where name = 'NewMyDB_log')
declare #Command nvarchar(500)
declare #Sql nvarchar(2000)
IF (EXISTS (select * from sys.databases where name = 'NewMyDB')
AND EXISTS (select * from sys.databases where name = 'MyDB'))
BEGIN
USE master
ALTER DATABASE MyDB SET SINGLE_USER WITH ROLLBACK IMMEDIATE
ALTER DATABASE NewMyDB SET SINGLE_USER WITH ROLLBACK IMMEDIATE
-- Set new database name
ALTER DATABASE MyDB MODIFY NAME = MyDB_OLD
ALTER DATABASE NewMyDB MODIFY NAME = MyDB
-- Update logical names
ALTER DATABASE MyDB_OLD MODIFY FILE (NAME=N'MyDB', NEWNAME=N'MyDB_OLD')
ALTER DATABASE [MyDB] MODIFY FILE (NAME=N'NewMyDB', NEWNAME=N'MyDB')
EXEC master.dbo.sp_detach_db #dbname = N'MyDB_Old'
EXEC master.dbo.sp_detach_db #dbname = N'MyDB'
-- Rename physical files
SET #Command = 'RENAME "' + #MyDBOriginalFileName + '" "MyDB_OLD.mdf"'; PRINT #Command
EXEC xp_cmdshell #Command
SET #Command = 'RENAME "' + #MyDBLogOriginalFileName + '" "MyDB_OLD_log.mdf"'; PRINT #Command
EXEC xp_cmdshell #Command
SET #Command = 'RENAME "' + #NewMyDBOriginalFileName + '" "MyDB.mdf"'; PRINT #Command
EXEC xp_cmdshell #Command
SET #Command = 'RENAME "' + #NewMyDBLogOriginalFileName + '" "MyDB_log.mdf"'; PRINT #Command
EXEC xp_cmdshell #Command
-- Attach with new file names
declare #NewMyDBFileNameAfterRename nvarchar(300) = replace(#NewMyDBOriginalFileName, 'NewMyDB', 'MyDB')
declare #NewMyDBLogFileNameAfterRename nvarchar(300) = replace(#NewMyDBOriginalFileName, 'NewMyDB_log', 'MyDB_log')
SET #Sql = 'CREATE DATABASE MyDB ON ( FILENAME = ''' + #NewMyDBFileNameAfterRename + '''), ( FILENAME = ''' + #NewMyDBLogFileNameAfterRename + ''') FOR ATTACH'
PRINT #Sql
EXEC (#Sql)
ALTER DATABASE MyDB SET MULTI_USER
END
-- Disable xp_cmdshell for security reasons:
GO
sp_configure 'show advanced options', 1
RECONFIGURE WITH OVERRIDE
GO
sp_configure 'xp_cmdshell', 0
RECONFIGURE WITH OVERRIDE
GO
You can do it using an ALTER DATABASE statement - like this:
ALTER DATABASE database_name
MODIFY FILE ( NAME = logical_file_name,
FILENAME = ' new_path/os_file_name_with_extension ' )
You need to modify each file separately, e.g. if you have multiple data files, you need to modify each of those.
For details, see the Technet documentation on this topic.
The simplest way to rename SQL server physical database files is:
Open and connect to the SQL server where the database you wanted to rename is located.
Execute the following script in the query window in order to change the physical and logical names. Remember to replace all the "OldDatabaseName" with the new name of the database ("NewDatabaseName") you want to change its name to. Replace all NewDatabaseName with the new name you want to set for your database
use OldDatabaseName
ALTER DATABASE OldDabaseName MODIFY FILE (NAME='OldDatabaseName', FILENAME='C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\NewDatabaseName.mdf');
ALTER DATABASE OldDatabaseName MODIFY FILE (NAME='OldDatabaseName_log', FILENAME='C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\NewDatabaseName_log.ldf');
ALTER DATABASE OldDatabaseName MODIFY FILE (NAME = OldDatabaseName, NEWNAME = NewDatabaseName);
ALTER DATABASE OldDatabaseName MODIFY FILE (NAME = OldDatabaseName_log, NEWNAME = NewDatabaseName_log);
And then Right click on the OldDatabaseName, select Tasks and then choose Take Offline
Go to the location (C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA\...) where the physical files are located and rename them to the NewDatabaseName you specified in number 2. Remember to check the absolute path of these files to be used on your computer.
Go back to Microsoft SQL Server Management Studio. Right click on the OldDatabaseName, select Tasks and then choose Bring Online.
Finally, go ahead and rename your OldDatabaseName to the NewDatabaseName. You are done :-)
Detach (right click on database)
Rename both files (ldf and mdf) :
C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\MSSQL\DATA
Attach (right click on "Databases" top folder)

How can I clone an SQL Server database on the same server in SQL Server 2008 Express?

I have an MS SQL Server 2008 Express system which contains a database that I would like to 'copy and rename' (for testing purposes) but I am unaware of a simple way to achieve this.
I notice that in the R2 version of SQL Server there is a copy database wizard, but sadly I can't upgrade.
The database in question is around a gig.
I attempted to restore a backup of the database I want to copy into a new database, but with no luck.
Install Microsoft SQL Management Studio, which you can download for free from Microsoft's website:
Version 2008
Microsoft SQL Management Studio 2008 is part of SQL Server 2008 Express with Advanced Services
Version 2012
Click download button and check ENU\x64\SQLManagementStudio_x64_ENU.exe
Version 2014
Click download button and check MgmtStudio 64BIT\SQLManagementStudio_x64_ENU.exe
Open Microsoft SQL Management Studio.
Backup original database to .BAK file (db -> Task -> Backup).
Create empty database with new name (clone). Note comments below as this is optional.
Click to clone database and open restore dialog (see image)
Select Device and add the backup file from step 3.
Change destination to test database
Change location of database files, it must be different from the original. You can type directly into text box, just add postfix. (NOTE: Order is important. Select checkbox, then change the filenames.)
Check WITH REPLACE and WITH KEEP_REPLICATION
Right-click the database to clone, click Tasks, click Copy Database.... Follow the wizard and you're done.
You could try to detach the database, copy the files to new names at a command prompt, then attach both DBs.
In SQL:
USE master;
GO
EXEC sp_detach_db
#dbname = N'OriginalDB';
GO
At Command prompt (I've simplified the file paths for the sake of this example):
copy c:\OriginalDB.mdf c:\NewDB.mdf
copy c:\OriginalDB.ldf c:\NewDB.ldf
In SQL again:
USE master;
GO
CREATE DATABASE OriginalDB
ON (FILENAME = 'C:\OriginalDB.mdf'),
(FILENAME = 'C:\OriginalDB.ldf')
FOR ATTACH;
GO
CREATE DATABASE NewDB
ON (FILENAME = 'C:\NewDB.mdf'),
(FILENAME = 'C:\NewDB.ldf')
FOR ATTACH;
GO
It turns out that I had attempted to restore from a backup incorrectly.
Initially I created a new database and then attempted to restore the backup here.
What I should have done, and what worked in the end, was to bring up the restore dialog and type the name of the new database in the destination field.
So, in short, restoring from a backup did the trick.
Thanks for all the feedback and suggestions guys
This is the script I use. A bit tricky but it works. Tested on SQL Server 2012.
DECLARE #backupPath nvarchar(400);
DECLARE #sourceDb nvarchar(50);
DECLARE #sourceDb_log nvarchar(50);
DECLARE #destDb nvarchar(50);
DECLARE #destMdf nvarchar(100);
DECLARE #destLdf nvarchar(100);
DECLARE #sqlServerDbFolder nvarchar(100);
SET #sourceDb = 'db1'
SET #sourceDb_log = #sourceDb + '_log'
SET #backupPath = 'E:\DB SQL\MSSQL11.MSSQLSERVER\MSSQL\Backup\' + #sourceDb + '.bak' --ATTENTION: file must already exist and SQL Server must have access to it
SET #sqlServerDbFolder = 'E:\DB SQL\MSSQL11.MSSQLSERVER\MSSQL\DATA\'
SET #destDb = 'db2'
SET #destMdf = #sqlServerDbFolder + #destDb + '.mdf'
SET #destLdf = #sqlServerDbFolder + #destDb + '_log' + '.ldf'
BACKUP DATABASE #sourceDb TO DISK = #backupPath
RESTORE DATABASE #destDb FROM DISK = #backupPath
WITH REPLACE,
MOVE #sourceDb TO #destMdf,
MOVE #sourceDb_log TO #destLdf
None of the solutions mentioned here worked for me - I am using SQL Server Management Studio 2014.
Instead I had to uncheck the "Take tail-log backup before restore" checkbox in the "Options" screen: in my version it is checked by default and prevents the Restore operation to be completed.
After unchecking it, the Restore operation proceeded without issues.
From SSMS :
1 - Backup original database to .BAK file (your_source_db -> Task -> Backup).
2 - Right clicking the "Databases" and 'Restore Database'
3 - Device > ... (button) > Add > select the your_source_db.bak
4 - In 'General' tab, in 'Destination' section, rename in 'Database' your_source_db to new_name_db
5 - In 'Files' tab, tick 'Relocate all files to folder',
Rename in 'Restore As' column the two lignes to keep consistency with new_name_db (.mdf, _log.ldf)
6 - In 'Options' tab, in 'Restore options' section, tick two fist options ('Overwrite...', 'Preserve...') and for 'Recovery state' : 'RESTORE WITH RECOVERY'
Make also sure that in 'Tail-Log backup' section options are unticked to avoid keeping source db in 'restoring state' !
Using MS SQL Server 2012, you need to perform 3 basic steps:
First, generate .sql file containing only the structure of the source DB
right click on the source DB and then Tasks then Generate Scripts
follow the wizard and save the .sql file locally
Second, replace the source DB with the destination one in the .sql file
Right click on the destination file, select New Query and Ctrl-H or (Edit - Find and replace - Quick replace)
Finally, populate with data
Right click on the destination DB, then select Tasks and Import Data
Data source drop down set to ".net framework data provider for SQL server" + set the connection string text field under DATA ex: Data Source=Mehdi\SQLEXPRESS;Initial Catalog=db_test;User ID=sa;Password=sqlrpwrd15
do the same with the destination
check the table you want to transfer or check box besides "source: ..." to check all of them
You are done.
If the database is not very large, you might look at the 'Script Database' commands in SQL Server Management Studio Express, which are in a context menu off the database item itself in the explorer.
You can choose what all to script; you want the objects and the data, of course. You will then save the entire script to a single file. Then you can use that file to re-create the database; just make sure the USE command at the top is set to the proper database.
In SQL Server 2008 R2, back-up the database as a file into a folder.
Then chose the restore option that appears in the "Database" folder.
In the wizard enter the new name that you want in the target database.
And choose restore frrom file and use the file you just created.
I jsut did it and it was very fast (my DB was small, but still)
Pablo.
The solution, based on this comment: https://stackoverflow.com/a/22409447/2399045 .
Just set settings: DB name, temp folder, db files folder.
And after run you will have the copy of DB with Name in "sourceDBName_yyyy-mm-dd" format.
-- Settings --
-- New DB name will have name = sourceDB_yyyy-mm-dd
declare #sourceDbName nvarchar(50) = 'MyDbName';
declare #tmpFolder nvarchar(50) = 'C:\Temp\'
declare #sqlServerDbFolder nvarchar(100) = 'C:\Databases\'
-- Execution --
declare #sourceDbFile nvarchar(50);
declare #sourceDbFileLog nvarchar(50);
declare #destinationDbName nvarchar(50) = #sourceDbName + '_' + (select convert(varchar(10),getdate(), 121))
declare #backupPath nvarchar(400) = #tmpFolder + #destinationDbName + '.bak'
declare #destMdf nvarchar(100) = #sqlServerDbFolder + #destinationDbName + '.mdf'
declare #destLdf nvarchar(100) = #sqlServerDbFolder + #destinationDbName + '_log' + '.ldf'
SET #sourceDbFile = (SELECT top 1 files.name
FROM sys.databases dbs
INNER JOIN sys.master_files files
ON dbs.database_id = files.database_id
WHERE dbs.name = #sourceDbName
AND files.[type] = 0)
SET #sourceDbFileLog = (SELECT top 1 files.name
FROM sys.databases dbs
INNER JOIN sys.master_files files
ON dbs.database_id = files.database_id
WHERE dbs.name = #sourceDbName
AND files.[type] = 1)
BACKUP DATABASE #sourceDbName TO DISK = #backupPath
RESTORE DATABASE #destinationDbName FROM DISK = #backupPath
WITH REPLACE,
MOVE #sourceDbFile TO #destMdf,
MOVE #sourceDbFileLog TO #destLdf
Another way that does the trick by using import/export wizard, first create an empty database, then choose the source which is your server with the source database, and then in the destination choose the same server with the destination database (using the empty database you created at first), then hit finish
It will create all tables and transfer all the data into the new database,
Script based on Joe answer (detach, copy files, attach both).
Run Managment Studio as Administrator account.
It's not necessary, but maybe access denied error on executing.
Configure sql server for execute xp_cmdshel
EXEC sp_configure 'show advanced options', 1
GO
RECONFIGURE
GO
EXEC sp_configure 'xp_cmdshell', 1
GO
RECONFIGURE
GO
Run script, but type your db names in #dbName and #copyDBName variables before.
USE master;
GO
DECLARE #dbName NVARCHAR(255) = 'Products'
DECLARE #copyDBName NVARCHAR(255) = 'Products_branch'
-- get DB files
CREATE TABLE ##DBFileNames([FileName] NVARCHAR(255))
EXEC('
INSERT INTO ##DBFileNames([FileName])
SELECT [filename] FROM ' + #dbName + '.sys.sysfiles')
-- drop connections
EXEC('ALTER DATABASE ' + #dbName + ' SET OFFLINE WITH ROLLBACK IMMEDIATE')
EXEC('ALTER DATABASE ' + #dbName + ' SET SINGLE_USER')
-- detach
EXEC('EXEC sp_detach_db #dbname = ''' + #dbName + '''')
-- copy files
DECLARE #filename NVARCHAR(255), #path NVARCHAR(255), #ext NVARCHAR(255), #copyFileName NVARCHAR(255), #command NVARCHAR(MAX) = ''
DECLARE
#oldAttachCommand NVARCHAR(MAX) =
'CREATE DATABASE ' + #dbName + ' ON ',
#newAttachCommand NVARCHAR(MAX) =
'CREATE DATABASE ' + #copyDBName + ' ON '
DECLARE curs CURSOR FOR
SELECT [filename] FROM ##DBFileNames
OPEN curs
FETCH NEXT FROM curs INTO #filename
WHILE ##FETCH_STATUS = 0
BEGIN
SET #path = REVERSE(RIGHT(REVERSE(#filename),(LEN(#filename)-CHARINDEX('\', REVERSE(#filename),1))+1))
SET #ext = RIGHT(#filename,4)
SET #copyFileName = #path + #copyDBName + #ext
SET #command = 'EXEC master..xp_cmdshell ''COPY "' + #filename + '" "' + #copyFileName + '"'''
PRINT #command
EXEC(#command);
SET #oldAttachCommand = #oldAttachCommand + '(FILENAME = "' + #filename + '"),'
SET #newAttachCommand = #newAttachCommand + '(FILENAME = "' + #copyFileName + '"),'
FETCH NEXT FROM curs INTO #filename
END
CLOSE curs
DEALLOCATE curs
-- attach
SET #oldAttachCommand = LEFT(#oldAttachCommand, LEN(#oldAttachCommand) - 1) + ' FOR ATTACH'
SET #newAttachCommand = LEFT(#newAttachCommand, LEN(#newAttachCommand) - 1) + ' FOR ATTACH'
-- attach old db
PRINT #oldAttachCommand
EXEC(#oldAttachCommand)
-- attach copy db
PRINT #newAttachCommand
EXEC(#newAttachCommand)
DROP TABLE ##DBFileNames
You could just create a new database and then go to tasks, import data, and import all the data from the database you want to duplicate to the database you just created.
This program copies a database to the same server under a different name. I relied on examples given on this site with some improvements.
-- Copies a database to the same server
-- Copying the database is based on backing up the original database and restoring with a different name
DECLARE #sourceDb nvarchar(50);
DECLARE #destDb nvarchar(50);
DECLARE #backupTempDir nvarchar(200)
SET #sourceDb = N'Northwind' -- The name of the source database
SET #destDb = N'Northwind_copy' -- The name of the target database
SET #backupTempDir = N'c:\temp' -- The name of the temporary directory in which the temporary backup file will be saved
-- --------- ---
DECLARE #sourceDb_ROWS nvarchar(50);
DECLARE #sourceDb_LOG nvarchar(50);
DECLARE #backupPath nvarchar(400);
DECLARE #destMdf nvarchar(100);
DECLARE #destLdf nvarchar(100);
DECLARE #sqlServerDbFolder nvarchar(100);
Declare #Ret as int = -1
Declare #RetDescription nvarchar(200) = ''
-- Temporary backup file name
SET #backupPath = #backupTempDir+ '\TempDb_' + #sourceDb + '.bak'
-- Finds the physical location of the files on the disk
set #sqlServerDbFolder = (SELECT top(1) physical_name as dir
FROM sys.master_files where DB_NAME(database_id) = #sourceDb );
-- Clears the file name and leaves the directory name
set #sqlServerDbFolder = REVERSE(SUBSTRING(REVERSE(#sqlServerDbFolder), CHARINDEX('\', REVERSE(#sqlServerDbFolder)) + 1, LEN(#sqlServerDbFolder))) + '\'
-- Finds the logical name for the .mdf file
set #sourceDb_ROWS = (SELECT f.name LogicalName FROM sys.master_files f INNER JOIN sys.databases d ON d.database_id = f.database_id
where d.name = #sourceDb and f.type_desc = 'ROWS' )
-- Finds the logical name for the .ldf file
set #sourceDb_LOG = (SELECT f.name LogicalName FROM sys.master_files f INNER JOIN sys.databases d ON d.database_id = f.database_id
where d.name = #sourceDb and f.type_desc = 'LOG' )
-- Composes the names of the physical files for the new database
SET #destMdf = #sqlServerDbFolder + #destDb + N'.mdf'
SET #destLdf = #sqlServerDbFolder + #destDb + N'_log' + N'.ldf'
-- If the source name is the same as the target name does not perform the operation
if #sourceDb <> #destDb
begin
-- Checks if the target database already exists
IF Not EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = #destDb)
begin
-- Checks if the source database exists
IF EXISTS (SELECT name FROM master.dbo.sysdatabases WHERE name = #sourceDb) and (#sqlServerDbFolder is not null)
begin
-- Opens the permission to run xp_cmdshell
EXEC master.dbo.sp_configure 'show advanced options', 1
RECONFIGURE WITH OVERRIDE
EXEC master.dbo.sp_configure 'xp_cmdshell', 1
RECONFIGURE WITH OVERRIDE
-- If the temporary backup directory does not exist it creates it
declare #md as nvarchar(100) = N'if not exist ' + #backupTempDir + N' md ' +#backupTempDir
exec xp_cmdshell #md, no_output
-- Creates a backup to the source database to the temporary file
BACKUP DATABASE #sourceDb TO DISK = #backupPath
-- Restores the database with a new name
RESTORE DATABASE #destDb FROM DISK = #backupPath
WITH REPLACE,
MOVE #sourceDb_ROWS TO #destMdf,
MOVE #sourceDb_LOG TO #destLdf
-- Deletes the temporary backup file
declare #del as varchar(100) = 'if exist ' + #backupPath +' del ' +#backupPath
exec xp_cmdshell #del , no_output
-- Close the permission to run xp_cmdshell
EXEC master.dbo.sp_configure 'xp_cmdshell', 0
RECONFIGURE WITH OVERRIDE
EXEC master.dbo.sp_configure 'show advanced options', 0
RECONFIGURE WITH OVERRIDE
set #ret = 1
set #RetDescription = 'The ' +#sourceDb + ' database was successfully copied to ' + #destDb
end
else
begin
set #RetDescription = 'The source database '''+ #sourceDb + ''' is not exists.'
set #ret = -3
end
end
else
begin
set #RetDescription = 'The target database '''+ #destDb + ''' already exists.'
set #ret = -4
end
end
else
begin
set #RetDescription = 'The target database ''' +#destDb + ''' and the source database '''+ #sourceDb + ''' have the same name.'
set #ret = -5
end
select #ret as Ret, #RetDescription as RetDescription
<!doctype html>
<head>
<title>Copy Database</title>
</head>
<body>
<?php
$servername = "localhost:xxxx";
$user1 = "user1";
$pw1 = "pw1";
$db1 = "db1";
$conn1 = new mysqli($servername,$user1,$pw1,$db1);
if($conn1->connect_error) {
die("Conn1 failed: " . $conn1->connect_error);
}
$user2 = "user2";
$pw2 = "pw2";
$db2 = "db2";
$conn2 = new mysqli($servername,$user2,$pw2,$db2);
if($conn2->connect_error) {
die("Conn2 failed: " . $conn2->connect_error);
}
$sqlDB1 = "SELECT * FROM table1";
$resultDB1 = $conn1->query($sqlDB1);
if($resultDB1->num_rows > 0) {
while($row = $resultDB1->fetch_assoc()) {
$sqlDB2 = "INSERT INTO table2 (col1, col2) VALUES ('" . $row["tableRow1"] . "','" . $row["tableRow2"] . "')";
$resultDB2 = $conn2->query($sqlDB2);
}
}else{
echo "0 results";
}
$conn1->close();
$conn2->close();
?>
</body>
If you are MS SQL 2014 and newer;
DBCC CLONEDATABASE (CurrentDBName, NewDBName)
GO
Details;

Resources