Populating ms sql with sql files - database

I already have an existing database I am trying to keep up to date on a daily basis. I get a daily dump of sql files. The batch script below created and populated the database the first time I run it, but that doesn't work when I try to update the database with it.
`#echo off
ECHO %USERNAME% started the batch process at %TIME% >output.txt
for %%f in (*.sql) do (
sqlcmd.exe -S servername -E -d DatabaseName -i %%f >>output.txt
)
pause`
Is there a different command for updating a database with sql files?
This is the output i get.
HUTRC started the batch process at 9:55:12.25 Changed database context to 'master'. Msg 15416, Level 16, State 1, Server HUTRC1-HP, Procedure sp_dbcmptlevel, Line 67 Usage: sp_dbcmptlevel [dbname [, compatibilitylevel]] Valid values of the database compatibility level are 100, 110, or 120.
What the sql file looks like. It is very long. I just got the first few lines.
USE [master]
GO
IF NOT EXISTS (SELECT [name] FROM sys.databases WHERE name = N'Migration')
BEGIN
CREATE DATABASE [Migration] COLLATE SQL_Latin1_General_CP1_CI_AS
END
GO
EXEC dbo.sp_dbcmptlevel #dbname=N'Migration', #new_cmptlevel=90

The error message tells you precisely where to look for the problem and what it is (emphasis added):
Procedure sp_dbcmptlevel, Line 67 Usage: sp_dbcmptlevel [dbname [, compatibilitylevel]] Valid values of the database compatibility level are 100, 110, or 120.
Examining your SQL script for sp_dbcmptlevel shows that it uses a different value:
EXEC dbo.sp_dbcmptlevel #dbname=N'Migration', #new_cmptlevel=90
^^
You'll need to either edit the SQL script to a valid compatibility level, or downgrade your server version to the same version as the source server.

Related

Backup MSSQL DB Schema

I'm trying to back up my production database to my local dev machine with the following constraints:
It will be a regular backup, so using the UI should (ideally?) be avoided.
Some tables are marked to be deleted, so these should not be included in the backup.
I would like to be able to pass the solution (file/package/etc) to other members of the team and they should only have to change a couple of variables in one file and then they can execute and get their own backup.
The DB is over 100GB and contains data that I won't need. I have identified the top largest tables and would only like to take say 5k rows from each - this should provide me with enough data for my purposes and limit space used on my local drives.
I have tried beginning with backing up the schema only using the following methods:
Using the UI to backup Schema only (Tasks -> Generate Scripts)
Get the following error:
Microsoft.SqlServer.Management.Smo.FailedOperationException: Discover dependencies failed. ---> System.ArgumentException: Item has already been added. Key in dictionary: 'Server[#Name='PRODSQL']/Database[#Name='Database1']/UnresolvedEntity[#Name='SomeObjectName' and #Schema='Some.Schema.SomeObjectName']' Key being added: 'Server[#Name='PRODSQL']/Database[#Name='Database1']/UnresolvedEntity[#Name='SomeObjectName' and #Schema='Some.Schema.SomeObjectName']' at System.Collections.SortedList.Add(Object key, Object value) at Microsoft.SqlServer.Management.Smo.DependencyTree..ctor(Urn[] urns, DependencyChainCollection dependencies, Boolean fParents, Server server) at Microsoft.SqlServer.Management.Smo.DependencyWalker.DiscoverDependencies(Urn[] urns, Boolean parents) --- End of inner exception stack trace --- at Microsoft.SqlServer.Management.SqlScriptPublish.GeneratePublishPage.worker_DoWork(Object sender, DoWorkEventArgs e) at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e) at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)
So I moved on.
Tasks -> Copy Database
I get a message saying there is not enough space on disk
Extract Data-tier Application
Get same error as in 1. above.
Powershell script, and a batch file calling sqlcmd on the generated .sql files after the PS script was run.
I was sure this method would work and it took me 2 days to get this far, but still working through multiple errors.
Basically I am doing the following:
Create db objects from the source DB (Schemas, SPs, Tables, Views, UDFs, Triggers, Indexes) and output them to .sql files - Roughly followed http://cfmumbojumbo.com/index.cfm/coding/using-powershell-to-backup-your-stored-procedures-and-triggers/ with some more work added.
If the database already exists on my server, kill, drop, then recreate it (DropCreate.sql):
IF(db_id(#DatabaseName) IS NOT NULL)
BEGIN
DECLARE #SQL VARCHAR(max)
SELECT #SQL = COALESCE(#SQL,'') + 'Kill ' + Convert(VARCHAR, SPId) + ';'
FROM MASTER..SysProcesses
WHERE DBId = DB_ID(#DatabaseName) AND SPId <> ##SPId
EXEC(#SQL);
END
DROP DATABASE MYDATABASE
CREATE DATABASE MYDATABASE ON PRIMARY (...)
The .bat file is essentially doing this
sqlcmd -S %Server% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\DropCreateDB.sql
#loop through and execute multiple .sql files in the directory
for /f %f in (`dir /b C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\StoredProcedures\`) do sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i %f
#Just one sql file in this directory, execute it
sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\Schemas\AllSchemas.sql
sqlcmd -S %Server% -d %Database% -U %UserName% -P %Password% -i C:\Database\DatabaseScripts\TestDatabase\2018-04-18-15-13-13\Tables\AllTables.sql
.............
The latest error I'm experiencing is:
Changed database context to 'master'.
Msg 6107, Level 14, State 1, Server MYSERVER, Line 1
Only user processes can be killed.
do was unexpected at this time.
Everywhere I turn I am experiencing new errors and have spent over 2 days on it, and I haven't even got to getting the data yet..
TLDR: Is there any easier way backup MSSQL Db schema and top n rows of data from certain tables?

What is wrong with the codes in this BCP utility running on SQL Server?

I have created a BCP utility and I have wrapped it in a bat file. I have then created a daily task using Task Scheduler in Windows Server 2012.
The function of the BCP utility is to rename a file called 'myfile.csv' (located in C:) by adding a date stamp to it and updating the file with the result of a SQL query.
The codes currently stand as follows:
cd:\Program Files\ Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn
set vardate=%DATE:~4,10%
set varDateWithoutSlashes=%vardate:/=-%
ren C:\myfile.csv myfile_%varDateWithoutSlashes%.csv
bcp "SELECT TOP 100 ReservationStayID,NameTitle,FirstName,LastName,ArrivalDate,DepartureDate FROM MyDatabase.dbo.GuestNameInfo" queryout C:\myfile.csv -t, -c -S [ipaddress] -U sa -P 1234
My problem is that when the task runs, it renames the file correctly with a the date stamp but it seems that the SELECT query does not run as the file is empty (except the headers, which have been pre-loaded by the way).
What is wrong with my codes?
I should also add the following:
Are the double quotes in the select statement above correct? Or should they be single quotes?
Should the ipaddress in my codes above be in square brackets or should I remove them?
I have left the "Location" filed 'as is' in the Task Scheduler (please see screenshot below). Should that be filled? If yes, by what?
Thanks for helping out!

sqlcmd utility returns "Could not find stored procedure"

I am trying to execute a stored procedure using this command line utility:
sqlcmd -m 1 -S inxcert -U user1 -P u8er1 -i "D:\ESP\RunSQL.sql" -h -1 -o "D:\ESP\testoutput.txt"
Following is what I have written in RunSQL.sql:
exec spc.load_tables
Though the stored procedure exist in the database, credentials are correct and SQL Server runs fine when run from SSMS I am getting the following error in the output file:
Msg 2812, Level 16, State 62, Server I0160SQL03\I0160SQL03, Line 1
Could not find stored procedure 'spc.mjr_vs_load_tables'.
Please help me to learn how to resolve the error.
Looks like it's executing against the default database (probably master) so not finding your procedure.
Try either adding:
USE [DBNAME]
to RunSQL.sql, or specifying:
-d DBNAME
to your sqlcmd parameters.

Restoring .bak file from batch file

I am automating a test environment setup process and below are part of the manual steps involved:
Copy a .bak file from a network drive to my D: drive
Launch SQL Management Studio, use the Restore Files and Filegroups option to restore the .bak file
For point #1, I used:
robocopy \\10.xx.xxx.x\dirA\Build\%myVar%\DB\ D:\ myDB_V%myVar%.bak
It copied just fine!
Now for point 2, I am trying the below command using sqlcmd from command prompt:
sqlcmd -E -S localhost\myInstance -Q "RESTORE DATABASE myDB_V2.2.2.10 FROM DISK='D:\myDB_V2.2.2.10.bak' WITH FILE = 1, MOVE 'myDB_V2.2.2.10' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf', MOVE 'myDB_V2.2.2.10_log' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf';"
It says
Msg 102, Level 15, State 1, Server localhost\myInstance, Line 1
Incorrect syntax near '.2'.
Msg 319, Level 15, State 1, Server localhost\myInstance, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
I could not interpret what it is saying.
Can anyone help me please?
Try this:
sqlcmd -E -S localhost\myInstance -Q "RESTORE DATABASE 'myDB_V2.2.2.10' FROM DISK='D:\myDB_V2.2.2.10.bak' WITH FILE = 1, MOVE 'myDB_V2.2.2.10' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf', MOVE 'myDB_V2.2.2.10_log' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf';"
Try this
sqlcmd -E -S localhost\myInstance -Q "RESTORE DATABASE [myDB_V2.2.2.10] FROM DISK='D:\myDB_V2.2.2.10.bak' WITH FILE = 1, MOVE 'myDB_V2.2.2.10' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf', MOVE 'myDB_V2.2.2.10_log' TO 'C:\Program Files\Microsoft SQLServer\MSSQL10.SQLEXPRESS\MSSQL\DATA\myDB_V2.2.2.10.mdf';"
It looks like the error is about your database name, "RESTORE DATABASE myDB_V2**.2**.2.10 [...]". I would suggest not using periods in your database name as periods are used to separate database objects in queries. If you are already using periods, try wrapping the database name in square brackets [].

Run SQLCMD through Batch Script to create a formatted XML file from a stored transaction

I'm trying to automate the process of running a stored procedure in a database by using SQLCMD. SQL Server Management Studio 2008 is installed on the Windows Server that this is all trying to happen in. SQLCMD will be called through a batch script and told to execute the stored procedure and save the output into an XML file. I cannot show the stored procedure as it has sensitive material, but it includes the usage of FOR XML PATH('').
I read several articles from all kinds of sites and people have said to use :XML ON to get the output in actual XML format and not in tabular format, as well as the switches of "-h-1 -y 0" to make sure that the output isn't truncated. I am trying to run the SQLCMD through a batch script so that it can all be automated.
My current batch script (the variables are all defined before this line in the script):
sqlcmd -X -Q -h-1 -y 0 "EXEC %TransactionName%" -d %Database% -S %ServerInstance% -o "%OutFilename%_%currDATE%.xml"
I tried adding :XML ON in the transaction as well as creating a seperate SQL script that reads:
Run_Transact.sql
:XML ON
EXEC storedProcedure
and so the batch file would then read:
sqlcmd -X -Q -h-1 -y 0 -i runTransact.sql -d %Database% -S %ServerInstance% -o "%OutFilename%_%currDATE%.xml"
I get back the error:
HResult 0x80004005, Level 16, State 1
No description provided
If I don't use :XML ON then I get output that looks like it is in tabular format and it includes a header as well as only the first record, but not all of it (it gets truncated).
My question is how can I get the output in the XML file to actually look like XML and not truncated as well?
Thanks so much in advance!
This approach works for me. I retrieved 2 milion lines xml usinq sqlcmd from xml column. My steps:
1) Create File with query, this is mine:
:XML ON
USE <DB_NAME>
SELECT TOP 1 <COLUMN_NAME> FROM <TABLE>
WHERE <SOMETHING>
FOR XML PATH('')
2) Execute command in sqlcmd
sqlcmd -d <DB_NAME> -i <PATH>\query.sql >result.txt
And replace what you need in <>
In your case you have stored procedure. Maybe it will cause problems? But you can try something like;
USE<DB_NAME>
DECLARE #XMLV XML
EXEC #XMLV = EXEC StoredProcedure
SELECT #XMLV
FOR XML PATH('')
I know the question is quite old, but maybe it will help someone.
SELECT ( SELECT 'White' AS Color1,
'Blue' AS Color2,
'Black' AS Color3,
'Light' AS 'Color4/#Special',
'Green' AS Color4,
'Red' AS Color5
FOR
XML PATH('Colors'),
TYPE
),
( SELECT 'Apple' AS Fruits1,
'Pineapple' AS Fruits2,
'Grapes' AS Fruits3,
'Melon' AS Fruits4
FOR
XML PATH('Fruits'),
TYPE
)
FOR XML PATH(''),
ROOT('SampleXML')
GO

Resources