I use Visual Studio 2008 database project for managing my database scripts and version control. by default visual studio script templates are encoded in UTF-8 format. when I try to concatenate the scripts for database release (using the DOS command "type *.sql > dbscripts.sql") and run the concatenated script from SQL server management studio, I get invalid character error. however if I convert each script to ANSI format (using notepad), and concatenate them then I don't get any errors when I run the concatenated script. but converting each script into a different encoding format is tedious. what is the best way to execute multiple SQL scripts regardless of the encoding format of the scripts? do I need to create a batch file and use sqlcmd to run it and redirect the output to a text file? are there any alternatives?
Thanks in advance.
The easy solution is to add the codepage flag to the command, -f 65001
sqlcmd /S myServer /d myDatabase -E -f 65001 -i MyUTFScriptWithoutBOM.sql
That's because of the BOM in the file. The TYPE command is too primitive to detect it and will just copy the 3 bytes to the output.
You can change the encoding used for the .sql file. Tedious too btw. File + Save As, click the arrow on the Save button, Save With Encoding and select your current code page. Like 1252 for Western Europe and Americas, it is probably the first one in the list. This is the same thing you did with Notepad.
Beware of the trouble you can get into if the server runs with a different code page and your scripts contain accented characters. I would myself just write a little utility that uses Directory.GetFiles and StreamReader/Writer to fix the problem.
found this link which generates "SQLCMD" commands using Windows Powershell. for concatenation, it looks like there is no solution except saving each file in ANSI format. I can at least generate a file with all sqlcmd commands using this without having to concatenate them
http://sqladm.blogspot.com/2010/02/generate-sqlcmd-statements.html
You should use the copy *.sql dbscripts.sql
Related
I am really new to Batch and DB2 and got little time to explore much about them. I just want to know how it is possible for a batch program to pass a value/s to a db2 file so I can manipulate my database.
I found several suggestions but none of them worked. Here's my batch codes so far:
Rem This is db2execute.bat
#echo off
db2cmd -c -w -i db2 -tf INSERT.db2 id=1
PAUSE
My .db2 file on the other hand:
CONNECT TO SAMPLEDB;
INSERT INTO TB1 VALUES('$(ID)');
I would really appreciate some kind help. Thanks.
At the present time, in the shipping versions of Db2-LUW, the CLP (command line processor) does not directly support parameters in script files in the style that your question suggests.
If your product-type and version offers the clpplus command, then you can instead try using the Oracle sqlplus style for passing parameters on the command line and referencing those parameters in your script. See the Db2 and Oracle documentation for details .
I'm trying to insert new row into a database containing text with special characters like long dash (—). When I do this manually in my SSMS - it works okay, but when I commit the script into my version management tool (Github desktop), these symbols show up as �. In Visual Studio special characters show up normally as well. What should i do so that I can add the script properly and it could be executed potentially against any SQL Server 2016 database?
How my changes appear in Github Desktop:
It appears that the behavior is caused by the encoding type for the *.sql file to where I put my script with special characters. The file uses UTF-8 encoding, while it should be saved using UTF-8-BOM encoding to be able to display those characters correctly.
I have a MSSQL database, which contains Unicode (utf8) data. My workstation is linux (currently Ubuntu) and looking for a tool to work with mssql database I found SQSH.
The problem is - when I select data in the sqsh console I get jibberish instead of unicode characters. Using switch "-J utf8" or "-J utf-8" didn't change anything.
The question is - how to set up sqsh to work with utf-8 data?
If it is not possible, do you know any alternative tools usable from linux for work with mssql databases filled with utf-8 data. I need to execute all kinds of T-SQL, run previsously prepared SQL script files, and pipe out results for processing afterwards. A good GUI (open source) could also be used, not limited to shell clients.
Are you using freetds with sqsh? If you are, edit your freetds.conf to set the charset.
http://www.freetds.org/userguide/localization.htm
Use Azure Data Studio to avoid data troubleshooting issue. it is a great SSMS alternative for Linux.
If you need command line tool I suggest to use official sqlcmd from mssql-tools.
It is available for all major Linux distributions including Ubuntu.
Connecting with sqlcmd
Another shell tool is mssql-cli
Features
Mssql-cli is a new and interactive command line tool that provides the
following key enhancements over sqlcmd in the Terminal environment:
T-SQL IntelliSense
Syntax highlighting
Pretty formatting for query results, including Vertical Format
Multi-line edit mode
Configuration file support
I had the same problem and it seems it has nothing to do with charchters encoding, but the problem was there are control character, unprintable characters, in the script.
I removed them from the sql script and everything works fine.
I have an SQL server 2008 database instance on one machine. Now I want to copy this database to another machine. I use the script wizard inside SQL Management Studio to generate a SQl-script with the schema and data. The script-file is rather big (around 17 GB).
Then I run the sql-script on the target machine it results in a :
Msg 105, Level 15, State 1 error with the message:
Unclosed quotation mark after the character string
I do understand the problem of what unclosed quotation marks mean. But I don't understand why the error happens. Isn't the script generator able to handle quotations inside text strings like...hello, what's up...correctly and create a script that will escape such characters?
Is their a limit on the length of text for the script wizard? Is this causing the problem.
I don't want to and I cannot open the script-file in a text editor (too large, text editor will crash) and manually fix the problems.
Do you have any ideas?
Solution for the SQL Server Import Issue
Pre-condition
In order to move the data from one SQL Server to another (e.g. from Production environment to Test environment) makes sense to use "Generate scripts" feature which is available in database options in SQL Server Management Studio. The result of this operation is text file with SQL commands that can be executed on another SQL Server. Usually these files are too big to execute them in SQL Server Management Studio, so we need to use sqlcmd command line utility from SQL Server installation package. In the most cases utility works smoothly and additional user actions are not necessary.
Issue description
In some rare cases the sqlcmd utility can fail with the import and raise the following error: "Unclosed quotation mark after the character string ..." which indicates that one of SQL queries has not been executed. This happens because sqlcmd works using stream processing, i.e. it reads some piece of data, processes it, reads next piece and so on. In some cases an input file can contain huge SQL instruction which size is bigger than the amount of the data that could be processed by sqlcmd at a time, so sqlcmd tries to execute broken SQL and fails.
Possible solutions
In order to fix this issue 2 approaches can be used:
The sqlcmd utility can accept the "-a" parameter which defines the maximum size of packet (piece of data) that will be used during processing. The maximum value is 32767, the default value is 4096, so it makes sense to always use this parameter with maximum value.
sqlcmd -i input.sql -a 32767 -o import_log.txt
If the first approach didn't help and issue still appears, there is another, more difficult solution:
Install the Cygwin
During the installation, after some standard screens, stop on the screen "Select packages"
In "Search" field, enter "sed", and in the tree below expand the "Base" category and choose version not less than 4.2.2 for installation
Complete installation
Note: "sed" is the Linux utility which allows stream-based file processing
After installation is completed, run "Cygwin64 Terminal" from the desktop. We will use it for next steps
Go to the directory where the SQL file generated by SQL Server Management Studio is located. You need to use Linux style slashes "/" instead of Windows style which is "\"
cd d:/temp
Change the encoding of the SQL file from UTF-16LE to UTF-8, because "sed" cannot process UTF-16LE, this conversion is safe for the data. The result will be a new file, that we will use in next step
iconv -f UTF-16LE -t UTF-8 input.sql > input_utf8.sql
Convert the new file, to have one SQL query in one batch. The result will be a new file, that we will use in next step
sed -e 's/^INSERT/GO\nINSERT/' input_utf8.sql > input_utf8_adapted.sql
Now the file "input_utf8_adapted.sql" should be processed by sqlcmd without any issues, so we can execute the following:
sqlcmd -i input_utf8_adapted.sql -a 32767 -o import_log.txt
After execution is done, please check import_log.txt to make sure that no errors appeared
I ended up on this question after trying to find a solution to a similar problem I had. I also needed to dump a DB (with data) via Generate scripts wizard and the resulting file was too big to be executed from SSMS. So I tried the sqlcmd but ended with the error
Sqlcmd: Error: Syntax error at line 10 near command '"' in file 'script.sql'.
It turned out the cause of the issue was a record containing data with jQuery syntax in it - $(".someclass"). It's because it is also a way how to insert a variable into sqlcmd.
The solution is to disable variable substitution by adding -x command line argument.
Not a direct answer to the question but to duck this issue you could use one of the following other methods of copying the database to the new location.
Copying Databases with Backup and Restore
Using Detach and Attach
Method 1 is usually preferable as it keeps the source DB online and detaching can cause information held in the master database about the source to be lost (e.g. full text enabled status)
EDIT: Just noted from your comment that you're running sqlcmd -S server\database -i script.sql. There is a -I switch that stands for "Enable Quoted Identifiers". Try to run the command with this switch.
Btw, to edit a large file, consider using a nice editor like Notepad++ or UltraEdit. I wouldn't use a workstation without em :)
Opening large sql script generated by SQL Server publisher cant be open in management studio, returning error about not enough available storage to open it.
Is there some other way to import db from large script ? (command line maybe)
Is this something you have to edit? If so, you may want to open it in Notepad++ or TextPad or Editplus.
Here are some options I can think of:
Use the batch separator GO between sets of commands. The reason for this is that without the GO, SSMS is trying to execute the entire script as a single command. This puts a heavier load on memory requirements than multiple batches would.
To run the script, you can use SQLCMD from the command line.
Also, for large scripts that load data, you may want to ensure that you have COMMIT commands in the script (where appropriate).
Consider splitting your script into multiple scripts.
If you split into multiple files and build the SQLCMD command line syntax, you can run all scripts from a single batch file fairly quickly.
Have you tried using the OSql tool?