We are using the SQL Server bcp command to export data to .csv file with the option -t ,. It works for some tables, but there are some columns that have comma in content, and the exported data are not correct. According to csv definition, if the content has comma in itself, it should be surround by double quotes.
For example, the content
hello,world,
should be exported as
"hello,world"
But bcp doesn't do that - how to resolve this issue?
Thanks,
Eric
If you're doing this for just a few tables/queries, you can escape those strings from a query. Like say you have a field x in a table y, you query would be:
SELECT '"'+REPLACE(x,'"','""')+'"' FROM y
There are no options in bcp (or sqlcmd) to automatically do this for you.
Related
I have an issue with commas in the column while exporting SQL Server query results to a .csv file. The data is splitting up to next columns with double quotes, using SSIS.
This is my sample data:
EmpId Location Dept
---------------------------
101 Nyc,AUS It,HR
102 Nyc,AUS It,HR
When exporting this data into a .csv using SSIS, the data is splitting next column though I used text-qualifier double quote (").
Any suggestions will be appreciated
You might want to use another column delimiter, such as ";" instead of "," when exporting it to .csv
When using text qualifier you have to make sure they are applied on the export also.
Open your .csv by notepad++ or any text editor and see if each Location-entry is surrounded by double quotes (")
I need to export only subset of columns from a very large table containing large number of columns.Also this table contains million of rows so i want to export only specific rows from this table.
I have recently started using Mysql earlier i was working on Oracle.
This worked for me:
mysql -u USERNAME --password=PASSWORD --database=DATABASE \
--execute='SELECT `field_1`, `field_2` FROM `table_name`' -X > file.xml
And then importing the file, using command:
LOAD XML LOCAL INFILE '/pathtofile/file.xml'
INTO TABLE table_name(field_1, field_2, ...);
What format do you need the data in? You could get a CSV using a query. For example
SELECT column1,column2,column3,... FROM table WHERE column1=criteria1,....
INTO OUTFILE '/tmp/output.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
http://www.tech-recipes.com/rx/1475/save-mysql-query-results-into-a-text-or-csv-file/
An administration tool like phpMyAdmin (http://www.phpmyadmin.net/) could also be used to run the query and then export the results in a variety of formats.
I've been tasked with extracting some data from an SQL Server 2000 database into a flat format on disk. I've little SQL Server experience.
There is a table which contains files stored in an "IMAGE" type column, together with an nvarchar column storing the filename.
It looks like there are numerous types of files stored in the table: Word docs, XLS, TIF, txt, zip files, etc.
I'm trying to extract just one row using BCP, doing something like this:
bcp "select file from attachments where id = 1234" queryout "c:\myfile.doc" -S <host> -T -n
This saves a file, but it is corrupt and I can't open it with Word. When I open the file with word, I can see a lot of the text, but I also get a lot of un-renderable characters. I've similar issues when trying to extract image files, e.g. TIF. Photo software won't open the files.
I presume I'm hitting some kind of character encoding problems.
I've played around with the -C (e.g. trying RAW) and -n options within BCP, but still can't get it to work.
The table in SQL Serer has a collation of "SQL_Latin1_General_CP1_CI_AS"
I'm running BCP remotely from my Windows 7 desktop.
Any idea where I'm going wrong? Any help greatly appreciated.
I got this working by changing the default options which BCP asks you about when you invoke the command:
The one that made the difference was changing the prefix-length field from 4 to 0.
bcp "select file from attachments where id = 1234" queryout "c:\myfile.doc" -S -T -n
after this
[image] : I (enter capital "I"]
0
0
Enter
save file Y
kallas.. ur file is there
I extracted some 10 tables in CSV with " as the text qualifier. Problem is my extract does not look right in Excel because of special characters in a few columns. Some columns are breaking into a new row when it should stay in the column.
I've been doing it manually using the management studio export feature, but what's the best extract the 10 tables to CSV with the double quote qualifier using a script?
Will I have to escape commas and double quotes? Best way to do this?
How should I handle newline codes in my columns, we need them for migration to a new system, but the PM wants to open the files and make modifications using Excel. Can they have it both ways?
I understand that much of the problem is that Excel is interpreting the file where a load utility into another database might not do anything special with new line, but what about double quotes and commas in the data, if I don't care about excel, must I escape that?
Many Thanks.
If you are using SQL Server 2005 or later, the export wizard will export the excel file out for you.
Right click the database, select Tasks-> Export Data...
Set the source to be the database.
Set the destination to excel.
At the end of the wizard, select the option to create an SSIS package. You can then create a job to execute the package on a schedule or on demand.
I'd suggest never using commas for your delimiter - they show up too frequently in other places. Use a tab, since a tab isn't too easy to include in Excel tables.
Make sure you never start a field with a space unless you want that space in the field.
Try changing your text lf's into the literal text \n. That is:
You might have:
0,1,"Line 1
Line 2", 3
I suggest you want:
0 1 "Line 1\nLine 2" 3
(assuming the spacing between lines are tabs)
Good luck
As far as I know, you cannot have new line in csv columns. If you know a column could have comma, double quotes or new line, then you can use this SQL statement to extract the value as valid csv
SELECT '"' + REPLACE(REPLACE(REPLACE(CAST([yourColumnName] AS VARCHAR(MAX)), '"', '""'), char(13), ''), char(10), '') + '"' FROM yourTable.
I have a CSV file which I am directly importing to a SQL server table. In the CSV file each column is separated by a comma. But my problem is that I have a column "address", and the data in this column contains commas. So what is happening is that some of the data of the address column is going to the other columns will importing to SQL server.
What should I do?
For this problem the solution is very simple.
first select => flat file source => browse your file =>
then go to the "Text qualifier" by default its none write here double quote like (") and follow the instruction of wizard.
Steps are -
first select => flat file source => browse your file => Text qualifier (write only ") and follow the instruction of wizard.
Good Luck
If there is a comma in a column then that column should be surrounded by a single quote or double quote. Then if inside that column there is a single or double quote it should have an escape charter before it, usually a \
Example format of CSV
ID - address - name
1, "Some Address, Some Street, 10452", 'David O\'Brian'
New version supports the CSV format fully, including mixed use of " and , .
BULK INSERT Sales.Orders
FROM '\\SystemX\DiskZ\Sales\data\orders.csv'
WITH ( FORMAT='CSV');
I'd suggest to either use another format than CSV or try using other characters as field separator and/or text delimiter. Try looking for a character that isn't used in your data, e.g. |, #, ^ or #. The format of a single row would become
|foo|,|bar|,|baz, qux|
A well behave parser must not interpret 'baz' and 'qux' as two columns.
Alternatively, you could write your own import voodoo that fixes any problems. For the later, you might find this Groovy skeleton useful (not sure what languages you're fluent in though)
Most systems, including Excel, will allow for the column data to be enclosed in single quotes...
col1,col2,col3
'test1','my test2, with comma',test3
Another alternative is to use the Macintosh version of CSV, which uses TAB's as delimiters.
The best, quickest and easiest way to resolve the comma in data issue is to use Excel to save a comma separated file after having set Windows' list separator setting to something other than a comma (such as a pipe). This will then generate a pipe (or whatever) separated file for you that you can then import. This is described here.
I don't think adding quote could help.The best way I suggest is replacing the comma in the content with other marks like space or something.
replace(COLUMN,',',' ') as COLUMN
Appending a speech mark into the select column on both side works. You must also cast the column as a NVARCVHAR(MAX) to turn this into a string if the column is a TEXT.
SQLCMD -S DB-SERVER -E -Q "set nocount on; set ansi_warnings off; SELECT '""' + cast ([Column1] as nvarchar(max)) + '""' As TextHere, [Column2] As NormalColumn FROM [Database].[dbo].[Table]" /o output.tmp /s "," -W