I am loading a CSV file in Netezza. One of the columns in this file has value like: $500,000-$749,999.
Even though this value is enclosed within double quotes, Netezza is not ignoring the comma. It throws an error like - expected end of row, "999".
There are two more columns after this field in the file. I tried adding EscapeChar ',' but it again gave an error that Delimeter and EscapeChar cannot have the same character.
Have anyone faced similar issue?
Workaround:
I can add 2 two more columns in my table, but then it would fail where field do not have comma value in it.
Try setting the QuotedValue option to DOUBLE
https://www.ibm.com/support/knowledgecenter/en/SSULQD_7.2.1/com.ibm.nz.load.doc/r_load_quotedvalue.html
Also, if all your columns are quoted, you could also set the requirequotes option to true
https://www.ibm.com/support/knowledgecenter/en/SSULQD_7.2.1/com.ibm.nz.load.doc/r_load_requirequotes.html
Related
I am trying to load a .csv table to MS SQL Server via Azure Data Factory, but I have a problem with the delimiter (;) since it appears as a character in some of the values included in some columns.
As a result, I get an error saying in the details "found more columns than expected column count".
Is there any way to change the delimiter directly on ADF before/while loading the .csv table (ex.: making it from ";" to "|||")?
Thanks in advance!
I have a problem with the delimiter (;) since it appears as a
character in some of the values included in some columns.
As you have quoted that your delimiter is ; but it is occurring as a character in some of the columns which means that there is no specific pattern of the occurrence. Hence, it is not possible in ADF.
The recommendation is to write a program using any preferred language (like python) which will iterate each row from the dataset and write a logic to replace the delimiter to ||| or you can also remove the unrequired ; and append the changes in new file. Later you can ingest this new file in ADF.
I have a CSV file I am attempting to create, and the recipient requires a header row. In this header row (and in the data) there is a field that used to be present that was removed. However, they did not remove the column that that held that data, so now, there is an empty column name surrounded by delimiters ("|"). How can I recreate this?
The expected results for the following columns should be:
RxType1|RxType2|RxType3|RxType4|RxType5||DelivID
(There is an empty column between RxType5 and DelivID) and the results would be:
|Rx|OTC|Legend|Generic|Other||Express
I am using SSRS, and have attempted adding an extra pipe the the column header for RxType5 with an empty column behind it, but the CSV seems to generate a header row based on the column names from the stored procedure and not from the RDL data. I have also attempted in the Stored Proc to create the column by using:
Select
'' AS ""
OR
'' AS "|"
but when I refresh the fields in SSRS, it puts that the column is called "ID_" (because a space, no character, or pipe is non-CLS compliant.
Any suggestions on how I can achieve this? Thanks so much :)
Try creating the column with a known name, like SELECT '' AS [RemoveMe], and then just remove that name from the row header text box.
I am dealing with a table where some columns have special character in it.
I am creating view by concatenating column names in a string and while executing it, it is giving error as 'Invalid column name' at execution statement.I tried with escaping those with escape character but it didn't worked. Tried '&' as an escaping character for '&' in column name.
Please help.
Depending on the special characters, you might be able to resolve this by putting square brackets around the column names:
CREATE TABLE dbo.result([!"£$%^&*()] int,[{}:#~#';'] varchar(50),[<>?/.,] int)
You can use the QUOTENAME function for this purpose:
SELECT QUOTENAME('!"£$%^&*')
Can you give detail ? which are these special characters ?
You can put column names inside [ ]
I have a CSV file I am importing through SSIS.Below is an sample of the data in my file
"MEM1001","OTHER","P" ,20101001,20781231,,20781231,20101001,
"Medic","General >21" ,
"A100100" ,"2210",20101001,20781231
I have added , as column delimiter and " as Text Qualifier in the connection manager.
But columns like "P" ,"Medic","General >21" ,"A100100" , are still coming enclosed with double quotes when I preview the data while rest the of the string columns are coming without double quotes.
I am guessing it has something to do with the spaces after the quotes.
Can somebody explain why this is happening and how can i make this columns to come without double quotes while importing the data from file to table.
I just stumbled across this post, I had the same issues, I was trying around and could not find any other solution.
The text qualifier " only works in csv files, when the quote is directly after the colon, no space after the colon and the text identifier/qualifier. I have no idea why.
If you aren't able to fix the input data, an option would be to create a derived column and to replace the double quotes.
This worked for me:
How to replace double quotes in derived column transformation?
Trim(REPLACE(COLA, "\"", ""))
You should also add the Trim(), otherwise you have empty spaces before and maybe after the word. This could be problematic in a merge join (in my case it was).
I don't know why this extra spaces cause this issue.
Here is what I would do. It may not be the best idea, but it should work.
You will need to add script task before data flow task that would replace all " ," and ", " to ",".
Thank you
Why not just go to the Connection Manager for that csv file, click on Columns, and under the Column delimiter box just enter a space followed by a comma? Worked for me.
I have data in the csv file similar to this:
Name,Age,Location,Score
"Bob, B",34,Boston,0
"Mike, M",76,Miami,678
"Rachel, R",17,Richmond,"1,234"
While trying to BULK INSERT this data into a SQL Server table, I encountered two problems.
If I use FIELDTERMINATOR=',' then it splits the first (and sometimes the last) column
The last column is an integer column but it has quotes and comma thousand separator whenever the number is greater than 1000
Is there a way to import this data (using XML Format File or whatever) without manually parsing the csv file first?
I appreciate any help. Thanks.
You can parse the file with http://filehelpers.sourceforge.net/
And with that result, use the approach here: SQL Bulkcopy YYYYMMDD problem or straight into SqlBulkCopy
Use MySQL load data:
LOAD DATA LOCAL INFILE 'path-to-/filename.csv' INTO TABLE `sql_tablename`
CHARACTER SET 'utf8'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
IGNORE 1 LINES;
The part optionally enclosed by '\"', or escape character and quote, will keep the data in the first column together for the first field.
IGNORE 1 LINES will leave the field name row out.
UTF8 line is optional but good to use if names have diacritics, like in José.