Snowflake copy into not recognising timestamp - snowflake-cloud-data-platform

Any suggestions on the below, i am trying to use copy into to move parquet files from S3 into snowflake table. Col1 is a timestamp and the rest are strings.
copy into table1 from
(select $1:col1, $1:col2, $1:col3
from #stage/path)
file_format = (format_name = parquet_format);
and getting the following error
Failed to cast variant value "20050111 00:00:00" to TIMESTAMP_NTZ
I have tried
copy into table1 from
(select to_timestamp($1:col1, 'yyyymmdd hh:mi:ss'), $1:col2, $1:col3
from #stage/path)
file_format = (format_name = parquet_format);
but getting the error
Error: too many arguments for function [TO_TIMESTAMP(GET(STAGE.$1, 'col1'), 'yyyymmdd hh:mi:ss')] expected 1, got 2 (line 130)
Any ideas

What's happening is this in this expression:
(select to_timestamp($1:col1, 'yyyymmdd hh:mi:ss')
This part: $1:col1 is resolving to an object rather than a primitive data type. This is what's leading to the error message about getting 2 arguments instead of one.
Changing this part of the expression to $1:col1::string casts the expression as a primitive type - string or varchar. This is the type of parameter the to_timestamp function is expecting. The final statement should be:
copy into table1 from
(select to_timestamp($1:col1::string, 'yyyymmdd hh:mi:ss'), $1:col2, $1:col3
from #stage/path)
file_format = (format_name = parquet_format);

Related

Cannot insert Array in Snowflake

I have a CSV file with the following data:
eno | phonelist | shots
"1" | "['1112223333','6195551234']" | "[[11,12]]"
The DDL statement I have used to create table in snowflake is as follows:
CREATE TABLE ArrayTable (eno INTEGER, phonelist array,shots array);
I need to insert the data from the CSV into the Snowflake table and the method I have used is:
create or replace stage ArrayTable_stage file_format = (TYPE=CSV)
put file://ArrayTable #ArrayTable_stage auto_compress=true
copy into ArrayTable from #ArrayTable_stage/ArrayTable.gz
file_format = (TYPE=CSV FIELD_DELIMITER='|' FIELD_OPTIONALLY_ENCLOSED_BY='\"\')
But when I try to run the code, I get the error:
Copy to table failed: 100069 (22P02): Error parsing JSON:
('1112223333','6195551234')
How to resolve this?
FIELD_OPTIONALLY_ENCLOSED_BY='\"\' base on the row you have that should just be '\"'
select parse_json('[\'1112223333\',\'6195551234\']');
works (the back slashes are to get around the sql parser)
but your output has parens (, ) which is different.
SELECT column2, TRY_PARSE_JSON(column2) as j
FROM #ArrayTable_stage/ArrayTable.gz
file_format = (TYPE=CSV FIELD_DELIMITER='|' FIELD_OPTIONALLY_ENCLOSED_BY='\"\')
WHERE j is null;
will show which values are failing to parse..
failing that you might want to use to_array to parse column2 and thus insert into you table the SELECTED/transformed data, that is failing to auto transform

Getting error trying to convert varchar to numeric

How do I convert varchar to numeric. This is the code I use:
SELECT TOP 1000*
FROM [StorageDB - BAW].[dbo].[EMAIL_DETAILS_HOC] eh
LEFT JOIN [StorageDB - BAW].[dbo].[Customer_Details_201906] cd ON cd.CSTMR_NUM = CAST(eh.[CIS_NUMBER] AS NUMERIC(20,0))
I'm getting this error:
Error converting data type varchar to numeric
You may try using TRY_CONVERT to flush out offending records, assuming your version of SQL Server supports it:
SELECT [CIS_NUMBER]
FROM [StorageDB - BAW].[dbo].[EMAIL_DETAILS_HOC]
WHERE TRY_CONVERT(NUMERIC(20,0), [CIS_NUMBER]) IS NULL;
There are two possibilities for getting this error first is you have length issue or second you must have non numeric value in CIS_number field so check it. Then convert it accordingly.
You Can Use This Code and Test it.
SELECT TOP 1000 *
FROM [StorageDB - BAW].[dbo].[EMAIL_DETAILS_HOC] eh
LEFT JOIN StorageDB - BAW].[dbo].[Customer_Details_201906] cd
on cd.CSTMR_NUM = cast(eh.[CIS_NUMBER ] as numeric(20,0))
Where (IsNumeric(eh.[CIS_NUMBER ])=1)

error converting datatype conversion on single vendoritemnumber in duplicate rows

Sql server query shows error :
Msg 8114, Level 16, State 5, Line 1
Error converting data type varchar to numeric.
Here is the query:
WITH employee AS (
SELECT distinct vendoritemnumber,VendorItemId,
VendorItemDescription,
VendorItemDescriptionAlias,VendorId,BrgItemId,itemconversionFactor,
orderbyuomid,pricebyuomid,vendorcasedescription,manufacturernumber,skunumber,[weight],
averageweight,currentprice,taxable,[status],createddate,inactivedate,lastpurchaseddate,
lastupdatedby,lastupdateddate,vendorpercent,vendorfreight,brandid,pack,size,inventorycategoryid,
binlocation,inventorylocation,inventorystatus,physicalinventoryconversionfactor,stateswhereused,
conceptwhereused,priceupdatedate,size_uom,pack_uom,isdeleted,'' as id,'' itemid,'' as oldprice,
'' as newprice,'' as dateupdated,'' as weekenddate,'' as lastpurdate
FROM VendorItems WHERE BrgItemId=6056)
SELECT * FROM employee
UNION ALL
SELECT '' as ven,v.VendorItemId,
v.VendorItemDescription,v.VendorItemDescriptionAlias,v.VendorId,v.BrgItemId,v.itemconversionFactor,
v.orderbyuomid,v.pricebyuomid,v.vendorcasedescription,v.manufacturernumber,v.skunumber,v.[weight],
v.averageweight,v.currentprice,v.taxable,v.[status],v.createddate,v.inactivedate,v.lastpurchaseddate,
v.lastupdatedby,v.lastupdateddate,v.vendorpercent,v.vendorfreight,v.brandid,v.pack,v.size,v.inventorycategoryid,
v.binlocation,v.inventorylocation,v.inventorystatus,v.physicalinventoryconversionfactor,v.stateswhereused,
v.conceptwhereused,v.priceupdatedate,v.size_uom,v.pack_uom,v.isdeleted,ph.id,ph.itemid,ph.oldprice,
ph.newprice,ph.dateupdated,ph.weekenddate,brg.lastpurchasedate
from [dbo].[VendorItems] v join PriceHistory ph
on ph.ItemId=v.VendorItemId join brgitems brg on
brg.brgitemid=v.BrgItemId
WHERE v.BrgItemId=6056
group by vendoritemnumber,v.VendorItemId,v.VendorItemDescription,v.VendorItemDescriptionAlias,v.VendorId,v.BrgItemId,v.itemconversionFactor,
v.orderbyuomid,v.pricebyuomid,v.vendorcasedescription,v.manufacturernumber,v.skunumber,v.[weight],
v.averageweight,v.currentprice,v.taxable,v.[status],v.createddate,v.inactivedate,v.lastpurchaseddate,
v.lastupdatedby,v.lastupdateddate,v.vendorpercent,v.vendorfreight,v.brandid,v.pack,v.size,v.inventorycategoryid,
v.binlocation,v.inventorylocation,v.inventorystatus,v.physicalinventoryconversionfactor,v.stateswhereused,
v.conceptwhereused,v.priceupdatedate,v.size_uom,v.pack_uom,v.isdeleted,ph.id,ph.itemid,ph.oldprice,
ph.newprice,ph.dateupdated,ph.weekenddate,brg.lastpurchasedate
I want to show single vendoritemnumber in duplicate rows..Any suggestion??
Where is the error??
SQL Server cannot convert an empty string to NUMERIC data type. For example, the following will yield the same error:
SELECT CONVERT(NUMERIC, '')
Replace empty strings ('') with 0 or NULL.
#manoj .Just FYI to know exact error point , double click on the error in message window .It shows the exact line of code on error, Also try to check if any empty values are there for that column and as mentioned by #Serge SQL throws error while any data type is converted to numeric and if its value is empty. Best solution is to add WHERE clause and filter those cases.

Query with integers not working

I've been searching here on stackoverflow and other sources but not found a solution to this
The query below works as expected expect for when either custinfo.cust_cntct_id or custinfo.cust_corrcntct_id = '' (blank not NULL) then I get no results. Both are integer fields and if both have an integer value then I get results. I still want a value returned for either cntct_email or corrcntct_email even if custinfo.cust_cntct_id or custinfo.cust_corrcntct_id = blank
Can someone help me out in making this work? The database is PostgreSQL.
SELECT
cntct.cntct_email AS cntct_email,
corrcntct.cntct_email AS corrcntct_email
FROM
public.custinfo,
public.invchead,
public.cntct,
public.cntct corrcntct
WHERE
invchead.invchead_cust_id = custinfo.cust_id AND
cntct.cntct_id = custinfo.cust_cntct_id AND
corrcntct.cntct_id = custinfo.cust_corrcntct_id;
PostgreSQL won't actually let you test an integer field for a blank value (unless you're using a truly ancient version - 8.2 or older), so you must be using a query generator that's "helpfully" transforming '' to NULL or a tool that's ignoring errors.
Observe this, on Pg 9.2:
regress=> CREATE TABLE test ( a integer );
CREATE TABLE
regress=> insert into test (a) values (1),(2),(3);
INSERT 0 3
regress=> SELECT a FROM test WHERE a = '';
ERROR: invalid input syntax for integer: ""
LINE 1: SELECT a FROM test WHERE a = '';
If you are attempting to test for = NULL, this is not correct. You must use IS NOT NULL or IS DISTINCT FROM NULL instead. Testing for = NULL always results in NULL, which is treated as false in a WHERE clause.
Example:
regress=> insert into test (a) values (null);
INSERT 0 1
regress=> SELECT a FROM test WHERE a = NULL;
a
---
(0 rows)
regress=> SELECT a FROM test WHERE a IS NULL;
a
---
(1 row)
regress=> SELECT NULL = NULL as wrong, NULL IS NULL AS right;
wrong | right
-------+-------
| t
(1 row)
By the way, you should really be using ANSI JOIN syntax. It's more readable and it's much easier to forget to put a condition in and get a cartesian product by accident. I'd rewrite your query for identical functionality and performance but better readability as:
SELECT
cntct.cntct_email AS cntct_email,
corrcntct.cntct_email AS corrcntct_email
FROM
public.custinfo ci
INNER JOIN public.invchead
ON (invchead.invchead_cust_id = ci.cust_id)
INNER JOIN public.cntct
ON (cntct.cntct_id = ci.cust_cntct_id)
INNER JOIN public.cntct corrcntct
ON (corrcntct.cntct_id = ci.cust_corrcntct_id);
Use of table aliases usually keeps it cleaner; here I've aliased the longer name custinfo to ci for brevity.

Error using to_char // to_timestamp

I have a database in PostgreSQL and I'm developing an application in PHP using this database.
The problem is that when I execute the following query I get a nice result in phpPgAdmin but in my PHP application I get an error.
The query:
SELECT t.t_name, t.t_firstname
FROM teachers AS t
WHERE t.id_teacher IN (
SELECT id_teacher FROM teacher_course AS tcourse
JOIN course_timetable AS coursetime
ON tcourse.course = coursetime.course
AND to_char(to_timestamp('2010-4-12', 'YYYY-MM-DD'),'FMD') = (coursetime.day +1)
)
AND t.id_teacher NOT IN (
SELECT id_teacher FROM teachers_fill WHERE date = '2010-4-12'
)
ORDER BY t.t_name ASC
And this is the error in PHP
operator does not exist: text = integer (to_timestamp('', 'YYYY-MM-DD'),'FMD') =
(courset... ^ HINT: No operator matches the given name and argument type(s).
You might need to add explicit type casts.
The purpose to solve this error is to use the ORIGINAL query in php with :
$date = "2010"."-".$selected_month."-".$selected_day;
SELECT ...
AND to_char(to_timestamp('$date', 'YYYY-MM-DD'),'FMD') = (coursetime.day +1)
)
AND t.id_teacher NOT IN (
SELECT id_teacher FROM teachers_fill WHERE date = '$date'
)
The error message seems quite clear to me. You are mixing strings and numbers. More precisely, you are converting a string ('2010-4-12') to a timestamp, then to a string, then comparing to an int. This is a type mess, and postgresql is quite strict with typing (for good reasons). What are you trying to do here ?
to_char(to_timestamp('2010-4-12', 'YYYY-MM-DD'),'FMD') = (coursetime.day +1))
Further, you should use a TIMESTAMP, just a DATE.
If (I'm not sure) you are tring to compare the day of week from a date formated as 'YYYY-MM-DD' to a given value (as an integer), you should better use date_part. For example (not tested):
date_part('dow' , to_date('2010-4-12', 'YYYY-MM-DD') ) = coursetime.day + 1

Resources