This question already has answers here:
LISTAGG function: "result of string concatenation is too long"
(14 answers)
Closed 6 years ago.
Is there some function that makes the same behavior of SYS.STRAGG in oracle but instead of returning VARCHAR (and being limited to the VARCHAR size), it returns a CLOB , and thus allows (virtually) infinite number of concatenated strings ?
for example , I have a query select x from y where z that returns 2.5M records and I want to return all these records concatenated together 1 shot
XML functions can be used for such aggregation but for 2.5M records it will be very slow.
Example:
SELECT
rtrim(
dbms_xmlgen.convert(
extract(
xmlroot(
xmlelement(
"x",
xmlagg(sys_xmlgen(object_name || ', '))
),
version '1.0'),
'/x/ROW/text()').getclobval(),
1),
', ') aggregated_data
FROM
all_objects
You might consider to use LISTAGG for pre-aggregation of small row groups into VARCHARs smaller than 4000/32767 bytes and then use the XML aggregation for the final result.
Related
I am moving a query from SQL Server to Snowflake. Part of the query creates a pivot table. The pivot table part works fine (I have run it in isolation, and it pulls numbers I expect).
However, the following parts of the query rely on the pivot table- and those parts fail. Some of the fields return as a string-type. I believe that the problem is Snowflake is having issues converting string data to numeric data. I have tried CAST, TRY_TO_DOUBLE/NUMBER, but these just pull up 0.
I will put the code down below, and I appreciate any insight as to what I can do!
CREATE OR REPLACE TEMP TABLE ATTR_PIVOT_MONTHLY_RATES AS (
SELECT
Market,
Coverage_Mo,
ZEROIFNULL(TRY_TO_DOUBLE('Starting Membership')) AS Starting_Membership,
ZEROIFNULL(TRY_TO_DOUBLE('Member Adds')) AS Member_Adds,
ZEROIFNULL(TRY_TO_DOUBLE('Member Attrition')) AS Member_Attrition,
((ZEROIFNULL(CAST('Starting Membership' AS FLOAT))
+ ZEROIFNULL(CAST('Member Adds' AS FLOAT))
+ ZEROIFNULL(CAST('Member Attrition' AS FLOAT)))-ZEROIFNULL(CAST('Starting Membership' AS FLOAT)))
/ZEROIFNULL(CAST('Starting Membership' AS FLOAT)) AS "% Change"
FROM
(SELECT * FROM ATTR_PIVOT
WHERE 'Starting Membership' IS NOT NULL) PT)
I realize this is a VERY big question with a lot of moving parts... So my main question is: How can I successfully change the data type to numeric value, so that hopefully the formulas work in the second half of the query?
Thank you so much for reading through it all!
EDITED FOR SHORTENING THE QUERY WITH UNNEEDED SYNTAX
CAST(), TRY_TO_DOUBLE(), TRY_TO_NUMBER(). I have also put the fields (Starting Membership, Member Adds) in single and double quotation marks.
Unless you are quoting your field names in this post just to highlight them for some reason, the way you've written this query would indicate that you are trying to cast a string value to a number.
For example:
ZEROIFNULL(TRY_TO_DOUBLE('Starting Membership'))
This is simply trying to cast a string literal value of Starting Membership to a double. This will always be NULL. And then your ZEROIFNULL() function is turning your NULL into a 0 (zero).
Without seeing the rest of your query that defines the column names, I can't provide you with a correction, but try using field names, not quoted string values, in your query and see if that gives you what you need.
You first mistake is all your single quoted columns names are being treated as strings/text/char
example your inner select:
with ATTR_PIVOT(id, studentname) as (
select * from values
(1, 'student_a'),
(1, 'student_b'),
(1, 'student_c'),
(2, 'student_z'),
(2, 'student_a')
)
SELECT *
FROM ATTR_PIVOT
WHERE 'Starting Membership' IS NOT NULL
there is no "starting membership" column and we get all the rows..
ID
STUDENTNAME
1
student_a
1
student_b
1
student_c
2
student_z
2
student_a
So you need to change 'Starting Membership' -> "Starting Membership" etc,etc,etc
As Mike mentioned, the 0 results is because the TRY_TO_DOUBLE always fails, and thus the null is always turned to zero.
now, with real "string" values, in real named columns:
with ATTR_PIVOT(Market, Coverage_Mo, "Starting Membership", "Member Adds", "Member Attrition") as (
select * from values
(1, 10 ,'student_a', '23', '150' )
)
SELECT
Market,
Coverage_Mo,
ZEROIFNULL(TRY_TO_DOUBLE("Starting Membership")) AS Starting_Membership,
ZEROIFNULL(TRY_TO_DOUBLE("Member Adds")) AS Member_Adds,
ZEROIFNULL(TRY_TO_DOUBLE("Member Attrition")) AS Member_Attrition
FROM ATTR_PIVOT
WHERE "Starting Membership" IS NOT NULL
we get what we would expect:
MARKET
COVERAGE_MO
STARTING_MEMBERSHIP
MEMBER_ADDS
MEMBER_ATTRITION
1
10
0
23
150
This question already has an answer here:
Virtually blank column in array?
(1 answer)
Closed 5 months ago.
I have this file with an input table in Google Sheets.
Keys
Tags
V1
V2
kEp
tag1
30
12
PgZ
tag2
8
2
pac
tag3
15
21
This is what i did; I added REGEXREPLACE(QUERY({A1:D},"Select Col1"),".+"," ") to get the empty column I
=ArrayFormula({
QUERY({A1:D}," Select Col1,Col2,Col3 ",1),
REGEXREPLACE(QUERY({A1:D},"Select Col1"),".+"," "),
QUERY({A1:D}," Select Col1,Col2,Col4 ",1)})
The ask
Is there is a simple way with the same range refrence this case A1:D to add an empty column to the array {} like this &""& ?
If 'empty' doesn't really have to be that empty, this is pretty simple...
=QUERY({A1:D4,A1:B4},"select Col1,Col2,Col3,' ',Col5,Col6,Col4 label ' '''")
You can try-
={QUERY({A1:D}," Select Col1,Col2,Col3 where Col1 is not null",1),
FLATTEN(SPLIT((REPT(" |",COUNTA(A:A))),"|")),
QUERY({A1:D}," Select Col1,Col2,Col4 where Col1 is not null",1)}
And simplified formula-
={QUERY(A:D,"select A,B,C where A is not null",1),
FLATTEN(SPLIT((REPT(" |",COUNTA(A:A))),"|")),
QUERY(A:D,"Select A,B,D where A is not null",1)}
Get different cuts of the range through OFFSET and join them along with empty arrays crafted with MAKEARRAY:
=LAMBDA(rg,where,how_many,
{
OFFSET(rg,0,0,,where),
MAKEARRAY(ROWS(rg),how_many,LAMBDA(r,c,)),
OFFSET(rg,0,where,,COLUMNS(rg)-where)
}
)(A1:INDEX(D:D,COUNTA(D:D)),1,2)
This question already has answers here:
VARCHAR to DECIMAL
(12 answers)
Closed 3 years ago.
When I write query like
Select 3 + '44. 5'
It gave me error conversion failed when converting varchar value to '44. 6' to data type int
Why it can not return proper error for decimal cannot converting to int.
Remove the single quote. The quote means that it is a string value not a real number value.
SELECT 3 + 44.5
There are two possible outputs you might want: 47.5 as a number, or 344.5 as a string:
SELECT 3 + CAST( '44.5' as decimal(3,1))
OR...
SELECT CONCAT(CAST(3 as varchar), '44.5')
First, this page will tell you everything you need to know about this topic (and can be used to cure insomnia): Data type conversion (Database Engine)
Next, what you posted is '44. 5' that cant be converted into a number. This fails no matter what numeric data type you try:
SELECT CAST('44. 5' AS NUMERIC)
This will fail because SQL can't determine if you're aggregating numbers or concatenating text:
Select 3 + '44.6';
These will work for math:
Select 3 + 44.5;
Select 3 + CAST('44.5' AS DECIMAL(3,1));
Select CAST(3 AS DECIMAL(3,1)) + '44.5';
These will work for concatination:
Select '3' + '44.5';
Select CAST(3 AS VARCHAR(10)) + '44.5'
Select CONCAT(3, '44.5');
This question already has answers here:
Get the number of digits after the decimal point of a float (with or without decimal part)
(6 answers)
Closed 4 years ago.
The column CostPrice of table t1 is money type.
The data in column CostPrice likes this:
141.1938
0.00
147.1041
119.592
1.23
I use this sql to get the decimal digits:
select distinct len(CostPrice-floor(CostPrice))-2 from t1;
But the result is only 2,this is not right,in fact,the result should be 2,3,4.
So how to fix the sql?
Added:
StackOverflow does not allow me to flag the flagging but asked me to edit the question instead, so:
This Question is not a duplicate to this one. The existing question is on the "float" datatype and this one is on "money", Converting "money" to "varchar" like in the "existing anser" will always return the same number of decimal places, so it does not answer this question.
You could (independantly from regional settings):
multiply the value by 10,000
convert the result to integer
convert the integer to a string
add 4 leading zeroes to the left (just in case...)
take the 4 characters from the right (the former decimal places)
replace each zero by a blank character
remove the trailing blanks using rtrim
return the length of the remaining string
To put this in an expression, it would be:
LEN(RTRIM(REPLACE(RIGHT('0000' + CONVERT(varchar(20), CONVERT(int, CostPrice*10000)), 4), '0', ' ')))
Try this
DECLARE
#money money = 141.1938
SELECT
#money
,SUBSTRING(CONVERT(varchar(20),#money,2), CHARINDEX('.', CONVERT(varchar(20),#money,2)) + 1, LEN(CONVERT(varchar(20),#money,2))) as RESULT
I am importing data into my SQL database from an Excel spreadsheet.
The imp table is the imported data, the app table is the existing database table.
app.ReceiptId is formatted as "A" followed by some numbers. Formerly it was 4 digits, but now it may be 4 or 5 digits.
Examples:
A1234
A9876
A10001
imp.ref is a free-text reference field from Excel. It consists of some arbitrary length description, then the ReceiptId, followed by an irrelevant reference number in the format " - BZ-0987654321" (which is sometimes cropped short, or even missing entirely).
Examples:
SHORT DESC A1234 - BZ-0987654321
LONGER DESCRIPTION A9876 - BZ-123
REALLY LONG DESCRIPTION A2345 - B
REALLY REALLY LONG DESCRIPTION A23456
The code below works for a 4-digit ReceiptId, but will not correctly capture a 5-digit one.
UPDATE app
SET
[...]
FROM imp
INNER JOIN app
ON app.ReceiptId = right(right(rtrim(replace(replace(imp.ref,'-',''),'B','')),5)
+ rtrim(left(imp.ref,charindex(' - BZ-',imp.ref))),5)
How can I change the code so it captures either 4 (A1234) or 5 (A12345) digits?
As ughai rightfully wrote in his comment, it's not recommended to use anything other then columns in the on clause of a join.
The reason for that is that using functions prevents sql server for using any indexes on the columns that it might use without the functions.
Therefor, I would suggest adding another column to imp table that will hold the actual ReceiptId and be calculated during the import process itself.
I think the best way of extracting the ReceiptId from the ref column is using substring with patindex, as demonstrated in this fiddle:
SELECT ref,
RTRIM(SUBSTRING(ref, PATINDEX('%A[0-9][0-9][0-9][0-9]%', ref), 6)) As ReceiptId
FROM imp
Update
After the conversation with t-clausen-dk in the comments, I came up with this:
SELECT ref,
CASE WHEN PATINDEX('%[ ]A[0-9][0-9][0-9][0-9][0-9| ]%', ref) > 0
OR PATINDEX('A[0-9][0-9][0-9][0-9][0-9| ]%', ref) = 1 THEN
SUBSTRING(ref, PATINDEX('%A[0-9][0-9][0-9][0-9][0-9| ]%', ref), 6)
ELSE
NULL
END As ReceiptId
FROM imp
fiddle here
This will return null if there is no match,
when a match is a sub string that contains A followed by 4 or 5 digits, separated by spaces from the rest of the string, and can be found at the start, middle or end of the string.
Try this, it will remove all characters before the A[number][number][number][number] and take the first 6 characters after that:
UPDATE app
SET
[...]
FROM imp
INNER JOIN app
ON app.ReceiptId in
(
left(stuff(ref,1, patindex('%A[0-9][0-9][0-9][0-9][ ]%', imp.ref + ' ') - 1, ''), 5),
left(stuff(ref,1, patindex('%A[0-9][0-9][0-9][0-9][0-9][ ]%', imp.ref + ' ') - 1, ''), 6)
)
When using equal, the spaces after is not evaluated