Snowflake: Unsupported subquery for DISTINCT - Column order matters? - snowflake-cloud-data-platform

I have two related tables (unnecessary columns not listed):
LOCATION
VENUE_ID - NUMBER(38,0)
VISIT
ID - NUMBER(38,0)
VENUE_ID - NUMBER(38,0)
DEVICE_ID - VARCHAR(16777216)
The tables are related such that visits are associated with a location via VENUE_ID.
I'm attempting to get the count of unique device ids by location, so I wrote the following query:
SELECT "d"."VENUE_ID"
, (
SELECT COUNT(*)
FROM (
SELECT DISTINCT "f0"."DEVICE_ID"
FROM "MAIN"."VISIT" AS "f0"
WHERE "d"."VENUE_ID" = "f0"."VENUE_ID"
) AS "t")
FROM "MAIN"."LOCATION" AS "d"
Unfortunately, this query resulted in the cryptic error SQL compilation error: Unsupported subquery type cannot be evaluated.
Through a bit of experimentation, I've found that I can get the query to return without error, but only if I add an additional (useless) subquery prior to the existing one in the SELECT:
SELECT "d"."VENUE_ID"
-- New Useless Subquery
, (
SELECT COUNT(*)
FROM "MAIN"."VISIT" AS "f"
WHERE "d"."VENUE_ID" = "f"."VENUE_ID")
--
, (
SELECT COUNT(*)
FROM (
SELECT DISTINCT "f0"."DEVICE_ID"
FROM "MAIN"."VISIT" AS "f0"
WHERE "d"."VENUE_ID" = "f0"."VENUE_ID"
) AS "t")
FROM "MAIN"."LOCATION" AS "d"
If I move the new subquery to anywhere in the select after the distinct subquery, the error returns. I've reviewed the documentation on subqueries in Snowflake and either I am not understanding how that applies to my query here or I'm facing undocumented behavior. Anyone have any idea what's going on here?

I think you're making this more complex than this needs to be. Below should be all you need:
SELECT l.venue_id
, count(distinct v.device_id)
FROM location l
LEFT JOIN visit v
on l.venue_id = v.venue_id
GROUP BY l.venue_id

The answer is a little cryptic, but what happens is this:
You are asking for ONE value and you need to guarantee that only ONE value is returned by your subquery. A distinct clause cannot guarantee that. In some databases that will work as long as the data returns one row, but the moment you get two rows then the database will throw an error.
Snowflake is strict on its subquery analysis. So you need to use a subquery that is guarantee to return always one value, for example select sum(..), select count(..)

Related

Subquery for simple addition?

I'm using the Northwind sample database.
PROBLEM: show the product names for products that have been ordered in quantities equal to or exceeding 120. Use a non-correlated subquery in the SQL statement.
Question: how can I write this as a non-correlated subquery? This simple statement I created produces the correct results:
SELECT [ProductName]
FROM Products
WHERE ([UnitsInStock] + [UnitsOnOrder] > = 120)
My use of a subquery results in error. This is my best attempt:
SELECT [ProductName]
FROM [dbo].[Products]
WHERE 120 < =
(SELECT [UnitsInStock] + [UnitsOnOrder]
FROM Products)
Error:
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
Do you understand WHY your query does not work? You know that the subquery returns multiple rows. The problem with your usage is that you cannot compare a scalar value to a logical table (the resultset of your query). This is a very common mistake.
I believe that this is the most likely solution intended for your exercise. The derived table solution (posted first) is not really a subquery.
select prd.ProductName
from dbo.Products as prd
where prd.ID in (select px.ID from dbo.Products as px
where px.UnitsInStock + px.UnitsOnOrder > = 120)
order by ...;
Some things to note.
You should always schema-qualify your object names.
Though subjective, IMO you should avoid adding square brackets as object name delimiters when they are not needed. That just makes your code more difficult to read.
Generally resultsets should be ordered. I added the clause but left it unfinished.
Get in the good habit of using statement terminators. Eventually they will be required.
I have no idea what the primary key is for the Products table but I assumed it is not the name column. I guessed it was ID - replace as necessary. Name might be an acceptable alternative if it is declared as unique and not nullable.
The issue you're having is that you haven't realized that SQL can be declarative - you can pretty much select from ANYTHING, you just have to get used to how to do it.
Also the error you're getting is you're treating you where clause sub query as correlated.
Check this:
SELECT ProductName
FROM
(
SELECT [ProductName]
FROM Products p
WHERE ([UnitsInStock] + [UnitsOnOrder] > = 120)
) x
Okay, so you are on the right path. The question is from the A set give me the set b with an order quantity equal or greater than 120. So your A set will only be products. the b set will be the products with quantity ordered >=120
Select product from A where productid in ( select productid from b where b.quantity >=120)
A is the product table,
b is your orders table where the product and quantity are been associated.
you can use the IN operator to filter the A set

Select where id not in (...) has any limit? - NOT IN VS NOT EXISTS

I want to select some rows from a catalog which are not related to two table (Table1 and Table2).
The rows in the catalog are related to the tables using the field idA.
I am using the following query:
;with CTE(id) AS(
select distinct idA from Table1
union
select distinct idA from Table2
)
select * from Catalog where IdA NOT IN (select id from CTE)
The very strange part is that this query does not return any result.
If I use the following query after de CTE it will return some rows
select Id from Catalog where not exists (select 1 from CTE where Catalog.IdA = CTE.Id);
Does anyone know what is the reason of this? AFAK both queries are equivalents.
Of course the first query is slower than the second one but that is not very important. The important part is, why those queries do not return the same results?
Maybe it's important to mention that Table1 has more than 3.9 million rows
but just 139,283 different values for idA.
Table2 has only some thousands rows
Any help or comment will be appreciated
If one or more NULL values are returned by a NOT IN subquery, the result of the predicate is unknown rather than true or false and no rows returned due to the unsatisfied condition. It is best to avoid NOT IN with nullable expressions to avoid this non-intuitive behavior.
Although slighly more verbose, a correlated NOT EXISTS subquery will always return true or false and avoid the NULL gotcha.

sub-query return more than 1 value this is not permitted when the subquery follows = , >=,>,<,<= [duplicate]

I run the following query:
SELECT
orderdetails.sku,
orderdetails.mf_item_number,
orderdetails.qty,
orderdetails.price,
supplier.supplierid,
supplier.suppliername,
supplier.dropshipfees,
cost = (SELECT supplier_item.price
FROM supplier_item,
orderdetails,
supplier
WHERE supplier_item.sku = orderdetails.sku
AND supplier_item.supplierid = supplier.supplierid)
FROM orderdetails,
supplier,
group_master
WHERE invoiceid = '339740'
AND orderdetails.mfr_id = supplier.supplierid
AND group_master.sku = orderdetails.sku
I get the following error:
Msg 512, Level 16, State 1, Line 2
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
Any ideas?
Try this:
SELECT
od.Sku,
od.mf_item_number,
od.Qty,
od.Price,
s.SupplierId,
s.SupplierName,
s.DropShipFees,
si.Price as cost
FROM
OrderDetails od
INNER JOIN Supplier s on s.SupplierId = od.Mfr_ID
INNER JOIN Group_Master gm on gm.Sku = od.Sku
INNER JOIN Supplier_Item si on si.SKU = od.Sku and si.SupplierId = s.SupplierID
WHERE
od.invoiceid = '339740'
This will return multiple rows that are identical except for the cost column. Look at the different cost values that are returned and figure out what is causing the different values. Then ask somebody which cost value they want, and add the criteria to the query that will select that cost.
Check to see if there are any triggers on the table you are trying to execute queries against. They can sometimes throw this error as they are trying to run the update/select/insert trigger that is on the table.
You can modify your query to disable then enable the trigger if the trigger DOES NOT need to be executed for whatever query you are trying to run.
ALTER TABLE your_table DISABLE TRIGGER [the_trigger_name]
UPDATE your_table
SET Gender = 'Female'
WHERE (Gender = 'Male')
ALTER TABLE your_table ENABLE TRIGGER [the_trigger_name]
SELECT COLUMN
FROM TABLE
WHERE columns_name
IN ( SELECT COLUMN FROM TABLE WHERE columns_name = 'value');
note: when we are using sub-query we must focus on these points:
if our sub query returns 1 value in this case we need to use (=,!=,<>,<,>....)
else (more than one value), in this case we need to use (in, any, all, some )
cost = Select Supplier_Item.Price from Supplier_Item,orderdetails,Supplier
where Supplier_Item.SKU=OrderDetails.Sku and
Supplier_Item.SupplierId=Supplier.SupplierID
This subquery returns multiple values, SQL is complaining because it can't assign multiple values to cost in a single record.
Some ideas:
Fix the data such that the existing subquery returns only 1 record
Fix the subquery such that it only returns one record
Add a top 1 and order by to the subquery (nasty solution that DBAs hate - but it "works")
Use a user defined function to concatenate the results of the subquery into a single string
The fix is to stop using correlated subqueries and use joins instead. Correlated subqueries are essentially cursors as they cause the query to run row-by-row and should be avoided.
You may need a derived table in the join in order to get the value you want in the field if you want only one record to match, if you need both values then the ordinary join will do that but you will get multiple records for the same id in the results set. If you only want one, you need to decide which one and do that in the code, you could use a top 1 with an order by, you could use max(), you could use min(), etc, depending on what your real requirement for the data is.
I had the same problem , I used in instead of = , from the Northwind database example :
Query is : Find the Companies that placed orders in 1997
Try this :
SELECT CompanyName
FROM Customers
WHERE CustomerID IN (
SELECT CustomerID
FROM Orders
WHERE YEAR(OrderDate) = '1997'
);
Instead of that :
SELECT CompanyName
FROM Customers
WHERE CustomerID =
(
SELECT CustomerID
FROM Orders
WHERE YEAR(OrderDate) = '1997'
);
Either your data is bad, or it's not structured the way you think it is. Possibly both.
To prove/disprove this hypothesis, run this query:
SELECT * from
(
SELECT count(*) as c, Supplier_Item.SKU
FROM Supplier_Item
INNER JOIN orderdetails
ON Supplier_Item.sku = orderdetails.sku
INNER JOIN Supplier
ON Supplier_item.supplierID = Supplier.SupplierID
GROUP BY Supplier_Item.SKU
) x
WHERE c > 1
ORDER BY c DESC
If this returns just a few rows, then your data is bad. If it returns lots of rows, then your data is not structured the way you think it is. (If it returns zero rows, I'm wrong.)
I'm guessing that you have orders containing the same SKU multiple times (two separate line items, both ordering the same SKU).
The select statement in the cost part of your select is returning more than one value. You need to add more where clauses, or use an aggregation.
The error implies that this subquery is returning more than 1 row:
(Select Supplier_Item.Price from Supplier_Item,orderdetails,Supplier where Supplier_Item.SKU=OrderDetails.Sku and Supplier_Item.SupplierId=Supplier.SupplierID )
You probably don't want to include the orderdetails and supplier tables in the subquery, because you want to reference the values selected from those tables in the outer query. So I think you want the subquery to be simply:
(Select Supplier_Item.Price from Supplier_Item where Supplier_Item.SKU=OrderDetails.Sku and Supplier_Item.SupplierId=Supplier.SupplierID )
I suggest you read up on correlated vs. non-correlated subqueries.
As others have suggested, the best way to do this is to use a join instead of variable assignment. Re-writing your query to use a join (and using the explicit join syntax instead of the implicit join, which was also suggested--and is the best practice), you would get something like this:
select
OrderDetails.Sku,
OrderDetails.mf_item_number,
OrderDetails.Qty,
OrderDetails.Price,
Supplier.SupplierId,
Supplier.SupplierName,
Supplier.DropShipFees,
Supplier_Item.Price as cost
from
OrderDetails
join Supplier on OrderDetails.Mfr_ID = Supplier.SupplierId
join Group_Master on Group_Master.Sku = OrderDetails.Sku
join Supplier_Item on
Supplier_Item.SKU=OrderDetails.Sku and Supplier_Item.SupplierId=Supplier.SupplierID
where
invoiceid='339740'
Even after 9 years of the original post, this helped me.
If you are receiving these types of errors without any clue, there should be a trigger, function related to the table, and obviously it should end up with an SP, or function with selecting/filtering data NOT USING Primary Unique column. If you are searching/filtering using the Primary Unique column there won't be any multiple results. Especially when you are assigning value for a declared variable. The SP never gives you en error but only an runtime error.
"System.Data.SqlClient.SqlException (0x80131904): Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
The statement has been terminated."
In my case obviously there was no clue, but only this error message. There was a trigger connected to the table and the table updating by the trigger also had another trigger likewise it ended up with two triggers and in the end with an SP. The SP was having a select clause which was resulting in multiple rows.
SET #Variable1 =(
SELECT column_gonna_asign
FROM dbo.your_db
WHERE Non_primary_non_unique_key= #Variable2
If this returns multiple rows, you are in trouble.

SQL Server : group all data by one column

I need some help in writing a SQL Server stored procedure. All data group by Train_B_N.
my table data
Expected result :
expecting output
with CTE as
(
select Train_B_N, Duration,Date,Trainer,Train_code,Training_Program
from Train_M
group by Train_B_N
)
select
*
from Train_M as m
join CTE as c on c.Train_B_N = m.Train_B_N
whats wrong with my query?
The GROUP BY smashes the table together, so having columns that are not GROUPED combine would cause problems with the data.
select Train_B_N, Duration,Date,Trainer,Train_code,Training_Program
from Train_M
group by Train_B_N
By ANSI standard, the GROUP BY must include all columns that are in the SELECT statement which are not in an aggregate function. No exceptions.
WITH CTE AS (SELECT TRAIN_B_N, MAX(DATE) AS Last_Date
FROM TRAIN_M
GROUP BY TRAIN_B_N)
SELECT A.Train_B_N, Duration, Date,Trainer,Train_code,Training_Program
FROM TRAIN_M AS A
INNER JOIN CTE ON CTE.Train_B_N = A.Train_B_N
AND CTE.Last_Date = A.Date
This example would return the last training program, trainer, train_code used by that ID.
This is accomplished from MAX(DATE) aggregate function, which kept the greatest (latest) DATE in the table. And since the GROUP BY smashed the rows to their distinct groupings, the JOIN only returns a subset of the table's results.
Keep in mind that SQL will return #table_rows X #Matching_rows, and if your #Matching_rows cardinality is greater than one, you will get extra rows.
Look up GROUP BY - MSDN. I suggest you read everything outside the syntax examples initially and obsorb what the purpose of the clause is.
Also, next time, try googling your problem like this: 'GROUP BY, SQL' or insert the error code given by your IDE (SSMS or otherwise). You need to understand why things work...and SO is here to help, not be your google search engine. ;)
Hope you find this begins your interest in learning all about SQL. :D

How does DISTINCT work in SQL Server 2008 R2? Are there other options? [duplicate]

I need to retrieve all rows from a table where 2 columns combined are all different. So I want all the sales that do not have any other sales that happened on the same day for the same price. The sales that are unique based on day and price will get updated to an active status.
So I'm thinking:
UPDATE sales
SET status = 'ACTIVE'
WHERE id IN (SELECT DISTINCT (saleprice, saledate), id, count(id)
FROM sales
HAVING count = 1)
But my brain hurts going any farther than that.
SELECT DISTINCT a,b,c FROM t
is roughly equivalent to:
SELECT a,b,c FROM t GROUP BY a,b,c
It's a good idea to get used to the GROUP BY syntax, as it's more powerful.
For your query, I'd do it like this:
UPDATE sales
SET status='ACTIVE'
WHERE id IN
(
SELECT id
FROM sales S
INNER JOIN
(
SELECT saleprice, saledate
FROM sales
GROUP BY saleprice, saledate
HAVING COUNT(*) = 1
) T
ON S.saleprice=T.saleprice AND s.saledate=T.saledate
)
If you put together the answers so far, clean up and improve, you would arrive at this superior query:
UPDATE sales
SET status = 'ACTIVE'
WHERE (saleprice, saledate) IN (
SELECT saleprice, saledate
FROM sales
GROUP BY saleprice, saledate
HAVING count(*) = 1
);
Which is much faster than either of them. Nukes the performance of the currently accepted answer by factor 10 - 15 (in my tests on PostgreSQL 8.4 and 9.1).
But this is still far from optimal. Use a NOT EXISTS (anti-)semi-join for even better performance. EXISTS is standard SQL, has been around forever (at least since PostgreSQL 7.2, long before this question was asked) and fits the presented requirements perfectly:
UPDATE sales s
SET status = 'ACTIVE'
WHERE NOT EXISTS (
SELECT FROM sales s1 -- SELECT list can be empty for EXISTS
WHERE s.saleprice = s1.saleprice
AND s.saledate = s1.saledate
AND s.id <> s1.id -- except for row itself
)
AND s.status IS DISTINCT FROM 'ACTIVE'; -- avoid empty updates. see below
db<>fiddle here
Old sqlfiddle
Unique key to identify row
If you don't have a primary or unique key for the table (id in the example), you can substitute with the system column ctid for the purpose of this query (but not for some other purposes):
AND s1.ctid <> s.ctid
Every table should have a primary key. Add one if you didn't have one, yet. I suggest a serial or an IDENTITY column in Postgres 10+.
Related:
In-order sequence generation
Auto increment table column
How is this faster?
The subquery in the EXISTS anti-semi-join can stop evaluating as soon as the first dupe is found (no point in looking further). For a base table with few duplicates this is only mildly more efficient. With lots of duplicates this becomes way more efficient.
Exclude empty updates
For rows that already have status = 'ACTIVE' this update would not change anything, but still insert a new row version at full cost (minor exceptions apply). Normally, you do not want this. Add another WHERE condition like demonstrated above to avoid this and make it even faster:
If status is defined NOT NULL, you can simplify to:
AND status <> 'ACTIVE';
The data type of the column must support the <> operator. Some types like json don't. See:
How to query a json column for empty objects?
Subtle difference in NULL handling
This query (unlike the currently accepted answer by Joel) does not treat NULL values as equal. The following two rows for (saleprice, saledate) would qualify as "distinct" (though looking identical to the human eye):
(123, NULL)
(123, NULL)
Also passes in a unique index and almost anywhere else, since NULL values do not compare equal according to the SQL standard. See:
Create unique constraint with null columns
OTOH, GROUP BY, DISTINCT or DISTINCT ON () treat NULL values as equal. Use an appropriate query style depending on what you want to achieve. You can still use this faster query with IS NOT DISTINCT FROM instead of = for any or all comparisons to make NULL compare equal. More:
How to delete duplicate rows without unique identifier
If all columns being compared are defined NOT NULL, there is no room for disagreement.
The problem with your query is that when using a GROUP BY clause (which you essentially do by using distinct) you can only use columns that you group by or aggregate functions. You cannot use the column id because there are potentially different values. In your case there is always only one value because of the HAVING clause, but most RDBMS are not smart enough to recognize that.
This should work however (and doesn't need a join):
UPDATE sales
SET status='ACTIVE'
WHERE id IN (
SELECT MIN(id) FROM sales
GROUP BY saleprice, saledate
HAVING COUNT(id) = 1
)
You could also use MAX or AVG instead of MIN, it is only important to use a function that returns the value of the column if there is only one matching row.
If your DBMS doesn't support distinct with multiple columns like this:
select distinct(col1, col2) from table
Multi select in general can be executed safely as follows:
select distinct * from (select col1, col2 from table ) as x
As this can work on most of the DBMS and this is expected to be faster than group by solution as you are avoiding the grouping functionality.
I want to select the distinct values from one column 'GrondOfLucht' but they should be sorted in the order as given in the column 'sortering'. I cannot get the distinct values of just one column using
Select distinct GrondOfLucht,sortering
from CorWijzeVanAanleg
order by sortering
It will also give the column 'sortering' and because 'GrondOfLucht' AND 'sortering' is not unique, the result will be ALL rows.
use the GROUP to select the records of 'GrondOfLucht' in the order given by 'sortering
SELECT GrondOfLucht
FROM dbo.CorWijzeVanAanleg
GROUP BY GrondOfLucht, sortering
ORDER BY MIN(sortering)

Resources