I was having a problem with a larger query in SQL Server which I traced back to this section of code which isn't performing as expected.
SELECT item_name,item_num,rst_units
FROM tbl_item left join tbl_sales_regional on item_num=rst_item
WHERE rst_month=7 and rst_customer='AB123'
The first table (tbl_item) has 10,000 records. The second table (tbl_sales_regional) has 83 for the shown criteria.
Instead of returning 10,000 records with 9,917 nulls, the execution plan shows SQL Server has rewritten as an inner join and consequently returns 83 results.
In order to achieve the intended result, I have to join off a subquery. Can someone provide an explanation why this doesn't work?
Not sure which fields belong where, but you seem to have some fields from tbl_sales_regional in your WHERE condition.
Move them into the ON clause:
SELECT item_name, item_num, rst_units
FROM tbl_item
LEFT JOIN
tbl_sales_regional
ON rst_item = item_num
AND rst_month = 7
AND rst_customer = 'AB123'
The WHERE clause operates on the results of the join so these conditions cannot possibly hold true for any NULL records from tbl_sales_regional returned by the join, as NULL cannot be equal to anything.
That's why the optimizer transforms your query into the inner join.
Any conditions you have in your WHERE clause are applied regardless of the left join, effectively making it an inner join.
You need to change it to:
SELECT item_name,item_num,rst_units
FROM tbl_item left join tbl_sales_regional on item_num=rst_item
AND rst_month=7 and rst_customer='AB123'
Related
Using Microsoft T-SQL, I know there is a difference in the result set between putting a filter in the join verses putting the filter in the where clause, because I get a different row count, and its hard to pin point, because it is a complex query with thousands of rows.
So does anybody know how this might lead to different results?
I feel safer with the where clause, but I am curious why the other one didn't get the same results.
Select
.....
left outer join
datamart_agent ag on a.CUSTOMER_TKN = ag.customer_tkn
and a.CUSTOMER_ACCT_TKN = ag.customer_acct_tkn
and a.ACCOUNT_PKG_TKN = ag.account_pkg_tkn
--where (commented out)
and ag.TYPE_DESC = 'Agent'
versus
Select
.....
left outer join
datamart_agent ag on a.CUSTOMER_TKN = ag.customer_tkn
and a.CUSTOMER_ACCT_TKN = ag.customer_acct_tkn
and a.ACCOUNT_PKG_TKN = ag.account_pkg_tkn
where
--and (commented out)
ag.TYPE_DESC = 'Agent'
If condition is in your where clause then it will drop all records for which no match was found. It is a like an eraser. I deletes all records for which no match was found (Type_Desc is null).
The safer thing, if you are actually seeking left join, is to put the condition in the on clause. If there was no match, it will still come in the result. When Type_Desc is null, it means you don't know who the agent is, but if you don't know, you don't want to delete the record....
SELECT
b.1, b.2, b.3, b.4, a.4, a.3, a.5
FROM
a
RIGHT JOIN
b ON a.id = b.id
This query is taking more than 7 minutes.
Both tables have around 100 000 records and just a select from each table runs around 12 seconds avg. In execution plan it is saying that table a has logical reads of around 8708 and 100% operator cost. Both tables have CI on ID.
Verify that an INDEX on the ID column exists in table A. For each row selected in B there will be a lookup of rows in A on the ID column. If an index does not exist on that column this will result in a table scan i.e. a lookup through 100k rows to find rows with that specific ID. Not efficient.
PS - General advice: write queries that don't use RIGHT JOIN, stick to INNER, LEFT and OUTER joins unless there is no other way (there almost always is).
Use this sql below to help you identify any missing indexes. My guess is you are missing at least one.
SELECT
statement AS [database.scheme.table],
column_id , column_name, column_usage,
migs.user_seeks, migs.user_scans,
migs.last_user_seek, migs.avg_total_user_cost,
migs.avg_user_impact
FROM sys.dm_db_missing_index_details AS mid
CROSS APPLY sys.dm_db_missing_index_columns (mid.index_handle)
INNER JOIN sys.dm_db_missing_index_groups AS mig
ON mig.index_handle = mid.index_handle
INNER JOIN sys.dm_db_missing_index_group_stats AS migs
ON mig.index_group_handle=migs.group_handle
ORDER BY mig.index_group_handle, mig.index_handle, column_id
I have this ugly source data with two columns, let's call them EmpID and SomeCode. Generally EmpID maps to the EmployeeListing table. But sometimes, people are entering the Employee IDs in the SomeCode field.
The person previously running this report in Excel 'solved' this problem by performing multiple vlookups with if statements, as well as running some manual checks to ensure results were accurate. As I'm moving these files to Access I am not sure how best to handle this scenario.
Ideally, I'm hoping to tell my queries to do a Left Join on SomeCode if EmpID is null, otherwise Left Join on EmpID
Unfortunately, there's no way for me to force validation or anything of the sort in the source data.
Here's the full SQL query I'm working on:
SELECT DDATransMaster.Fulfillment,
DDATransMaster.ConfirmationNumber,
DDATransMaster.PromotionCode,
DDATransMaster.DirectSellerNumber,
NZ([DDATransMaster]![DirectSellerNumber],[DDATransMaster]![PromotionCode]) AS EmpJoin,
EmployeeLookup.ID AS EmpLookup,
FROM FROM DDATransMaster
LEFT JOIN EmployeeLookup ON NZ([DDATransMaster]![DirectSellerNumber],[DDATransMaster]![PromotionCode]) = EmployeeLookup.[Employee #])
You can create a query like this:
SELECT
IIf(EmpID Is Null, SomeCode, EmpID) AS join_field,
field2,
etc
FROM YourTable
Or if the query will always be used within an Access session, Nz is more concise.
SELECT
Nz(EmpID, SomeCode) AS join_field,
field2,
etc
FROM YourTable
When you join that query to your other table, the Access query designer can represent the join between join_field and some matching field in the other table. If you were to attempt the IIf or Nz as part of the join's ON clause, the query designer can't display the join correctly in Design View --- it could still work, but may not be as convenient if you're new to Access SQL.
See whether this SQL gives you what you want.
SELECT
dda.Fulfillment,
dda.ConfirmationNumber,
dda.PromotionCode,
dda.DirectSellerNumber,
NZ(dda.DirectSellerNumber,dda.PromotionCode) AS EmpJoin,
el.ID AS EmpLookup
FROM
DDATransMaster AS dda
LEFT JOIN EmployeeLookup AS el
ON NZ(dda.DirectSellerNumber,dda.PromotionCode) = el.[Employee #])
But I would use the Nz part in a subquery.
SELECT
sub.Fulfillment,
sub.ConfirmationNumber,
sub.PromotionCode,
sub.DirectSellerNumber,
sub.EmpJoin,
el.ID AS EmpLookup
FROM
(
SELECT
Fulfillment,
ConfirmationNumber,
PromotionCode,
DirectSellerNumber,
NZ(DirectSellerNumber,PromotionCode) AS EmpJoin
FROM DDATransMaster
) AS sub
LEFT JOIN EmployeeLookup AS el
ON sub.EmpJoin = el.[Employee #])
What about:
LEFT JOIN EmployeeListing ON NZ(EmpID, SomeCode)
as your join, nz() uses the second parameter if the first is null, I'm not 100% sure this sort of join works in access. Worth 20 seconds to try though.
Hope it works.
You Could use a Union:
SELECT DDATransMaster.Fulfillment,
DDATransMaster.ConfirmationNumber,
DDATransMaster.PromotionCode,
DDATransMaster.DirectSellerNumber,
EmployeeLookup.ID AS EmpLookup
FROM DDATransMaster
LEFT JOIN EmployeeLookup ON
DDATransMaster.DirectSellerNumber = EmployeeLookup.[Employee #]
where DDATransMaster.DirectSellerNumber IS NOT NULL
Union
SELECT DDATransMaster.Fulfillment,
DDATransMaster.ConfirmationNumber,
DDATransMaster.PromotionCode,
DDATransMaster.DirectSellerNumber,
EmployeeLookup.ID AS EmpLookup
FROM DDATransMaster
LEFT JOIN EmployeeLookup ON
DDATransMaster.PromotionCode = EmployeeLookup.[Employee #]
where DDATransMaster.DirectSellerNumber IS NULL;
If you add more than one table to the FROM clause (in a query), how does this impact the result set? Does it first select from the first table then from the second and then create a union (i.e., only the rowspace is impacted?) or does it actually do something like a join (i.e., extend the column space)? And when you use multiple tables in the FROM clause, does the WHERE clause filter both sub-result-sets?
Specifying two tables in your FROM clause will execute a JOIN. You can then use the WHERE clause to specify your JOIN conditions. If you fail to do this, you will end-up with a Cartesian product (every row in the first table indiscriminately joined to every row in the second).
The code will look something like this:
SELECT a.*, b.*
FROM table1 a, table2 b
WHERE a.id = b.id
However, I always try to explicitly specify my JOINs (with JOIN and ON keywords). That makes it abundantly clear (for the next developer) as to what you're trying to do. Here's the same JOIN, but explicitly specified:
SELECT a.*, b.*
FROM table1 a
INNER JOIN table2 b ON b.id = a.id
Note that now I don't need a WHERE clause. This method also helps you avoid generating an inadvertent Cartesian product (if you happen to forget your WHERE clause), because the ON is specified explicitly.
I am building a view in SQL Server 2000 (and 2005) and I've noticed that the order of the join statements greatly affects the execution plan and speed of the query.
select sr.WTSASessionRangeID,
-- bunch of other columns
from WTSAVW_UserSessionRange us
inner join WTSA_SessionRange sr on sr.WTSASessionRangeID = us.WTSASessionRangeID
left outer join WTSA_SessionRangeTutor srt on srt.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeClass src on src.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeStream srs on srs.WTSASessionRangeID = sr.WTSASessionRangeID
--left outer join MO_Stream ms on ms.MOStreamID = srs.MOStreamID
left outer join WTSA_SessionRangeEnrolmentPeriod srep on srep.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeStudent stsd on stsd.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionSubrange ssr on ssr.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionSubrangeRoom ssrr on ssrr.WTSASessionSubrangeID = ssr.WTSASessionSubrangeID
left outer join MO_Stream ms on ms.MOStreamID = srs.MOStreamID
On SQL Server 2000, the query above consistently generates a plan of cost 946. If I uncomment the MO_Stream join in the middle of the query and comment out the one at the bottom, the cost drops to 263. The execution speed drops accordingly. I always thought that the query optimizer would interpret the query appropriately without considering join order, but it seems that order matters.
So since order does seem to matter, is there a join strategy I should be following for writing faster queries?
(Incidentally, on SQL Server 2005, with almost identical data, the query plan costs were 0.675 and 0.631 respectively.)
Edit: On SQL Server 2000, here are the profiled stats:
946-cost query: 9094ms CPU, 5121 reads, 0 writes, 10123ms duration
263-cost query: 172ms CPU, 7477 reads, 0 writes, 170ms duration
Edit: Here is the logical structure of the tables.
SessionRange ---+--- SessionRangeTutor
|--- SessionRangeClass
|--- SessionRangeStream --- MO_Stream
|--- SessionRangeEnrolmentPeriod
|--- SessionRangeStudent
+----SessionSubrange --- SessionSubrangeRoom
Edit: Thanks to Alex and gbn for pointing me in the right direction. I also found this question.
Here's the new query:
select sr.WTSASessionRangeID // + lots of columns
from WTSAVW_UserSessionRange us
inner join WTSA_SessionRange sr on sr.WTSASessionRangeID = us.WTSASessionRangeID
left outer join WTSA_SessionRangeTutor srt on srt.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeClass src on src.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeEnrolmentPeriod srep on srep.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join WTSA_SessionRangeStudent stsd on stsd.WTSASessionRangeID = sr.WTSASessionRangeID
// SessionRangeStream is a many-to-many mapping table between SessionRange and MO_Stream
left outer join (
WTSA_SessionRangeStream srs
inner join MO_Stream ms on ms.MOStreamID = srs.MOStreamID
) on srs.WTSASessionRangeID = sr.WTSASessionRangeID
// SessionRanges MAY have Subranges and Subranges MAY have Rooms
left outer join (
WTSA_SessionSubrange ssr
left outer join WTSA_SessionSubrangeRoom ssrr on ssrr.WTSASessionSubrangeID = ssr.WTSASessionSubrangeID
) on ssr.WTSASessionRangeID = sr.WTSASessionRangeID
SQLServer2000 cost: 24.9
I have to disagree with all previous answers, and the reason is simple: if you change the order of your left join, your queries are logically different and as such they produce different result sets. See for yourself:
SELECT 1 AS a INTO #t1
UNION ALL SELECT 2
UNION ALL SELECT 3
UNION ALL SELECT 4;
SELECT 1 AS b INTO #t2
UNION ALL SELECT 2;
SELECT 1 AS c INTO #t3
UNION ALL SELECT 3;
SELECT a, b, c
FROM #t1 LEFT JOIN #t2 ON #t1.a=#t2.b
LEFT JOIN #t3 ON #t2.b=#t3.c
ORDER BY a;
SELECT a, b, c
FROM #t1 LEFT JOIN #t3 ON #t1.a=#t3.c
LEFT JOIN #t2 ON #t3.c=#t2.b
ORDER BY a;
a b c
----------- ----------- -----------
1 1 1
2 2 NULL
3 NULL NULL
4 NULL NULL
(4 row(s) affected)
a b c
----------- ----------- -----------
1 1 1
2 NULL NULL
3 NULL 3
4 NULL NULL
The join order does make a difference to the resulting query. This is documented in BOL in the docs for FROM:
<joined_table>
Is a result set that is the product of two or more tables. For multiple joins, use parentheses to change the natural order of the joins.
You can alter the join order using parenthesis around the joins (BOL does show this in the syntax at the top of the docs, but it is easy to miss).
This is known as chiastic behaviour. You can also use the query hint OPTION (FORCE ORDER) to force a specific join order, but this can result in what are called "bushy plans" which may not be the most optimal for the query being executed.
Obviously, the SQL Server 2005 optimizer is a lot better than the SQL Server 2000 one.
However, there's a lot of truth in your question. Outer joins will cause execution to vary wildly based on order (inner joins tend to be optimized to the most efficient route, but again, order matters). If you think about it, as you build up left joins, you need to figure out what the heck is on the left. As such, each join must be calculated before every other join can be done. It becomes sequential, and not parallel. Now, obviously, there are things you can do to combat this (such as indexes, views, etc). But, the point stands: The table needs to know what's on the left before it can do a left outer join. And if you just keep adding joins, you're getting more and more abstraction to what, exactly is on the left (especially if you use joined tables as the left table!).
With inner joins, however, you can parallelize those quite a bit, so there's less of a dramatic difference as far as order's concerned.
A general strategy for optimizing queries containing JOINs is to look at your data model and the data and try to determine which JOINs will reduce number of records that must be considered the most quickly. The fewer records that must be considered, the faster the query will run. The server will generally produce a better query plan too.
Along with the above optimization make sure that any fields used in JOINs are indexed
You query is probably wrong anyway. Alex is correct. Eric may be correct too, but the query is wrong.
Lets' take this subset:
WTSA_SessionRange sr
left outer join
WTSA_SessionSubrange ssr on ssr.WTSASessionRangeID = sr.WTSASessionRangeID
left outer join
WTSA_SessionSubrangeRoom ssrr on ssrr.WTSASessionSubrangeID = ssr.WTSASessionSubrangeID
You are joining WTSA_SessionSubrangeRoom onto WTSA_SessionSubrange. You may have no rows from WTSA_SessionSubrange.
The join should be this:
WTSA_SessionRange sr
left outer join
(SELECT WTSASessionRangeID, columns I need
FROM
WTSA_SessionSubrange ssr
left outer join
WTSA_SessionSubrangeRoom ssrr on ssrr.WTSASessionSubrangeID = ssr.WTSASessionSubrangeID
) foo on foo.WTSASessionRangeID = sr.WTSASessionRangeID
This is why the join order is affecting results because it's a different query, declaratively speaking.
You'd also need to change the MO_Stream and WTSA_SessionRangeStream join too.
it depends on which of the join fields are indexed - if it has to table scan the first field, but use an index on the second, it's slow. If your first join field is an index, it'll be quicker. My guess is that 2005 optimizes it better by determining the indexed fields and performing those first
At DevConnections a few years ago a session on SQL Server performance stated that (a) order of outer joins DOES matter, and (b) when a query has a lot of joins, it will not look at all of them before making a determination on a plan. If you know you have joins that will help speed up a query, they should be early on in the FROM list (if you can).