I have been working on converting an Access database over to MS SQL. For my initial testing I have imported the back end data onto my system with SQL Server Express 2014. So far I have been able to get everything to work with my Access front end except for one query.
Setting the primary keys on the tables in question has helped some but not fully. When I run the query in Access it will take about 10 seconds to run, but when I run it from a second computer it will take upwards to 30 seconds to run. However if I run the query directly Server Management Studio it runs within a second.
I'm not sure if the slow down is due to the fact I'm running SQL off of my laptop, because it's SQL Server Express, or a combination of the two. I'm hoping someone can provide some more information for me.
Here is a copy of the query:
SELECT tbl_defects.*,
tbl_parts.part_type,
tbl_parts.number,
tbl_parts.mold,
tbl_parts.date_created,
tbl_parts.blade,
tbl_parts.product,
tbl_defects.defects_id
FROM tbl_parts
RIGHT JOIN (tbl_dispositions
RIGHT JOIN tbl_defects
ON tbl_dispositions.dispositions_id =
tbl_defects.disposition)
ON tbl_parts.parts_id = tbl_defects.part
ORDER BY tbl_defects.defects_id DESC;
On the tbl_Defects the primary key is Defects_ID and it is set to index. On tbl_disposition the primary key is Disposition_ID and it is set to index. The third table tbl_parts the primary key is Parts_ID and it is set to index as well.
If I switch any of the Right Joins to be Inner Joins the query will run properly but it will be missing about 2000 records.
It has been my experience too that some queries with multiple LEFT (or RIGHT) JOINs perform badly when run in Access on linked tables.
I suggest creating a Pass-Through query for this, if possible (if you don't need to edit the result set). This will run directly on the server, as it would in SSMS.
Or you could create a view on SQL Server and link that.
Can you try running the query using the more convention left join approach?
FROM tbl_Defects LEFT JOIN
tbl_Dispositions
ON tbl_Dispositions.Dispositions_ID = tbl_Defects.Disposition LEFT JOIN
tbl_Parts
ON tbl_Parts.Parts_ID = tbl_Defects.Part
If you need parentheses:
FROM (tbl_Defects LEFT JOIN
tbl_Dispositions
ON tbl_Dispositions.Dispositions_ID = tbl_Defects.Disposition
) LEFT JOIN
tbl_Parts
ON tbl_Parts.Parts_ID = tbl_Defects.Part
Perhaps something is getting garbled in the execution plan due to the joins. RIGHT JOIN can be quite tricky, because the parsing of the FROM clause goes from left to right. I'm pretty sure this will return the same result set, however.
Related
I have a query that looks essentially like this:
SELECT *
FROM [TheDB].[dbo].[TheTable] [T1]
INNER JOIN [TheDB].[dbo].[TheTable] [T2] ON [T1].[Col] = [T2].[Col]
WHERE
[T1].[ID] > [T2].[ID]
AND [T1].[Type] = 'X'
AND [T2].[Type] = 'X'
A fairly simple query designed to find some cases where we have some duplicate data. (The > is used rather than <> just to keep from having the instances show up twice.)
We're explicitly naming the DB, schema, and table in each case. Seems fine.
The odd thing is that if my SSMS is pointing to the DB in question, TheDB, the query never returns...it just spins and spins. If, on the other hand, I point my SSMS to a different DB (changing it from, say, TheDB to TheOtherDB) or if I prefix my query with a line like USE TheOtherDB, then it returns instantly.
This is very repeatable, across restarts of SSMS, and for different users.
What's going on? I'm guessing there's a wildly wrong optimization plan or something that is being used in some cases but not others...but the whole thing seems very strange...that pointing to the DB I'm actually using makes it slower.
To answer some of the questions/comments in the comments:
What's the goal of the query? To find 'duplicate' rows. Do I need to use *? No...and in the real query I'm not. It's just part of the query anonymization. I can change it to [T1].[ID] with the same result.
Could be a bad cached plan...(any trivial modification to the query text should do, really) I can change the query in various ways (like described above, returning just the [T1].[ID] for example and continue to get the same results.
Perhaps the context databases have different compatibility levels...? In this case, both DBs are set to have a compatibility level of "Sql Server 2019 (150)".
I am doing a basic join in SQL Server at work. I thought it was taking an incredibly long time for what it is (joining a table with about 2000 records to another with about 500).
Here's what the query looks like:
select
an.AccountNumber
,d.AccountCharacteristic
,st.TotalBalance
from AccountLevelData as d
join Statements as st
on st.AccountNumber = d.AccountNumber
join #AccountNumberList as an
on an.AccountNumber = d.AccountNumber
where st.TimePeriod = eomonth(getdate(), -1)
The #AccountNumberList is just a temp table I'm using to filter down the result set to a subset of accounts that I care about. I started debugging this query and noticed it ran lightning fast if I filtered #AccountNumberList down to containing only 128 accounts, but took extremely long to finish or would not finish at all if it contained 129 accounts.
I suppose this has to do with a quirk in SQL Server or SQL Server Management Studio, but I was unable to find any resources on why or how this happens. I did notice that 128 = 0b10000000, so maybe it has something to do with bits - once the number of rows in the table takes more than seven bits to represent the query optimizer does something weird?
since the Statements table is the largest one, then you could flip the query a bit like this :
SELECT
an.AccountNumber
, d.AccountCharacteristic
, st.TotalBalance
FROM
#AccountNumberList an
JOIN Statements st ON st.AccountNumber = an.AccountNumber AND st.TimePeriod = eomonth(getdate(), -1)
JOIN AccountLevelData d ON d.AccountNumber = an.AccountNumber
If this still runs slower than it should, then you need to check the tables indexes and their performance.
For instance, you could exclude Statements from the above query, see how it perform with AccountLevelData table, if the performance is as expected, then comment out AccountLevelData join, and then test the Statements join. continue your testing until you hit the jackpot. (Don't forget to check the execution plan for each test).
Check all indexes, (basically, disable all indexes, and leave the one you want to use, test the query, if the query performed faster and issue is solved, then you can check the indexes that causing the issue).
You might need also to update the statistics for the indexes. So, do that as well.
Its because you may not set index to the table.You can add a index to the table and look at the query execution plan to optimize the query better.
So I have two queries that I'm working on, one comes from an Oracle DB and the other SQL Server DB. I'm trying to use PowerBI via Power Query as the cross over between the two. Because of the size of the Oracle DB I'm having a problem with running it, so my thought is to use one query as a clause/sub-query of the other to limit the number of results.
Based on the logic of MSFT's M language I'd assume there's a way to do sub-queries of another but I've yet to figure it out. Does anyone have any ideas on how to do this?
What I have learned to do is create a connection to the two under lying data sets, but do not load them. Then you can merge them, but do not use the default Table.NestedJoin().
After the PQ generates this, change it to:
= Table.Join(dbo_SCADocument,{"VisitID"},VISIT_KEY,{"VisitID"}) .
Also remove the trailing name. The reason is , it keeps query folding alive. For some reason, Table.NestedJoin() kills query folding. Note, if there are similar fields in the two sources other than the join it will fail.
Also it brings everything from both sources, but that is easy to alter . Also you will need to turn off the function firewall as this does not allow you to join potential sensitive data with non sensitive data. This is setting your privacy level to ignore all.
I would attempt this using the Merge command. I'm more of a UI guy, so I would click the Merge button. This generates a PQL statement for Table.Join
https://msdn.microsoft.com/en-us/library/mt260788.aspx
Setting Join Kind to Inner will restrict the output to matching rows.
I say attempt as your proposed query design will likely slow your query, not improve it. I would expect PQ to run both queries against the 2 servers and download all their data, then attempt the Join in Memory.
We just upgraded our SQL Server 2005 to SQL server 2008 R2 and noticed some performance problems.
The query below was already slow but now in 2008 it just times out. We rebuild the catalog to make sure its freshly made on 2008
DECLARE #FREETEXT varchar(255) = 'TEN-T'
select Distinct ...
from
DOSSIER_VERSION
inner join
DOSSIER_VERSION_LOCALISED ...
where
CONTAINS(DOSSIER_VERSION.*,#FREETEXT)
or
CONTAINS(DOSSIER_VERSION_LOCALISED.*,#FREETEXT)
The query takes minutes if you have both conditions enabled.
If you just put the following in the where
CONTAINS(DOSSIER_VERSION.*,#FREETEXT)
Its super fast. Same goes for the case if its just
CONTAINS(DOSSIER_VERSION_LOCALISED.*,#FREETEXT)
Since we are or'ing the results I would expect the time for this query to run to be less than the sum but as stated above it takes minutes/times out.
Can anyone tell me what is going on here? If I use a union (which is conceptually the same as the or) the performance problem is gone but I would like to know what issue I am running into here since I want to avoid rewriting queries.
Regards, Tom
See my answers to these very similar questions:
Adding more OR searches with
CONTAINS Brings Query to Crawl
SQL Server full text query across
multiple tables - why so slow?
The basic idea is that using LEFT JOINs to CONTAINSTABLE (or FREETEXTTABLE) performs significantly better than having multiple CONTAINS (or FREETEXT) ORed together in the WHERE clause.
We've got a weird problem with joining tables from SQL Server 2005 and MS Access 2003.
There's a big table on the server and a rather small table locally in Access. The tables are joined via 3 fields, one of them a datetime field (containing a day; idea is to fetch additional data (daily) from the big server table to add data to the local table).
Up until the weekend this ran fine every day. Since yesterday we experienced strange non-time-outs in Access with this query. Non-time-out means that the query runs forever with rather high network transfer, but no timeout occurs. Access doesn't even show the progress bar. Server trace tells us that the same query is exectuted over and over on the SQL server without error but without result either. We've narrowed it down to the problem seemingly being accessing server table with a big table and either JOIN or WHERE containing a date, but we're not really able to narrow it down. We rebuilt indices already and are currently restoring backup data, but maybe someone here has any pointers of things we could try.
Thanks, Mike.
If you join a local table in Access to a linked table in SQL Server, and the query isn't really trivial according to specific limitations of joins to linked data, it's very likely that Access will pull the whole table from SQL Server and perform the join locally against the entire set. It's a known problem.
This doesn't directly address the question you ask, but how far are you from having all the data in one place (SQL Server)? IMHO you can expect the same type of performance problems to haunt you as long as you have some data in each system.
If it were all in SQL Server a pass-through query would optimize and use available indexes, etc.
Thanks for your quick answer!
The actual query is really huge; you won't be happy with it :)
However, we've narrowed it down to a simple:
SELECT * FROM server_table INNER JOIN access_table ON server_table.date = local_table.date;
If the server_table is a big table (hard to say, we've got 1.5 million rows in it; test tables with 10 rows or so have worked) and the local_table is a table with a single cell containing a date. This runs forever. It's not only slow, It just does nothing besides - it seems - causing network traffic and no time out (this is what I find so strange; normally you get a timeout, but this just keeps on running).
We've just found KB article 828169; seems to be our problem, we'll look into that. Thanks for your help!
Use the DATEDIFF function to compare the two dates as follows:
' DATEDIFF returns 0 if dates are identical based on datepart parameter, in this case d
WHERE DATEDIFF(d,Column,OtherColumn) = 0
DATEDIFF is optimized for use with dates. Comparing the result of the CONVERT function on both sides of the equal (=) sign might result in a table scan if either of the dates is NULL.
Hope this helps,
Bill
Try another syntax ? Something like:
SELECT * FROM BigServerTable b WHERE b.DateFld in (SELECT DISTINCT s.DateFld FROM SmallLocalTable s)
The strange thing in your problem description is "Up until the weekend this ran fine every day".
That would mean the problem is really somewhere else.
Did you try creating a new blank Access db and importing everything from the old one ?
Or just refreshing all your links ?
Please post the query that is doing this, just because you have indexes doesn't mean that they will be used. If your WHERE or JOIN clause is not sargable then the index will not be used
take this for example
WHERE CONVERT(varchar(49),Column,113) = CONVERT(varchar(49),OtherColumn,113)
that will not use an index
or this
WHERE YEAR(Column) = 2008
Functions on the left side of the operator (meaning on the column itself) will make the optimizer do an index scan instead of a seek because it doesn't know the outcome of that function
We rebuilt indices already and are currently restoring backup data, but maybe someone here has any pointers of things we could try.
Access can kill many good things....have you looked into blocking at all
run
exec sp_who2
look at the BlkBy column and see who is blocking what
Just an idea, but in SQL Server you can attach your Access database and use the table there. You could then create a view on the server to do the join all in SQL Server. The solution proposed in the Knowledge Base article seems problematic to me, as it's a kludge (if LIKE works, then = ought to, also).
If my suggestion works, I'd say that it's a more robust solution in terms of maintainability.