My application access data from a table in SQL Server. Consider the table name is PurchaseDetail with some other columns.
The select query has below where clauses.
1. name - name has 10000 values only.
2. createdDateTime
The actual query is
select *
from PurchaseDetail
where name in (~2000 name)
and createdDateTime = 'someDateValue';
The SQL tuning advisor gave some recommendation. I tried with those recommended indexes. The performance increased a bit but not completely.
Is there any wrong in my query? or Is there any possible to change/improve my select query?
Because I didn't use IN in where clause before. My table having more than 100 million records.
Any suggestion please?
In this case using IN for that much data is not good at all.
this best way is to use INNER JOIN instead.
It would be nicer if insert those names into a temp table and INNER JOIN it with your SELECT query.
Using SQL Server 2014:
Is there any performance difference between the following statements?
DELETE FROM MyTable where PKID IN (SELECT PKID FROM #TmpTableVar)
AND
DELETE FROM MyTable INNER JOIN #TmpTableVar t ON MyTable.PKID = t.PKID
In your given example the execution plans will be the same (most probably).
But having same execution plans doesn't mean that they are the best execution plans you can possibly have for this statement.
The problem I see in both of your queries is the use of the Table Variable.
SQL Server always assumes that there is only 1 row in the table variable. Only in SQL Server 2014 and later version this assumption has been changed to 100 rows.
So no matter how many rows you have this the table variable SQL Server will always assume you have one row in the #TmpTableVar.
You can change your code slightly to give SQL Server a better idea of how many rows there will be in that table by replacing it with a Temporary table and since it is a PK_ID Column in your table variable you can also create an index on that table, to give best chance to sql server to come up with the best possible execution plan for this query.
SELECT PKID INTO #Temp
FROM #TmpTableVar
-- Create some index on the temp table here .....
DELETE FROM MyTable
WHERE EXISTS (SELECT 1
FROM #Temp t
WHERE MyTable.PKID = t.PKID)
Note
In operator will work fine since it is a primary key column in the table variable. but if you ever use IN operator on a nullable column, the results may surprise you, The IN operator goes all pear shape as soon as it finds a NULL values in the column it is checking on.
I personally prefer Exists operator for such queries but inner joins should also work just fine but avoid IN operators if you can.
I have a big query to get multiple rows by id's like
SELECT *
FROM TABLE
WHERE Id in (1001..10000)
This query runs very slow and it ends up with timeout exception.
Temp fix for it is querying with limit, break this query into 10 parts per 1000 id's.
I heard that using temp tables may help in this case but also looks like ms sql server automatically doing it underneath.
What is the best way to handle problems like this?
You could write the query as follows using a temporary table:
CREATE TABLE #ids(Id INT NOT NULL PRIMARY KEY);
INSERT INTO #ids(Id) VALUES (1001),(1002),/*add your individual Ids here*/,(10000);
SELECT
t.*
FROM
[Table] AS t
INNER JOIN #ids AS ids ON
ids.Id=t.Id;
DROP TABLE #ids;
My guess is that it will probably run faster than your original query. Lookup can be done directly using an index (if it exists on the [Table].Id column).
Your original query translates to
SELECT *
FROM [TABLE]
WHERE Id=1000 OR Id=1001 OR /*...*/ OR Id=10000;
This would require evalutation of the expression Id=1000 OR Id=1001 OR /*...*/ OR Id=10000 for every row in [Table] which probably takes longer than with a temporary table. The example with a temporary table takes each Id in #ids and looks for a corresponding Id in [Table] using an index.
This all assumes that there are gaps in the Ids between 1000 and 10000. Otherwise it would be easier to write
SELECT *
FROM [TABLE]
WHERE Id BETWEEN 1001 AND 10000;
This would also require an index on [Table].Id to speed it up.
In my C# application I'm using the following query to search for a particular string:
;WITH selectRows AS (SELECT *, row = ROW_NUMBER() OVER (ORDER BY <column_name>) FROM <table_name>)
SELECT row FROM selectRows WHERE <column_name> LIKE '%<search_string>%' COLLATE <collate> ORDER BY row;
This particular query always worked fine for me, even when the colum_name for the OVER ORDER BY clause was a column that contained null values. Yesterday I tried to search on a somewhat bigger SQL table (+- 1 million records), it suprised me that I got different row_numbers returned without changing the query between the executions. This only seem to happen on bigger tables and when the column_name for the OVER ORDER BY clause contains any null values. When the column_name is pointed to a column WITHOUT null values the query returns the same result over and over again.
I also tried the following query, but this did not work as well:
;WITH selectRows AS (SELECT *, row = ROW_NUMBER() OVER (ORDER BY ISNULL(<column_name>, '')) FROM <table_name>)
SELECT row FROM selectRows WHERE <column_name> LIKE '%<search_string>%' COLLATE <collate> ORDER BY row;
Note: both queries were tested on SQL Server 2012 and SQL Server 2008. The searched table also had a Primary Key (clustered) index on a Identity column and a nonclustered index on the column_name that is used for the OVER ORDER BY clause.
Thanks in advance!
You have specified an ordering criterion that is not a total order. Example: You order on a column that is always zero. That way the CTE can output a different row order each time.
You are filtering after ordering. That means your filter runs on different rows each time. It might happen to have a lot of matching rows or not.
In general SQL queries are not 100% deterministic thanks to certain constructs. There are more than this one.
Fix: Specify a total order. Use anything to break ties such as ORDER BY X, ID. As a habit I always specify a total order.
I have searched for paging in SQL Server. I found most of the solution look like that
What is the best way to paginate results in SQL Server
But it don't meet my expectation.
Here is my situation:
I work on JasperReport, for that: to export the report I just need pass the any Select query into the template, it will auto generated out the report
EX : I have a select query like this:
Select * from table A
I don't know any column names in table A. So I can't use
Select ROW_NUMBER() Over (Order By columsName)
And I also don't want it order by any columns.
Anyone can help me do it?
PS: In Oracle , it have rownum very helpful in this case.
Select * from tableA where rownum > 100 and rownum <200
Paging with Oracle
You should use ROW_NUMBER with an ORDER BY - because without an ORDER BY there is no determinism in how rows are returned. You can run the same query three times and get the results back in three different orders. Especially if merry-go-round scans come into play.
So unless you want your report to have the possibility of showing the same rows to users on multiple pages, or some rows never on any page, you need to find a way to order the result set to make it deterministic.
From my opinion, you can use sql query to find out how many columns in a table, and then find out a proper one for ' order by ' to depend on.
The script of how to get out columns of an table refer to : How can I get column names from a table in SQL Server?
Check out this link
http://msdn.microsoft.com/en-us/library/ms186734.aspx
SQL Server has similar function ROW_NUMBER. Though it behaves a bit differently.
SQL Server provides no guarantee of row order unless you have have specified a column in order by clause. I would recommend that you give an order by clause that has unique values.
Thank for all your help. Because of order by are required when paging in MS SQL Server, so I used ResultSetMetaData to get the Columns name and do paging as well.
You can use the below query aswell.
declare #test table(
id int,
value1 varchar(100),
value2 int)
insert into #test values(1,'10/50',50)
insert into #test values(2,'10/60',60)
insert into #test values(3,'10/60',61)
insert into #test values(4,'10/60',10)
insert into #test values(5,'10/60',11)
insert into #test values(6,'10/60',09)
select *
from ( select row_number() over (order by (select 0)) as rownumber,* from #test )test
where test.rownumber<=5