I have a table with 6 columns where one column is Id (big int , primary), and one column CreatedDate (datetime), and it has rows more than one million.
when retrieving data from this table using the below query takes more than 1 minute.
select * from MyTable where CreatedDate between '2019-05-01' and '2019-05-30'
I used below query also, but it also takes more than 1 minute.
declare #minId bigint, #maxId bigint
select #minId = min(Id) from MyTable where CreatedDate > = '2019-05-01'
select #maxId = max(id) from MyTable where CreatedDate <= '2019-05-30'
select #minId, #maxId
select * from MyTable where Id between #minId and #maxId
It has only one index (Id - primary key), and I assume adding index to CreatedDate may affect insert/update operations.
I want to join this result to another table to get some report data to display in a grid, but when executing this query time out occurs.
How can I retrieve data quickly?
Create index on CreatedDate will help in retrieval while it will have some side-effect in Insert
Avoid selecting all columns with the '*' wildcard, unless you intend to use them all. Selecting redundant columns may result in unnecessary performance degradation.
Try to create the following index:
CREATE NONCLUSTERED INDEX [IX_CreatedDate_ID]
ON dbo.YourTable
(CreatedDate, ID)
GO
Pay attention to the order of the index (CreatedDate, ID). CreatedDate is the first column in index. It is very important. So when you will use WHERE CreatedDate BETWEEN '2019-05-01' and '2019-05-30', then your query plan will have index seek.
And your query should look like this:
SELECT
CreatedDate
, ID from MyTable
WHERE CreatedDate BETWEEN '2019-05-01' and '2019-05-30'
try it using Stored Procedure. The stored procedure is ready to go code which is precompiled code take less time to fetch data.
Sample stored procedure code
Related
I have a logging table that is live which saves my value to a table frequently.
My plan is to take those values and put them on a temporary table with
SELECT * INTO #temp from Block
From there I guess my block table is empty and the logger can keep on logging new values.
The next step is that I want to save them in a existing table. I wanted to use
INSERT INTO TABLENAME(COLUMN1,COLUMN2...) SELECT (COLUMN1,COLUMN2...) FROM #temp
The problem is that the #temp table has duplicates primary keys. And I only want to store the last ID.
I have tried DISTINCT but it didn't work. Could not get ROW_Count to work. Are there any ideas on how I should do it? I wish to make it with as few reads as possible.
Also, in the future I plan to send them to another database, how do I do that on SQL Server? I guess it's something like FROM Table [in databes]?
I couldn't get the blocks to copy. But here goes:
create TABLE Product_log (
Grade char(64),
block_ID char(64) PRIMARY KEY NOT NULL,
Density char(64),
BatchNumber char(64) NOT NULL,
BlockDateID Datetime
);
That is my table i want to store the data in. There I do not wish to have duplicates on the id. The problem is, while logging I get duplicates since I log on change. Lets say that the batchid is 1, if it becomes 2 while logging. I will get a blockid twice, both with batch number 1 and 2. How do I pick the latter?
Hope I explained enough for guidance. While logging they look like this:
id SiemensTiaV15_s71200_BatchTester_NewBatchIDValue_VALUE SiemensTiaV15_s71200_BatchTester_TestWriteValue_VALUE SiemensTiaV15_s71200_BatchTester_TestWriteValue_TIMESTAMP SiemensTiaV15_s71200_MainTank_Density_VALUE SiemensTiaV15_s71200_MainTank_Grade_VALUE
1 00545 S0047782 2020-06-09 11:18:44.583 0 xxxxx
2 00545 S0047783 2020-06-09 11:18:45.800 0 xxxxx
Please use below query,
select * from
(select id, SiemensTiaV15_s71200_BatchTester_NewBatchIDValue_VALUE,SiemensTiaV15_s71200_BatchTester_TestWriteValue_VALUE, SiemensTiaV15_s71200_BatchTester_TestWriteValue_TIMESTAMP, SiemensTiaV15_s71200_MainTank_Density_VALUE,SiemensTiaV15_s71200_MainTank_Grade_VALUE,
row_number() over (partition by SiemensTiaV15_s71200_BatchTester_NewBatchIDValue_VALUE order by SiemensTiaV15_s71200_BatchTester_TestWriteValue_TIMESTAMP desc) as rnk
from table_name) qry
where rnk=1;
INTO #temp FROM Block; INSERT INTO Product_log(Grade, block_ID, Density, BatchNumber, BlockDateID)
selct NewBatchIDValue_VALUE, TestWriteValue_VALUE, TestWriteValue_TIMESTAMP,
Density_VALUE, Grade_VALUE from
(select NewBatchIDValue_VALUE, TestWriteValue_VALUE,
TestWriteValue_TIMESTAMP, Density_VALUE, Grade_VALUE, row_number() over
(partition by BatchTester_NewBatchIDValue_VALUE order by
BatchTester_TestWriteValue_VALUE) as rnk from #temp) qry
where rnk = 1;
My question would be better served as a comment on Limit result set in sql window function , but I don't have the necessary reputation to comment.
Given a table of moving vehicle locations, for each vehicle I wish to find the most recent recorded position (and other data about the vehicle at that time). Based on answers in the other question, I can run a query like:
Table definition:
CREATE TABLE VehiclePositions
(
Id BIGINT NOT NULL,
VehicleID NVARCHAR(12) NULL,
Timestamp DATETIME NULL,
PositionX FLOAT NULL,
PositionY FLOAT NULL,
PositionZ SMALLINT NULL,
Speed SMALLINT NULL,
Heading SMALLINT NULL
)
Query:
select *
from
(select
*,
row_number() over (partition by VehicleID order by Timestamp desc) as ranking
from VehiclePositions) as x
where
ranking = 1
Now, the problem is that this does a full table scan. I thought that by creating an appropriate index, I could avoid this:
CREATE INDEX idx_VehicPosition ON VehiclePositions(VehicleID, Timestamp);
However, SQL Server will happily ignore this index in the query and still perform the stable scan.
Note: I can get SQL Server to use the index, but the code is rather ugly:
DECLARE #ids TABLE (id NVARCHAR(12) UNIQUE)
INSERT INTO #ids
SELECT DISTINCT VehicleID
FROM VehiclePositions
SELECT ep.*
FROM VehiclePositions vp
WHERE Timestamp = (SELECT Max(TimeStamp) FROM VehiclePositions vp2
WHERE vp2.VehicleID = vp.VehicleID)
AND VehicleID IN (SELECT DISTINCT id FROM #ids)
(The VehicleID IN... is because it seems SQL Server doesn't implement seek-skip optimisations. It still comes up with a pretty non-optimal query plan that visits the index twice, but at least it doesn't execute in linear time).
Is there a way to make SQL Server run the window function query intelligently?
I'm using SQL Server 2014...
Help will be appreciated
What i would do :
SELECT *
FROM
(SELECT MAX(Timestamp) as maxtime,
VehicleID
FROM VehiclePositions
GROUP BY VehicleID ) as maxed INNER JOIN
(SELECT Id ,
VehicleID ,
Timestamp ,
PositionX ,
PositionY,
PositionZ,
Speed ,
Heading
FROM VehiclePositions) as vals
ON maxed.maxtime = vals.Timestamp
AND maxed.VehicleID = vals.VehicleID
to my knowledge you cant get around your index getting scanned twice.
As long as you are selecting all vehicles from the table and are select all column (or at least columns that are not in your index), I would expect the table scan to keep popping up.
In many cases, that will actually be the most efficient query plan. Only if you have a many rows per vehicle (like several pages) a seek strategy might be faster.
If you do have a lot of rows per vehicle, you might consider partitioning your table on Timestamp...
You can filter results in windows function using 'qualify', as follows:
select *
from VehiclePositions
qualify row_number() over (partition by VehicleID order by Timestamp desc) = 1
I use this statement to create a list for excel
SELECT DISTINCT Year, Version
FROM myView
WHERE id <> 'old'
ORDER BY Year DESC, Version DESC
The problem is that the execution time is over 30s because of the almost 2 million rows.
The result has only around 1000 rows.
What are my options to extract only those two columns in order to speed up the execution time? I also need to make sure that inserts to the underlying table are recognized.
Do I need a new table to copy the values from the view? And a trigger to manage the updates?
Thank you
So, presumably there's a table with Year and id underlying your view. Given this (trivial) example:
CREATE TABLE myTable ([id] varchar(10), [Year] int, [Version] int);
Just create an index on that table that matches the way you're querying your data. Given your query of:
SELECT DISTINCT Year, Version
FROM myView
WHERE id <> 'old'
ORDER BY Year DESC, Version DESC
This query matches the WHERE and ORDER BY clauses and should give you all the performance you need:
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[myTable]') AND name = N'IX_YearVersion_Filtered')
DROP INDEX [IX_YearVersion_Filtered] ON [dbo].[myTable] WITH ( ONLINE = OFF )
GO
CREATE NONCLUSTERED INDEX [IX_YearVersion_Filtered] ON [dbo].[myTable]
(
[Year] DESC,
[Version] DESC
)
WHERE ([id]<>'old')
GO
with cte_x
as
(SELECT Year, Version
FROM myView
WHERE id not in ('old')
group by Year, Version)
SELECT DISTINCT Year, Version
FROM cte_x
ORDER BY Year DESC, Version DESC
I have a stored procedure that uses a temporary table to make some joins in a select clause. The select clause contains the value from the Id column of the temporary table like this:
CREATE TABLE #TempTable
(
Id INT PRIMARY KEY,
RootVal INT
)
The Select looks like this:
Select value1, value2, #TempTable.Id AS ValKey
From MainTable INNER JOIN #TempTable ON MainTable.RootVal = #TempTable.RootVal
The query takes over a minute to run in real life but if I remove the "#TempTable.Id" from the select list it runs in a second.
Does anyone know why there is such a huge cost to including a value from a #temp table compared to just using it in a join?
Most likely:
data type mismatch
eg nvarchar vs int
lack of index on MainTable.RootVal
Why have Id as PK and then JOIN on another column?
I have a very meaty stored procedure in a SQL Server 2000 DB which returns a single resultset. I don't want to (not allowed to) touch the original SP but would like add pagination to the returned records.
Is it possible to wrap this SP with another that takes the returned resultset and only gives me rows X to Y ?
create procedure ProcWrap
as
declare #T table (ID int, Name nvarchar(50))
insert into #T
exec ProcToWrap
select *
from #T
where ID < 10
Edit 1
Don't have SQL Server 2000 to test on and I don't remember if table variables where available then. Here is a procedure using a temp table instead. Added a RowNum identity column that you can use for pagination.
create procedure ProcWrap2
as
create table #T (RowNum int identity, ID int, Name nvarchar(50))
insert into #T
exec ProcToWrap
select *
from #T
where RowNum between 10 and 19
drop table #T
Edit 2
Output from ProcToWrap in this case is columns ID and Name. RowNum is generated automatically.
Get the results from the SP and put them in a temporary table, then you can select X results from that table.
As others have said you will have to put the results of the procedure in a temp table then select the rows you want from that.
To get a set of rows from your results you need to use the ROW_NUMER() function:
SELECT
ROW_NUMBER() OVER (ORDER BY ID) AS row_number, *
FROM
Your_Temp_Table
WHERE row_number BETWEEN 11 AND 20 -- For the second page of results with 10 per page.
EDIT: Just realised you are using SQL Server 2000 which does not have ROW_NUMBER(), sorry
EDIT2: Since you are storing the results of the query in a temp table you can add an incrementing integer field to that result set and use that as a simulation for the ROW_NUMBER() in order to select the row you need.
EDIT3: Here's a link to an article discussing pagination in SQL Server 2000