I need to calculate the difference of a column between two lines of a table. Is there any way I can do this directly in SQL? I'm using Microsoft SQL Server 2008.
I'm looking for something like this:
SELECT value - (previous.value) FROM table
Imagining that the "previous" variable reference the latest selected row. Of course with a select like that I will end up with n-1 rows selected in a table with n rows, that's not a probably, actually is exactly what I need.
Is that possible in some way?
Use the lag function:
SELECT value - lag(value) OVER (ORDER BY Id) FROM table
Sequences used for Ids can skip values, so Id-1 does not always work.
SQL has no built in notion of order, so you need to order by some column for this to be meaningful. Something like this:
select t1.value - t2.value from table t1, table t2
where t1.primaryKey = t2.primaryKey - 1
If you know how to order things but not how to get the previous value given the current one (EG, you want to order alphabetically) then I don't know of a way to do that in standard SQL, but most SQL implementations will have extensions to do it.
Here is a way for SQL server that works if you can order rows such that each one is distinct:
select rank() OVER (ORDER BY id) as 'Rank', value into temp1 from t
select t1.value - t2.value from temp1 t1, temp1 t2
where t1.Rank = t2.Rank - 1
drop table temp1
If you need to break ties, you can add as many columns as necessary to the ORDER BY.
WITH CTE AS (
SELECT
rownum = ROW_NUMBER() OVER (ORDER BY columns_to_order_by),
value
FROM table
)
SELECT
curr.value - prev.value
FROM CTE cur
INNER JOIN CTE prev on prev.rownum = cur.rownum - 1
Oracle, PostgreSQL, SQL Server and many more RDBMS engines have analytic functions called LAG and LEAD that do this very thing.
In SQL Server prior to 2012 you'd need to do the following:
SELECT value - (
SELECT TOP 1 value
FROM mytable m2
WHERE m2.col1 < m1.col1 OR (m2.col1 = m1.col1 AND m2.pk < m1.pk)
ORDER BY
col1, pk
)
FROM mytable m1
ORDER BY
col1, pk
, where COL1 is the column you are ordering by.
Having an index on (COL1, PK) will greatly improve this query.
LEFT JOIN the table to itself, with the join condition worked out so the row matched in the joined version of the table is one row previous, for your particular definition of "previous".
Update: At first I was thinking you would want to keep all rows, with NULLs for the condition where there was no previous row. Reading it again you just want that rows culled, so you should an inner join rather than a left join.
Update:
Newer versions of Sql Server also have the LAG and LEAD Windowing functions that can be used for this, too.
select t2.col from (
select col,MAX(ID) id from
(
select ROW_NUMBER() over(PARTITION by col order by col) id ,col from testtab t1) as t1
group by col) as t2
The selected answer will only work if there are no gaps in the sequence. However if you are using an autogenerated id, there are likely to be gaps in the sequence due to inserts that were rolled back.
This method should work if you have gaps
declare #temp (value int, primaryKey int, tempid int identity)
insert value, primarykey from mytable order by primarykey
select t1.value - t2.value from #temp t1
join #temp t2
on t1.tempid = t2.tempid - 1
Another way to refer to the previous row in an SQL query is to use a recursive common table expression (CTE):
CREATE TABLE t (counter INTEGER);
INSERT INTO t VALUES (1),(2),(3),(4),(5);
WITH cte(counter, previous, difference) AS (
-- Anchor query
SELECT MIN(counter), 0, MIN(counter)
FROM t
UNION ALL
-- Recursive query
SELECT t.counter, cte.counter, t.counter - cte.counter
FROM t JOIN cte ON cte.counter = t.counter - 1
)
SELECT counter, previous, difference
FROM cte
ORDER BY counter;
Result:
counter
previous
difference
1
0
1
2
1
1
3
2
1
4
3
1
5
4
1
The anchor query generates the first row of the common table expression cte where it sets cte.counter to column t.counter in the first row of table t, cte.previous to 0, and cte.difference to the first row of t.counter.
The recursive query joins each row of common table expression cte to the previous row of table t. In the recursive query, cte.counter refers to t.counter in each row of table t, cte.previous refers to cte.counter in the previous row of cte, and t.counter - cte.counter refers to the difference between these two columns.
Note that a recursive CTE is more flexible than the LAG and LEAD functions because a row can refer to any arbitrary result of a previous row. (A recursive function or process is one where the input of the process is the output of the previous iteration of that process, except the first input which is a constant.)
I tested this query at SQLite Online.
You can use the following funtion to get current row value and previous row value:
SELECT value,
min(value) over (order by id rows between 1 preceding and 1
preceding) as value_prev
FROM table
Then you can just select value - value_prev from that select and get your answer
Related
I need to select random rows from my sql table, when search this cases in google, they suggested to ORDER BY NEWID() but it reduces the performance. Since my table has more than 2'000'000 rows of data, this solution does not suit me.
I tried this code to get random data :
SELECT TOP 10 *
FROM Table1
WHERE (ABS(CAST((BINARY_CHECKSUM(*) * RAND()) AS INT)) % 100) < 10
It also drops performance sometimes.
Could you please suggest good solution for getting random data from my table, I need minimum rows from that tables like 30 rows for each request. I tried TableSAMPLE to get the data, but it returns nothing once I added my where condition because it return the data by the basis of page not basis of row.
Try to calc the random ids before to filter your big table.
since your key is not identity, you need to number records and this will affect performances..
Pay attention, I have used distinct clause to be sure to get different numbers
EDIT: I have modified the query to use an arbitrary filter on your big table
declare #n int = 30
;with
t as (
-- EXTRACT DATA AND NUMBER ROWS
select *, ROW_NUMBER() over (order by YourPrimaryKey) n
from YourBigTable t
-- SOME FILTER
WHERE 1=1 /* <-- PUT HERE YOUR COMPLEX FILTER LOGIC */
),
r as (
-- RANDOM NUMBERS BETWEEN 1 AND COUNT(*) OF FILTERED TABLE
select distinct top (#n) abs(CHECKSUM(NEWID()) % n)+1 rnd
from sysobjects s
cross join (SELECT MAX(n) n FROM t) t
)
select t.*
from t
join r on r.rnd = t.n
If your uniqueidentifier key is a random GUID (not generated with NEWSEQUENTIALID() or UuidCreateSequential), you can use the method below. This will use the clustered primary key index without sorting all rows.
SELECT t1.*
FROM (VALUES(
NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID())
,(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID())
,(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID())) AS ThirtyKeys(ID)
CROSS APPLY(SELECT TOP (1) * FROM dbo.Table1 WHERE ID >= ThirtyKeys.ID) AS t1;
Is there a way to have a column from another table with value which is always the same inside a View> Example:
SELECT *,
(SELECT value FROM tblStudentPrefixes WHERE PrefixName = 'SeniorPrefix')
AS StudentPrefix
FROM tblStudents
Will the above nested query get executed fro each row? Is there a way to execute it once and use for all rows.
Please note, I'm specifically talking about a View, not a Stored Procedure. I know this can be done in a Stored Procedure.
This actually depends on your table set up. Unless prefixName is constrained to be unique you could come across errors, where the subquery returns more than one row. If it is not constrained to be unique, but happens to be unique for SeniorPrefix then your query will be executed 1000 times. To demonstrate I have used the following DDL:
CREATE TABLE #tblStudents (ID INT IDENTITY(1, 1), Filler CHAR(100));
INSERT #tblStudents (Filler)
SELECT TOP 10000 NULL
FROM sys.all_objects a, sys.all_objects b;
CREATE TABLE #tblStudentPrefixes (Value VARCHAR(10), PrefixName VARCHAR(20));
INSERT #tblStudentPrefixes (Value, PrefixName) VALUES ('A Value', 'SeniorPrefix');
Running your query gives the following IO output:
Table '#tblStudentPrefixes'. Scan count 10000, logical reads 10000
Table '#tblStudents'. Scan count 1, logical reads 142
The key being the 1000 logical reads on tblStudentPrefixes. The other problem with it not being constrained to be unique is that if you have duplicates your query will fail with the error:
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
If you can't constrain PrefixName to be unique, then you can stop it executing for each row and avoid the errors by using TOP:
SELECT *,
(SELECT TOP 1 value FROM #tblStudentPrefixes WHERE PrefixName = 'SeniorPrefix' ORDER BY Value)
AS StudentPrefix
FROM #tblStudents
The IO now becomes:
Table '#tblStudentPrefixes'. Scan count 1, logical reads 1
Table '#tblStudents'. Scan count 1, logical reads 142
However, I would still recommend switching to a CROSS JOIN here:
SELECT s.*, p.Value AS StudentPrefix
FROM #tblStudents AS s
CROSS JOIN
( SELECT TOP 1 value
FROM #tblStudentPrefixes
WHERE PrefixName = 'SeniorPrefix'
ORDER BY Value
) AS p;
Inspection of the execution plans shows that a sub-select using a table spool which is very unnecessary for a single value:
So in summary, it depends on your table set up whether it will execute for each row, but regardless you are giving the optimiser a better chance if you switch to a cross join.
EDIT
In light of the fact that you need to return rows from tblstudent when there is no match for SeniorPrefix in tblStudentPrefixes, and that PrefixName is not currenty constrianed to be unique then the best solution is:
SELECT *,
(SELECT MAX(value) FROM #tblStudentPrefixes WHERE PrefixName = 'SeniorPrefix')
AS StudentPrefix
FROM #tblStudents;
If you do constrain it to be unique, then the following 3 queries produce (essentially) the same plan and the same results, it is simply personal preference:
SELECT *,
(SELECT value FROM #tblStudentPrefixes WHERE PrefixName = 'SeniorPrefix')
AS StudentPrefix
FROM #tblStudents;
SELECT s.*, p.Value AS StudentPrefix
FROM #tblStudents AS s
LEFT JOIN #tblStudentPrefixes AS p
ON p.PrefixName = 'SeniorPrefix';
SELECT s.*, p.Value AS StudentPrefix
FROM #tblStudents AS s
OUTER APPLY
( SELECT Value
FROM #tblStudentPrefixes
WHERE PrefixName = 'SeniorPrefix'
) AS p;
I hope I understand your question right, but try this
SELECT *
FROM tblStudents
Outer Apply
(
SELECT value
FROM tblStudentPrefixes
WHERE PrefixName = 'SeniorPrefix'
) as tble
This is OK. Subquery would be executed for every row on every row (which could provide bad performance).
You could try also:
SELECT tblStudents.*,StudentPrefix.value
FROM tblStudents,
(SELECT value
FROM tblStudentPrefixes
WHERE PrefixName = 'SeniorPrefix')StudentPrefix
I have the following access query which I need to run in mssql:
SELECT
[PUB_op-mstr].[om-job],
Last([PUB_op-mstr].[om-emp]) AS [LastOfom-emp],
Max([PUB_op-mstr].[om-dt-end]) AS [MaxOfom-dt-end],
[PUB_op-mstr].[om-wkctr]
FROM
PUB_wc_mstr INNER JOIN [PUB_op-mstr]
ON
PUB_wc_mstr.wc_wkctr = [PUB_op-mstr].[om-wkctr]
GROUP BY
[PUB_op-mstr].[om-job],
[PUB_op-mstr].[om-wkctr],
PUB_wc_mstr.wc_dept
HAVING
(((Max([PUB_op-mstr].[om-dt-end]))>=Date()-7
And
(Max([PUB_op-mstr].[om-dt-end]))<Date())
AND ((PUB_wc_mstr.wc_dept)="633" Or (PUB_wc_mstr.wc_dept)="646"));
MS SQL doesn't support LAST aggregate function. So, you can just replace it with Min / Max. Or you have write your own SELECT like
[LastOfom-emp] = (SELECT ...
LAST() in Access gives the last element of the column you're looking in.
Example: T1 has one column c1, which contains:
one
two
three
The statement:
SELECT LAST(c1) FROM T1
gives: three
Porting this function to SQL Server is doable, but only if there is (at least) one sorted column in the table. To get the last element of the column, you would have to do:
SELECT TOP(1) c1 FROM T1 ORDER BY c1 DESC;
This would give you the wrong result, namely "two" (because the column isn't sorted). So, to find in this case the right answer, you would need another column, for example an incrementing ID
c1 c2
one 1
two 2
three 3
Now you can:
SELECT TOP(1) c1 FROM T1 ORDER BY c2 DESC;
Since c2 is sorted, you now get the result "three".
Assuming Last([PUB_op-mstr].[om-emp]) is the om-emp value for the maximum om-dt-end, try:
select [om-job], [LastOfom-emp], [MaxOfom-dt-end], [om-wkctr] from
(SELECT [PUB_op-mstr].[om-job],
[PUB_op-mstr].[om-emp] AS [LastOfom-emp],
[PUB_op-mstr].[om-dt-end] AS [MaxOfom-dt-end],
[PUB_op-mstr].[om-wkctr],
row_number() over (partition by [PUB_op-mstr].[om-job],
[PUB_op-mstr].[om-wkctr],
PUB_wc_mstr.wc_dept
order by [PUB_op-mstr].[om-dt-end] desc) rn
FROM PUB_wc_mstr
JOIN [PUB_op-mstr]
ON PUB_wc_mstr.wc_wkctr = [PUB_op-mstr].[om-wkctr]
WHERE PUB_wc_mstr.wc_dept IN ('633','646')
) sq
where rn=1 and
[MaxOfom-dt-end]>=Dateadd(d,-7, getdate()) And
[MaxOfom-dt-end]< getdate()
I'm trying to select records with a statement
SELECT *
FROM A
WHERE
LEFT(B, 5) IN
(SELECT * FROM
(SELECT LEFT(A.B,5), COUNT(DISTINCT A.C) c_count
FROM A
GROUP BY LEFT(B,5)
) p1
WHERE p1.c_count = 1
)
AND C IN
(SELECT * FROM
(SELECT A.C , COUNT(DISTINCT LEFT(A.B,5)) b_count
FROM A
GROUP BY C
) p2
WHERE p2.b_count = 1)
which takes a long time to run ~15 sec.
Is there a better way of writing this SQL?
If you would like to represent Set Difference (A-B) in SQL, here is solution for you.
Let's say you have two tables A and B, and you want to retrieve all records that exist only in A but not in B, where A and B have a relationship via an attribute named ID.
An efficient query for this is:
# (A-B)
SELECT DISTINCT A.* FROM (A LEFT OUTER JOIN B on A.ID=B.ID) WHERE B.ID IS NULL
-from Jayaram Timsina's blog.
You don't need to return data from the nested subqueries. I'm not sure this will make a difference withiut indexing but it's easier to read.
And EXISTS/JOIN is probably nicer IMHO then using IN
SELECT *
FROM
A
JOIN
(SELECT LEFT(B,5) AS b1
FROM A
GROUP BY LEFT(B,5)
HAVING COUNT(DISTINCT C) = 1
) t1 On LEFT(A.B, 5) = t1.b1
JOIN
(SELECT C AS C1
FROM A
GROUP BY C
HAVING COUNT(DISTINCT LEFT(B,5)) = 1
) t2 ON A.C = t2.c1
But you'll need a computed column as marc_s said at least
And 2 indexes: one on (computed, C) and another on (C, computed)
Well, not sure what you're really trying to do here - but obviously, that LEFT(B, 5) expression keeps popping up. Since you're using a function, you're giving up any chance to use an index.
What you could do in your SQL Server table is to create a computed, persisted column for that expression, and then put an index on that:
ALTER TABLE A
ADD LeftB5 AS LEFT(B, 5) PERSISTED
CREATE NONCLUSTERED INDEX IX_LeftB5 ON dbo.A(LeftB5)
Now use the new computed column LeftB5 instead of LEFT(B, 5) anywhere in your query - that should help to speed up certain lookups and GROUP BY operations.
Also - you have a GROUP BY C in there - is that column C indexed?
If you are looking for just set difference between table1 and table2,
the below query is simple that gives the rows that are in table1, but not in table2, such that both tables are instances of the same schema with column names as
columnone, columntwo, ...
with
col1 as (
select columnone from table2
),
col2 as (
select columntwo from table2
)
...
select * from table1
where (
columnone not in col1
and columntwo not in col2
...
);
I am trying to write a SQL Server query but have had no luck and was wondering if anyone may have any ideas on how to achieve my query.
What i'm trying to do:
I have a table with several columns naming the ones that i am dealing with TaskID, StatusCode, Timestamp. Now this table just holds tasks for one of our systems that run throughout the day and when something runs it gets a timestamp and the statuscode depending on the status for that task.
Sometimes what happens is the task table will be updated with a new timestamp but the statusCode will not have changed since the last update of the task so for two or more consecutive rows of a given task the statusCode can be the same. When i say consecutive rows i mean with regards to timestamp.
So example task 88 could have twenty rows at statusCode 2 after which the status code changes to something else.
Now what i am trying to do with no luck at the moment is to retrieve a list from this table of all the tasks and the statuscodes and the timestamps but in the case where i have more than one consecutive row for a task with the same statuscode i just want to take the first row with the lowest timestamp and ignore the rest of the row until the statuscode for that task changes.
To make it simpler in this case you can assume that i have a taskid which i am filtering on so i am just looking at a single task.
Does anyone have any ideas as to how i can do this or perhaps something that i coudl probably read to help me?
Thanks
Irfan.
This are a couple ways of getting what you want:
SELECT
T1.task_id,
T1.status_code,
T1.status_timestamp
FROM
My_Table T1
LEFT OUTER JOIN My_Table T2 ON
T2.task_id = T1.task_id AND
T2.status_timestamp < T1.status_timestamp
LEFT OUTER JOIN My_Table T3 ON
T3.task_id = T1.task_id AND
T3.status_timestamp < T1.status_timestamp AND
T3.status_timestamp > T2.status_timestamp
WHERE
T3.task_id IS NULL AND
(T2.status_code IS NULL OR T2.status_code <> T1.status_code)
ORDER BY
T1.status_timestamp
or
SELECT
T1.task_id,
T1.status_code,
T1.status_timestamp
FROM
My_Table T1
LEFT OUTER JOIN My_Table T2 ON
T2.task_id = T1.task_id AND
T2.status_timestamp = (
SELECT
MAX(status_timestamp)
FROM
My_Table T3
WHERE
T3.task_id = T1.task_id AND
T3.status_timestamp < T1.status_timestamp)
WHERE
(T2.status_code IS NULL OR T2.status_code <> T1.status_code)
ORDER BY
T1.status_timestamp
Both methods rely on there being no exact matches of the status_timestamp values (two rows can't have the same exact status_timestamp for a given task_id.)
Something like
select TaskID,StatusCode,Min(TimeStamp)
from table
group by TaskID,StatusCode
order by 1,2
Note that is statuscode can duplicate, you will need an additional field, but hopefully this can point you in the right direction...
Something like the following should get you in the right direction....
CREATE TABLE #T
(
TaskId INT
,StatusCode INT
,StatusTimeStamp DATETIME
)
INSERT INTO #T
SELECT 1, 1, '2009-12-01 14:20'
UNION SELECT 1, 2, '2009-12-01 16:20'
UNION SELECT 1, 2, '2009-12-02 09:15'
UNION SELECT 1, 2, '2009-12-02 12:15'
UNION SELECT 1, 3, '2009-12-02 18:15'
;WITH CTE AS
(
SELECT TaskId
,StatusCode
,StatusTimeStamp
,ROW_NUMBER() OVER (PARTITION BY TaskId, StatusCode ORDER BY TaskId, StatusTimeStamp DESC) AS RNUM
FROM #T
)
SELECT TaskId
,StatusCode
,StatusTimeStamp
FROM CTE
WHERE RNUM = 1
DROP TABLE #T