Resume a WHILE loop from where it stopped SQL - sql-server

I have a while loop query that I only want to run until 11PM everyday - I'm aware this can be achieved with a WAITFOR statement, and then just END the query.
However, on the following day, once I re-run my query, I want it to continue from where it stopped on the last run. So I'm thinking of creating a log table that will contain the last processed ID.
How can I achieve this?
DECLARE #MAX_Value BIGINT = ( SELECT MAX(ID) FROM dbo.TableA )
DECLARE #MIN_Value BIGINT = ( SELECT MIN(ID) FROM dbo.TableA )
WHILE (#MIN_Value < #MAX_Value )
BEGIN
INSERT INTO dbo.MyResults
/* Do some processing*/
….
….
….
SET #MIN_Value = MIN_Value + 1
/*I only want the above processing to run until 11PM*/
/* Once it’s 11PM, I want to save the last used #MIN_Value
into my LoggingTable (dbo.Logging) and kill the above processing.*/
/* Once I re-run the query I want my processing to restart from the
above #MIN_Value which is recorded in dbo.Logging */
END

Disclaimer: I do not recommend using WHILE loops in SQL Server but considering the comment that you want a solution in SQL, here you go:
-- First of all, I strongly recommend using a different way of assigning variable values to avoid scenarios with the variable being NULL when the table is empty, also you can do it in a single select.
-- Also, if something started running at 10:59:59 it will let the processing for the value finish and will not simply rollback at 11.
CREATE TABLE dbo.ProcessingValueLog (
LogEntryId BIGINT IDENTITY(1,1) NOT NULL,
LastUsedValue BIGINT NOT NULL,
LastUsedDateTime DATETIME NOT NULL DEFAULT(GETDATE()),
CompletedProcessing BIT NOT NULL DEFAULT(0)
)
DECLARE #MAX_Value BIGINT = 0;
DECLARE #MIN_Value BIGINT = 0;
SELECT
#MIN_Value = MIN(ID),
#MAX_Value = MAX(ID)
FROM
dbo.TableA
SELECT TOP 1
#MIN_Value = LastUsedValue
FROM
dbo.ProcessingValueLog
WHERE
CompletedProcessing = 1
ORDER BY
LastUsedDateTime DESC
DECLARE #CurrentHour TINYINT = HOUR(GETDATE());
DECLARE #LogEntryID BIGINT;
WHILE (#MIN_Value < #MAX_Value AND #CurrentHour < 23)
BEGIN
INSERT INTO dbo.ProcessingValueLog (LastUsedValue)
VALUE(#MIN_Value)
SELECT #LogEntryID = SCOPE_IDENTITY()
// Do some processing...
SET #MIN_Value = #MIN_Value + 1;
UPDATE dbo.ProcessingValueLog
SET CompletedProcessing = 1
WHERE LogEntryId = #LogEntryID
SET #CurrentHour = HOUR(GETDATE())
END

Related

Avoid while loop to check row "state change"

I have a table that stores an Id, a datetime and an int crescent value. This value increases until it "breaks" and returns to a 0-near value. Ex: ...1000, 1200, 1350, 8, 10, 25...
I need to count how many times this "overflow" happens, BUT... I'm talking about a table that stores 200k rows per day!
I had already solved it! But using a procedure with a cursor that iterates over it with a while-loop. But I KNOW it isn't the faster way to do it.
Can someone help me to find some another way?
Thanks!
->
Table structure:
Id Bigint Primary Key, CreatedAt DateTime, Value Not Null Int.
Problem:
If Delta-Value between two consecutive rows is < 0, increase a counter.
Table has 200k new rows every-day.
No trigger allowed.
[FIRST EDIT]
Table has the actual structure:
CREATE TABLE ValuesLog (
Id BIGINT PRIMARY KEY,
Machine BIGINT,
CreatedAt DATETIME,
Value INT
)
I need:
To check when the [Value] of some [Machine] suddenly decreases.
Some users said to used LEAD/LAG. But it has a problem... if I chose many machines, the LEAD/LAG fuctions doesn't care about "what machine it is". So, if I find for machine-1 and machine-2, if machine-1 increase but the machine-2 descrease, LEAD/LAG will give me a false positive.
So, how my table actually looks:
Many rows of the actual table
(The image above are selecting for 3 ou 4 machines. But, IN THIS EXAMPLE, the machines are not messed up. But can occurs! And in this case, LEAD/LAG doesn't care if the line above are machine-1 or machine-2)
What I want:
In that line 85, the [value] breaks and restart. Id like to count every occorrence when it happens, the selected machines.
So:
"Machine-1 restarted 6 times... Machine-9 restarted 10 times..."
I had done something LIKE this:
CREATE PROCEDURE CountProduction #Machine INT_ARRAY READONLY, #Start DATETIME, #End DATETIME AS
SET NOCOUNT ON
-- Declare counter and insert start values
DECLARE #Counter TABLE(
machine INT PRIMARY KEY,
lastValue BIGINT DEFAULT 0,
count BIGINT DEFAULT 0
)
INSERT INTO #Counter(machine) SELECT n FROM #Machine
-- Declare cursor to iteract over results of values log
DECLARE valueCursor CURSOR LOCAL FOR
SELECT
Value,
Aux.LastValue,
Aux.count
FROM
ValueLog,
#Machine AS Machine,
#Counter AS Counter
WHERE
ValueLog.Machine = Machine.n
AND Counter.machine = ValueLog.Machine
AND ValueLog.DateCreate BETWEEN #Start AND #End;
-- Start iteration
OPEN valueCursor
DECLARE #RowMachine INT
DECLARE #RowValue BIGINT
DECLARE #RowLastValue BIGINT
DECLARE #RowCount BIGINT
FETCH NEXT FROM valueCursor INTO #RowMachine, #RowValue, #RowLastValue, #RowCount
-- Iteration
DECLARE #increment INT
WHILE ##FETCH_STATUS = 0
BEGIN
IF #RowValue < #RowLastValue
SET #increment = 1
ELSE
SET #increment = 0
-- Update counters
UPDATE
#Counter
SET
lastValue = #RowValue,
count = count + #increment
WHERE
inj = #RowMachine
-- Proceed to iteration
FETCH NEXT FROM valueCursor INTO #RowMachine, #RowValue, #RowLastValue, #RowCount
END
-- Closing iteration
CLOSE valueCursor
DEALLOCATE valueCursor
SELECT machine, count FROM #Counter
Use LEAD(). If the next row < current row, count that occurrence.
Solved using #jeroen-mostert suggested
DECLARE #Start DATETIME
DECLARE #End DATETIME
SET #Start = '2019-01-01'
SET #End = GETDATE()
SELECT
Machine,
COUNT(DeltaValue) AS Prod
FROM
(SELECT
Log.Machine,
Log.Value - LAG(Log.Value) OVER (PARTITION BY Log.Machine ORDER BY Log.Id) AS DeltaValue
FROM
ValueLog AS Log,
(SELECT
Id,
Machine,
Value
FROM
ValueLog
) AS AuxLog
WHERE
AuxLog.Id = Log.Id
AND Proto.DateCreate BETWEEN #Start AND #End
AND Proto.Machine IN (1, 9, 10)
) as TB1
WHERE
DeltaValue < 0
GROUP BY
Machine
ORDER BY
Machine
In this case, the inner LAG/LEAD function didn't mess up the content (what happened for some reason when I created a view... I'll try to understand later).
Thanks everybody! I'm new at DB, and this question make me crazy for a whole day.

How to get and use the value returned by a stored procedure to a INSERT INTO... SELECT... statement

I am just new in SQL language and still studying it. I'm having hard time looking for answer on how can I use the stored procedure and insert value into a table.
I have this stored procedure:
CREATE PROCEDURE TestID
AS
SET NOCOUNT ON;
BEGIN
DECLARE #NewID VARCHAR(30),
#GenID INT,
#BrgyCode VARCHAR(5) = '23548'
SET #GenID = (SELECT TOP (1) NextID
FROM dbo.RandomIDs
WHERE IsUsed = 0
ORDER BY RowNumber)
SET #NewID = #BrgyCode + '-' + CAST(#GenID AS VARCHAR (30))
UPDATE dbo.RandomIDs
SET dbo.RandomIDs.IsUsed = 1
WHERE dbo.RandomIDs.NextID = #GenID
SELECT #NewID
END;
and what I'm trying to do is this:
INSERT INTO dbo.Residents([ResidentID], NewResidentID, [ResLogdate],
...
SELECT
[ResidentID],
EXEC TestID ,
[ResLogdate],
....
FROM
source.dbo.Resident;
There is a table dbo.RandomIDs containing random 6 digit non repeating numbers where I'm pulling out the value via the stored procedure and updating the IsUsed column of the table to 1. I'm transferring data from one database to another database and doing some processing on the data while transferring. Part of the processing is generating a new ID with the required format.
But I can't get it to work Sad I've been searching the net for hours now but I'm not getting the information that I need and that the reason for my writing. I hope someone could help me with this.
Thanks,
Darren
your question is little bit confusing, because you have not explained what you want to do. As i got your question, you want to fetch random id from randomids table and after performed some processing on nextid you want to insert it into resident table [newresidentid] and end of the procedure you fetch data from resident table. if i get anything wrong feel free to ask me.
your procedure solution is following.
CREATE PROCEDURE [TestId]
AS
SET NOCOUNT ON;
BEGIN
DECLARE #NEWID NVARCHAR(30)
DECLARE #GENID BIGINT
DECLARE #BRGYCODE VARCHAR(5) = '23548'
DECLARE #COUNT INTEGER
DECLARE #ERR NVARCHAR(20) = 'NO IDS IN RANDOM ID'
SET #COUNT = (SELECT COUNT(NEXTID) FROM RandomIds WHERE [IsUsed] = 0)
SET #GENID = (SELECT TOP(1) [NEXTID] FROM RandomIds WHERE [IsUsed] = 0 ORDER BY [ID] ASC)
--SELECT #GENID AS ID
IF #COUNT = 0
BEGIN
SELECT #ERR AS ERROR
END
ELSE
BEGIN
SET #NEWID = #BRGYCODE + '-' + CAST(#GENID AS varchar(30))
UPDATE RandomIds SET [IsUsed] = 1 WHERE [NextId] = #GENID
INSERT INTO Residents ([NewResidentId] , [ResLogDate] ) VALUES (#NEWID , GETDATE())
SELECT * FROM Residents
END
END
this procedure will fetch data from your randomids table and perform some processing on nextid than after it directs insert it into resident table and if you want to insert some data through user you can use parameter after declaring procedure name
E.G
CREATE PROCEDURE [TESTID]
#PARAM1 DATATYPE,
#PARAM2 DATATYPE
AS
BEGIN
END
I'm not convinced that your requirement is a good one but here is a way to do it.
Bear in mind that concurrent sessions will not be able to read your update until it is committed so you have to kind of "lock" the update so you will get a block until you're going to commit or rollback. This is rubbish for concurrency, but that's a side effect of this requirement.
declare #cap table ( capturedValue int);
declare #GENID int;
update top (1) RandomIds set IsUsed=1
output inserted.NextID into #cap
where IsUsed=0;
set #GENID =(select max( capturedValue) from #cap )
A better way would be to use an IDENTITY or SEQUENCE to solve your problem. This would leave gaps but help concurrency.

Create trigger that compares dates in SQL Server

I'm very new to triggers and I can't seem to wrap my head around them. Let's assume I have two tables here:
CREATE TABLE Project
(
id INT NOT NULL IDENTITY(1, 1),
startDate DATETIME,
endDate DATETIME
);
CREATE TABLE Work
(
date DATETIME,
projectId INT
);
I insert some data:
INSERT INTO Project VALUES ('2017-04-18', '2017-05-01'); /*id = 1*/
INSERT INTO Work VALUES ('2017-04-17', 1);
Assuming there's only 1 project with id = 1, this should go well. However, it doesn't make much sense that my work starts the day before the project starts (not in this case). How would I create a trigger that basically says date cannot be < startDate OR > endDate?
Something like this should work:
CREATE TRIGGER t_CheckInterval ON dbo.Work
AFTER UPDATE, INSERT
AS
IF NOT EXISTS (
--if no records are returned then work date lies outside the project
-- (start, end) interval
SELECT 1
FROM inserted AS i
JOIN Project AS p
ON p.Id = i.projectId AND i.[date] BETWEEN p.startDate AND p.endDate
)
BEGIN
RAISERROR ('Error: Your error message here.', 16, 1)
ROLLBACK TRANSACTION
END
GO
In your case instead of using trigger for these kind of checkings, I suggest to use a CHECK CONSTRAINT, something like this:
CREATE FUNCTION dbo.ufn_CheckWorkDate
(
#WorkDate DateTime,
#ProjectID INT
)
RETURNS BIT
AS
BEGIN
DECLARE #Result BIT
IF EXISTS (SELECT * FROM Project WHERE id = #ProjectID AND #WorkDate BETWEEN startdate AND endDate)
SET #Result = 1
ELSE
SET #Result = 0
RETURN #Result
END
GO
ALTER TABLE Work
WITH CHECK ADD CONSTRAINT CK_CheckWorkDate
CHECK (dbo.ufn_CheckWorkDate(date, projectid) = 1)

Reset the ID counter on a stored procedure in SQL Server

I'm developing a system that manages work orders for vehicles. The ID of work orders is composed as follows: OT-001-16.
Where OT- is a string, 001 is the counter, followed by - character and finally the number 16 is the current year.
Example:
If the current year is 2018, the ID should be OT-001-18.
The problem is when the year changes, the counter must restart from 001. I have a stored procedure to do that, but i think i'm doing a lot more work.
This is my stored procedure code:
CREATE PROCEDURE ot (#name varchar(100), #area varchar(100), #idate varchar(100), #edate varchar(100))
AS
BEGIN
SET NOCOUNT ON;
DECLARE #aux varchar(100);
DECLARE #aux2 varchar(100);
DECLARE #aux3 int;
DECLARE #aux4 varchar(100);
SELECT #aux = id_workorder FROM idaux;
IF (#aux IS NULL)
SET #aux = CONCAT('OT-000-', RIGHT(YEAR(GETDATE()), 2));
SET
#aux2 = SUBSTRING(
#aux, CHARINDEX('-', #aux) + 1,
LEN(#aux) - CHARINDEX('-', #aux) - CHARINDEX('-', REVERSE(#aux)));
SET #aux3 = CAST(#aux2 AS int) + 1;
SET #aux4 = #aux3;
IF #aux3 < 1000
IF #aux3 >= 10
SET #aux4 = CONCAT('0', #aux4);
ELSE
SET #aux4 = CONCAT('00', #aux4);
ELSE
SET #aux4 = #aux4;
DECLARE #f varchar(100);
DECLARE #y varchar(50);
SELECT TOP 1
#y = id_workorder
FROM workorder
WHERE (RIGHT(id_workorder, 2)) = (RIGHT(YEAR(GETDATE()), 2))
ORDER BY id_workorder DESC;
DECLARE #yy varchar(10);
SET
#yy = RIGHT(#y, 2);
DECLARE #yn varchar(10);
SET
#yn = RIGHT(YEAR(GETDATE()), 2);
BEGIN
IF #yn = #yy
BEGIN
DECLARE #laux varchar(20)
SET #f = 'OT-' + #aux4 + '-' + RIGHT(YEAR(GETDATE()), 2);
INSERT INTO workorder (id_workorder, name, area, initial_date, end_date)
VALUES (#f, #name, #area, #idate, #edate);
SELECT
#laux = id_workorder
FROM idaux
IF (#laux IS NULL)
BEGIN
INSERT idaux (id_workorder) VALUES (#f);
END
ELSE
BEGIN
UPDATE idaux SET id_workorder = #f;
END
END
ELSE
BEGIN
SET #f = CONCAT('OT-001-', (RIGHT(YEAR(GETDATE()), 2)));
INSERT INTO workorder (id_workorder, name, area, initial_date, end_date)
VALUES (#f, #name, #area, #idate, #edate);
SELECT #laux = id_workorder FROM idaux;
IF (#laux IS NULL)
BEGIN
INSERT idaux (id_workorder) VALUES (#f);
END
ELSE
BEGIN
UPDATE idaux SET id_workorder = #f;
END
END
END
END
Basically, i created an auxiliar table to save the last Work Order ID, then from this table called idaux i take the ID and i compared to new possible ID by a string handling. Then if the year of the last ID saved are equal to the current year the counter increases, but if not the counter is restarted to 001, the new ID is updated in the auxiliar table and the Work Order is inserted to the table workorder.
My stored procedure works, but i need your help to optimize the stored procedure. Any question post on comments.
Here is how I'd setup the stored procedure and the underlying table to keep track of your work orders:
create database tmpWorkOrders;
go
use tmpWorkOrders;
go
/*
The work order ID (as you wish to see it) and the
work order counter (per year) will be separated into
two separate columns (with a unique constraint).
The work order ID (you wish to see) is automatically
generated for you and stored "persisted":
http://stackoverflow.com/questions/916068/sql-server-2005-computed-column-is-persisted
*/
create table WorkOrders
(
SurrogateKey int identity(1, 1) primary key not null,
WorkOrderYear int not null,
WorkOrderCounter int not null,
WorkOrderID as
N'OT-' + right(N'000' + cast(WorkOrderCounter as nvarchar), 3)
+ N'-' + right(cast(WorkOrderYear as nvarchar), 2)persisted,
WorkOrderDescription nvarchar(200),
constraint UQ_WorkOrderIDs
unique nonclustered (WorkOrderYear, WorkOrderCounter)
);
go
create procedure newWorkOrder
(#WorkOrderYear int = null,
#WorkOderCounter int = null,
#WorkOrderDescription nvarchar(200) = null
)
as
begin
/*
If no year is given the the current year is assumed
*/
if #WorkOrderYear is null
begin
set #WorkOrderYear = year(current_timestamp);
end;
/*
If no work order counter (for the above year) is given
then the next available one will be given
*/
if #WorkOderCounter is null
begin
set #WorkOderCounter
= isnull(
(
select max(WorkOrderCounter)
from WorkOrders
where WorkOrderYear = #WorkOrderYear
) + 1,
0
);
end;
else
/*
If a work order counter has been passed to the
stored procedure then it must be validated first
*/
begin
/*
Does the work order counter (for the given year)
already exist?
*/
if exists
(
select 1
from dbo.WorkOrders as wo
where wo.WorkOrderYear = #WorkOrderYear
and wo.WorkOrderCounter = #WorkOderCounter
)
begin
/*
If the given work order counter already exists
then the next available one should be assigned.
*/
while exists
(
select 1
from dbo.WorkOrders as wo
where wo.WorkOrderYear = #WorkOrderYear
and wo.WorkOrderCounter = #WorkOderCounter
)
begin
set #WorkOderCounter = #WorkOderCounter + 1;
end;
end;
end;
/*
The actual insert of the new work order ID
*/
insert into dbo.WorkOrders
(
WorkOrderYear,
WorkOrderCounter,
WorkOrderDescription
)
values
(#WorkOrderYear,
#WorkOderCounter,
#WorkOrderDescription
);
end;
go
/*
Some test runs with the new table and stored procedure...
*/
exec dbo.newWorkOrder #WorkOrderYear = null,
#WorkOderCounter = null,
#WorkOrderDescription = null;
exec dbo.newWorkOrder #WorkOrderYear = null,
#WorkOderCounter = 3,
#WorkOrderDescription = null;
exec dbo.newWorkOrder #WorkOrderYear = null,
#WorkOderCounter = 0,
#WorkOrderDescription = null;
exec dbo.newWorkOrder #WorkOrderYear = null,
#WorkOderCounter = 0,
#WorkOrderDescription = null;
exec dbo.newWorkOrder #WorkOrderYear = null,
#WorkOderCounter = 0,
#WorkOrderDescription = null;
/*
...reviewing the result of the above.
*/
select *
from dbo.WorkOrders as wo;
Note, that the "next available" work order counter is once given (1) as the maximum + 1 and once (2) increased until it does not violate the unique key constraint on the table anymore. Like this you have two different possibilities to go about it.
There are a number of observations based on your code that you could alter to optimize and guarantee your results.
I am not aware of your Table Structure, but it seems you are using natural keys for your IDs.
Instead, use a surrogate key, such as INT/BIGINT to not only add efficiency in your table joins (no strings required), but potentially add another layer of security in your current design.
Alternatively, normalize the column into the flags they are. For example: OT-001-05 has three elements: OT is a type of work order, 001 is the ID, and 15 is the year. Presently, OT determines the ID which determines the year.
SELECT #aux = id_workorder FROM idaux;
idaux was not described. Is it a single value? If tabular, guarantee the result or it might break in the future.
Even if you add MAX(id_workorder), your result will not work as you think. Since this is a VARCHAR, the greatest value of the leftmost character not tied will return.
#aux, CHARINDEX('-', #aux) + 1,
LEN(#aux) - CHARINDEX('-', #aux) - CHARINDEX('-', REVERSE(#aux)));
This is fine, but overall you could make the code clearer and easier to debug by splitting all three of those elements into their own variable. Your still using your method, but simplified a little (personally, CHARINDEX can be a pain).
SET #aux = #Type -- 'OT'
SET #aux2 = #ID -- The result of your previous code
SET #aux3 = #Year -- your YY from GETDATE()
-- then join
SET #Work_Order = CONCAT(#aux, '-', #aux2, '-', #aux3)
Update:
Currently, your column in idaux has the ID in the MIDDLE of your column. This will produce disastrous results since any comparison of IDs will happen in the middle of the column. This means at best you might get away with PATINDEX but are still performing a table scan on the table. No index (save for FULLTEXT) will be utilized much less optimized.
I should add, if you put the ID element into its own column, you might find using BINARY collations on the column will improve its performance. Note I have not tested attempting a BINARY collation on a mixed column

how can we reduce the cost of temp table scan while feching count of records by using sql server

here just i am trying to know count of records in the temp table and passing to another variable, so that it was showing in the execution plan table scan about 100 %
please find the below query which i was doing
DECLARE #tmpANHdr TABLE (
hdrId INT IDENTITY,
CBFId NVARCHAR(32),
ACPT_REJ_FLAG NVARCHAR(8),
PROC_FILE_NAME NVARCHAR(50))
INSERT INTO #tmpANHdr
SELECT TOP 100 AHR_CB_FTS_FILE_ID,
AHR_ACCT_REJ_FLAG,
AHR_PROC_FILE_NAME
FROM FTS_ACK_NAK_HEADER WITH (NOLOCK)
WHERE AHR_FLAG IS NULL
OR AHR_FLAG = 0
DECLARE #varRecordCount INT
SELECT #varRecordCount = Count(1)
FROM #tmpANHdr
SET #varIndex = 1
IF( #varIndex <= #varRecordCount )
BEGIN
PRINT 'hi'
END
You could put a primary key (index) on the identity column and that might make it perform better:
DECLARE #tmpANHdr TABLE (
hdrId INT IDENTITY PRIMARY KEY,
CBFId NVARCHAR(32),
ACPT_REJ_FLAG NVARCHAR(8),
PROC_FILE_NAME NVARCHAR(50))
Also, do you end up using the table variable for anything else? Why not just do this?
DECLARE #varRecordCount INT
SELECT #varRecordCount = Count(TOP 100 *)
FROM FTS_ACK_NAK_HEADER WITH (NOLOCK)
WHERE AHR_FLAG IS NULL
OR AHR_FLAG = 0
A scan on a table variable with no more than 100 rows is likely to be pretty cheap anyway. The table scan operation might be costed at 100% of the plan for the statement but that is 100% of a pretty small number.
You can avoid this cost entirely though by just looking at ##rowcount after the insert as below.
DECLARE #varRecordCount INT;
INSERT INTO #tmpANHdr
SELECT TOP 100 AHR_CB_FTS_FILE_ID,
AHR_ACCT_REJ_FLAG,
AHR_PROC_FILE_NAME
FROM FTS_ACK_NAK_HEADER WITH (NOLOCK)
WHERE AHR_FLAG IS NULL
OR AHR_FLAG = 0
SET #varRecordCount = ##ROWCOUNT

Resources