I have been tasked to replicate a large amount of data being entered into our application database to see how it would perform.
I have produced the following while loop,
DECLARE #DATABASESID uniqueidentifier
SET #DATABASESID = (SELECT SID FROM mcm_system)
DECLARE #START_MIN_PROCESS INT;
SET #START_MIN_PROCESS = 200000;
WHILE #START_MIN_PROCESS > 1
BEGIN
BEGIN TRAN
INSERT INTO READING (ID, SID, POINT_DETAIL_ID, TAKEN, VALUE, STATUS, POINT_DATA_ID, WHO, MACHINE_STATE, RESTORED_ID, JOTTING, TAG, VALUEA, VALUEB, VALUEC)
SELECT
NEWID(), #DATABASESID, Point_Detail.ID,
DATEADD(MINUTE, -#START_MIN_PROCESS, GETDATE()), ABS(CHECKSUM(NewId())) % 60,
'N', '00000000-0000-0000-0000-000000000000',
#USERID, '', '00000000-0000-0000-0000-000000000000',
'', 'N', 0, 0, 0
FROM
POINT_DETAIL
WHERE
TYPE = 0
PRINT DATEADD(MINUTE, -#START_MIN_PROCESS, GETDATE())
SET #START_MIN_PROCESS = #START_MIN_PROCESS - 15;
COMMIT
END
The point_detail table referenced has 9000 records and the loop is designed to insert a record with a random value, and a date in the past for every point_detail record. The date is then incremented 15 minutes into the future and the loop runs again. I use the Print to keep track of progress as I am trying to replicate a years worth of data.
It runs fine at first and eventually grinds to a halt. Is there any way I can optimize my statement or another alternative method?
Related
I wrote some SQL statements that work for updating a single customer. I have to update all the customers when this code gets pushed out.
Right now the customer ID is hardcoded and the SQL statements insert one record based on that ID. Prototype works, now I want to do like 10,000 inserts for all of the customers using the same algorithm.
DECLARE #customerID BIGINT = 47636;
DECLARE #limitFourAdjustment MONEY;
DECLARE #appliesToDateTime DATETIME2(7) = SYSUTCDATETIME();
DECLARE #dp_y INT = DATEPART(YEAR, #appliesToDateTime);
DECLARE #dp_m INT = DATEPART(MONTH, #appliesToDateTime);
DECLARE #dp_w INT = DATEPART(WEEK, #appliesToDateTime);
DECLARE #dp_d INT = DATEPART(DAY, #appliesToDateTime);
DECLARE #dp_h INT = DATEPART(HOUR, #appliesToDateTime);
DECLARE #d_h DATETIME2(7) = DATEADD(HOUR, DATEDIFF(HOUR, 0, #appliesToDateTime), 0);
SELECT
#limitFourAdjustment = -COALESCE(SUM(COALESCE(Amount, 0)), 0)
FROM
[dbo].Transactions
WHERE
CustomerID = #customerID AND
IsSystemVoid = 0 AND
TransactionTypeID IN (SELECT ID FROM TransactionTypes WHERE TransactionTypeGroupID = 3)
INSERT INTO dbo.CustomerAccounts_TransactionSummation (CustomerID, LimitTypeID, Y, M, W, D, H, YMDH, Amount)
VALUES (#customerID, 4, #dp_y, #dp_m, #dp_w, #dp_d, #dp_h, #d_h, #limitFourAdjustment);
I tried adding a while loop, seems like not the fastest solution. Maybe collect the ID's first and then feed it to through the loop? My first attempt below doesn't work because I just get the last customer ID, not a unique one every time.
SELECT #numberOfCustomers = COUNT(*)
FROM dbo.Customers
WHILE(#numberOfCustomers > 0)
BEGIN
SELECT #customerID = ID FROM dbo.Customers
OTHER LOGIC FROM ABOVE
SET #numberOfCustomers = #numberOfCustomers - 1;
END
So the question is, how to run these SQL statements (first code block) on every customer's ID?
The key to working with databases is getting your mind around set based operations as opposed to procedural operations. Databases are designed to operate naturally on sets of data at a time, but you have to change how you think about the problem to one where you are manipulating the entire set of data as opposed to one record at a time.
So here is the SQL which I think carry out your complete update in one hit:
INSERT INTO dbo.CustomerAccounts_TransactionSummation (CustomerID, LimitTypeID, Y, M, W, D, H, YMDH, Amount)
SELECT
id
, 4
, #dp_y
, #dp_m
, #dp_w
, #dp_d
, #dp_h
, #d_h
, -COALESCE(SUM(COALESCE(Amount, 0)), 0) limitFourAdjustment
FROM [dbo].Transactions
WHERE IsSystemVoid = 0
and TransactionTypeID IN (SELECT ID FROM TransactionTypes WHERE TransactionTypeGroupID = 3)
--and CustomerID = #customerID
Note that the insert can be combined directly with a select as opposed to using values.
I want to create reports like below picture's report on SSRS.
Yellow parts mean SET_PHASE,
Green parts mean PROD_PHASE
And my query result like this:
I want to show for per line, all order and I want to show for per order, SETUP and PRODUCTION depends on duratıon time.
SET_PHASE's duration time is SET_DURATION,
PROD_PHASE's duration time is PROD_DURATION
I hope so my query is clear :) Could you help me about issue?
Answer:
Hello Alan,
Current situation I have just these data:
PROD100059335 SETUP PRODUCTION 1 14 LINE 4
PROD100058991 SETUP PRODUCTION 1 5 LINE 6
PROD100059259 SETUP PRODUCTION 2 24 LINE 4
PROD100059188 SETUP PRODUCTION 1 3 LINE 2
PROD100059248 SETUP PRODUCTION 1 15 LINE 2
PROD100059055 SETUP PRODUCTION 2 23 LINE 2
PROD100058754 SETUP PRODUCTION 5 18 LINE 6
And If I use your query I just show "PROD100058754", "PROD100059259", "PROD100059055" these order. I don't understand why other data lost.
until "DECLARE #n TABLE(n int)" part I can show other data. but after that I can not show.
And I applied procedure on SSRS my report shows like this:
I couldn't do correctly and I don't know how can I fix them:(
for example "PROD100059259" this order normally has setup phase but on the report I don't have yellow field.
Do you have any suggestions for me?
OK, here is an attempt to give you what you want but there are a few caveats:
The durations are scaled and no operation can take less than 1 time slot so the setup vs production duration is only approximate
I haven't found a good way of labelling each bar so I've used tooltips
First the code... I've added lots of comments so hopefully you can follow it thru, it's based on your sample data.
NOTE: I've update the table as it now seems like you are using integer durations rather than the 00:00 format from your first example.
-- CREATE A TEST TABLE AND POPULATE IT
DECLARE #data TABLE(STR_ORDER_ID varchar(20), SET_DURATION varchar(10), PROD_DURATION varchar(10), Set_decimal int, Prod_Decimal int, Line varchar(10))
INSERT INTO #data
VALUES
('PROD100059335', NULL, NULL, 1, 14, 'LINE 4'),
('PROD100058991', NULL, NULL,1, 5, 'LINE 6'),
('PROD100059259', NULL, NULL,2, 24, 'LINE 4'),
('PROD100059188', NULL, NULL,1, 3, 'LINE 2'),
('PROD100059248', NULL, NULL,1, 15, 'LINE 2'),
('PROD100059055', NULL, NULL,2, 23, 'LINE 2'),
('PROD100058754', NULL, NULL,5, 18, 'LINE 6')
DECLARE #Gap int = 2 -- determines how many columns we use to separate each order
-- ASSUME durations are in hours/minutes or minutes/seconds and convert them to decimal minutes or decimal seconds respectively
-- COMMENTED THIS AS WE NO LONGER NEED IT. No longer required as durations are now integer values.
--UPDATE d
-- SET
-- Set_decimal = (CAST(LEFT(d.SET_DURATION, len(d.SET_DURATION)-3) AS INT) * 60) + CAST(RIGHT(d.SET_DURATION, 2) AS INT) ,
-- Prod_Decimal = (CAST(LEFT(d.PROD_DURATION, len(d.PROD_DURATION)-3) AS INT) * 60) + CAST(RIGHT(d.PROD_DURATION, 2) AS INT)
--FROM #data d
-- CREATE A NORMALISED TABLE, this will just help to make the next steps simpler
DECLARE #normData TABLE(RowId INT IDENTITY (1,1), Line varchar(10), STR_ORDER_ID varchar(20), OperationOrder int, Operation varchar(10), Duration int)
INSERT INTO #normData (Line, STR_ORDER_ID, OperationOrder, Operation, Duration)
SELECT * FROM (
SELECT Line, STR_ORDER_ID, 1 as OperationOrder , 'SET' as Operation , Set_decimal FROM #data
UNION
SELECT Line, STR_ORDER_ID, 2 , 'PROD' , Prod_decimal FROM #data
UNION
SELECT Line, STR_ORDER_ID, 3 , 'GAP' , #Gap FROM #data ) u -- this adds dummy data that will act as gaps in hte timeline. Change 5 to whatever value suits you best
ORDER BY Line, STR_ORDER_ID, OperationOrder
-- find the largest line running total duration per line and scale it to fit to 240 (so we dont go over 256 column limit in SSRS)
DECLARE #MaxDur INT = (SELECT MAX(rt) FROM (
select *
, SUM(Duration) OVER(PARTITION BY Line ORDER BY Line, STR_ORDER_ID, OperationOrder) AS Rt
from #normData) mRt)
-- Now scale the values back so they fit but don't let any value become less than 1
IF #MaxDur > 240
BEGIN
UPDATE nd
SET Duration = CASE WHEN nd.Duration / (#MaxDur/240) <1 THEN 1 ELSE nd.Duration / (#MaxDur/240) END
FROM #normData nd
END
/* check what we have so far by uncommenting this bit
select *
, SUM(Duration) OVER(PARTITION BY Line ORDER BY Line, STR_ORDER_ID, OperationOrder) AS Rt
from #normData
--*/
-- ================================================================ --
-- At this point you 'may' have enough data to plot a bar chart. == --
-- ================================================================ --
-- CREATE A SIMPLE NUMBERS TABLE, we'll need this to act as our time series
DECLARE #n TABLE(n int)
DECLARE #i int = 0
DECLARE #t int = #MaxDur --(SELECT max(Duration) +5 FROM #normData) -- simple loop counter target set to slightly bigger than our highest duration
WHILE #i<#t
BEGIN
INSERT INTO #n SELECT #i
SET #i = #i +1
END
-- Join our numbers table to our real data
-- This will give us at least 1 row per time slot and associated activity during that time slot.
-- We can plot this driectly as a matrix.
SELECT *
FROM #n n
LEFT JOIN (
-- Sub queries below give use a runnintg total, we then join this back to itself to get the previous
-- running total and this will give us the 'time range' for each operation.
SELECT
a.*
, ISNULL(b.Rt,0)+1 AS TimeStart
, a.RT AS TimeEnd
FROM
(SELECT *
, SUM(Duration) OVER(PARTITION BY Line ORDER BY Line, STR_ORDER_ID, OperationOrder) AS Rt
from #normData
) a
LEFT JOIN
(SELECT *
, SUM(Duration) OVER(PARTITION BY Line ORDER BY Line, STR_ORDER_ID, OperationOrder) AS Rt
from #normData
) b
ON a.RowId = b.RowId + 1 and a.Line = b.Line
) d
ON n.n between d.TimeStart and d.TimeEnd
ORDER BY Line, STR_ORDER_ID, OperationOrder, n, TimeStart, TimeEnd
You can use the code above in your dataset.
The report design:
The report is very simple. It's a matrix with a single row group based on Line and a single column group based on n which is our time slot number.
I've added a blank row to act as a spacer between the 'bars'.
The expression of the cell background is
=SWITCH(
Fields!OperationOrder.Value = 1, "Yellow",
Fields!OperationOrder.Value = 2, "Green",
Fields!OperationOrder.Value = 3, Nothing,
True, Nothing
)
There is also a tooltip which displays STR_ORDER_ID and the operation name.
You get the following output.
I understand from sources like this that GETDATE() should always (well, eventually, depending on how fast the loop is) return a different value in a loop.
I have this TSQL:
DECLARE #WaitUntilTime DATETIME = DATEADD(SECOND, 10, GETDATE())
WHILE (DATEDIFF(SECOND, GETDATE(), #WaitUntilTime) > 0)
BEGIN
SELECT GETDATE(), #WaitUntilTime
END
But it's always outputting the same value for GETDATE(), and the loop never ends after 10 seconds like I want it to. Why?
I also found this answer which sounds like a similar case, but it talks about GETDATE() being evaluated once for a column in a SELECT query. I don't know if that applies to a while-loop as well.
By the way, I know about WAITFOR but I can't use it because I want to introduce a delay within a single batch.
DECLARE #WaitUntilTime DATETIME = DATEADD(SECOND, 10, GETDATE())
DECLARE #Dummy int
WHILE (DATEDIFF(SECOND, GetDate(), #WaitUntilTime ) > 0)
BEGIN
Set #Dummy=1
END
SELECT GetDate(), #WaitUntilTime
Returns after a 10 second delay
(No column name) (No column name)
2016-10-18 13:53:20.000 2016-10-18 13:53:20.140
This is the error I am getting:
Msg 512, Level 16, State 1, Procedure tr_UpdateFolio, Line 361
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
I've run up and down this code over and over for hours now. What am I not seeing? Any help would be appreciated. I've been working on this for a few days now, and this is pretty much the one thing I need working in order to get this project done.
-- 2. Write a trigger named tr_UpdateFolio that will be invoked when the Folio table 'status'
-- field ONLY (column update) is changed.
ALTER TRIGGER tr_UpdateFolio ON FOLIO--switch alter back to create
AFTER UPDATE
AS
IF UPDATE([Status])
BEGIN
DECLARE #Status char(1)
DECLARE #FolioID smallint
DECLARE #CheckinDate smalldatetime
DECLARE #Nights tinyint
DECLARE #CurrentDate smalldatetime = '7/27/2016 2:00:00 PM'
SELECT
#Status = i.Status,
#FolioID = i.FolioID,
#CheckinDate = i.CheckinDate,
#Nights = i.Nights
FROM
INSERTED i
-- If Folio status is updated to 'C' for Checkout, trigger two different Insert statements to
-- (1) INSERT in the Billing table, the amount for the total lodging cost as BillingCategoryID1
-- - (normally the FolioRate * number of nights stay, but you must also factor in any late checkout fees*).
-- *Checkout time is Noon on the checkout date. Guest is given a one hour
-- grace period to check out. After 1PM (but before 4PM), a 50% surcharge is added to the FolioRate. After 4PM, an
-- additional full night's FolioRate is applied. You can recycle code from A7 (part 5), but note it's not the exact same
-- function - we only need the late charge (if any).
IF #Status = 'C'
SET #Nights = 1
--#CurrentDate may need to switch back to getdate()
IF DATEDIFF(HOUR, #CheckinDate + #Nights, #CurrentDate) >= 16
SET #Nights = #Nights + 1
ELSE IF DATEDIFF(HOUR, #CheckinDate + #Nights, #CurrentDate) >= 13
SET #Nights = #Nights + .5
UPDATE FOLIO
SET Nights = #Nights
WHERE FolioID = #FolioID
INSERT INTO BILLING (FolioID, BillingCategoryID, BillingDescription, BillingAmount, BillingItemQty, BillingItemDate)
VALUES (25, 1, 'Room', dbo.GetRackRate(11, #CurrentDate) * #Nights, 1, #CurrentDate)
-- (2) The second INSERT statement in the same trigger will insert the Lodging Tax* - as a separate entry in the
-- Billing table for tax on lodging (BillingCategoryID2). *Use the dbo.GetRoomTaxRate function from A7 to determine
-- the Lodging Tax.
INSERT INTO BILLING (FolioID, BillingCategoryID, BillingDescription, BillingAmount, BillingItemQty, BillingItemDate)
VALUES (25, 2, 'Lodging Tax', dbo.GetRoomTaxRate(20), 1, #CurrentDate)
END
GO
-- 3. Write a trigger named tr_GenerateBill that will be invoked when an entry is INSERTED in to the Billing
-- table. If BillngCategoryID is 2 (for testing purposes only) then call the function dbo.ProduceBill (from A7).
ALTER TRIGGER tr_GenerateBill ON BILLING
AFTER INSERT
AS
BEGIN
DECLARE #FolioID smallint
DECLARE #BillingCategoryID smallint
SELECT #FolioID = i.FolioID, #BillingCategoryID = i.BillingCategoryID
FROM INSERTED i
IF #BillingCategoryID = 2
SELECT * FROM dbo.ProduceBill(#FolioID)
END
GO
dbo.producebill should be fine, but the error occurs when I try to run this block
-- 4A. Assume today is (July 27, 2016 at 2PM)*. Anita is due to check out today (from Part 1 above).
-- Write an Update Statement to change the status of her Folio to 'CheckedOut.
-- (Be careful to include a WHERE clause so ONLY here folio is updated).
-- Note: This should automatically invoke tr_UpdateFolio above (factoring in the late charge),
-- which automatically invokes tr_GenerateBill above, and calls dbo.ProduceBill , and produces a bill.
UPDATE FOLIO
SET [Status] = 'C'
WHERE ReservationID = 5020
I'm going nuts trying to figure this out. Thanks.
Let's start with the easier one (tr_GenerateBill). This one is a little odd for sure. You have a select statement in an insert trigger. This means that when you insert a row you are expecting it to return row(s). This is not the typical behavior of an insert but you can work with it.
If you insert 3 rows and only 2 of them have a BillingCategoryID of 2 what should the trigger do? What does that table valued function look like?
This is kind of a guess but you should be able to rewrite that entire trigger to something along these lines.
ALTER TRIGGER tr_GenerateBill ON BILLING
AFTER INSERT
AS
BEGIN
SELECT *
FROM inserted i
cross apply dbo.ProduceBill(i.FolioID) pb
where i.BillingCategoryID = 2
END
Your second trigger is more challenging. It is not totally clear what you are trying to do here but I think this is pretty close.
UPDATE f
set Nights = case when DATEDIFF(HOUR, #CheckinDate + i.Nights, #CurrentDate) >= 16 then 2 else 1 end --adding .5 to a tiny int is pointless. It will ALWAYS be the same value.
from inserted i
join Folio f on f.FolioID = i.FolioID
INSERT INTO BILLING (FolioID, BillingCategoryID, BillingDescription, BillingAmount, BillingItemQty, BillingItemDate)
Select i.FolioID
, 1
, 'Room'
, dbo.GetRackRate(11, #CurrentDate) * case when DATEDIFF(HOUR, #CheckinDate + i.Nights, #CurrentDate) >= 16 then 2 else 1 end, 1, #CurrentDate)
from inserted i
INSERT INTO BILLING (FolioID, BillingCategoryID, BillingDescription, BillingAmount, BillingItemQty, BillingItemDate)
select i.FolioID
, 2
, 'Lodging Tax'
, dbo.GetRoomTaxRate(20), 1, #CurrentDate)
from inserted i
I come from a programming background, and i'm having some difficulty wrapping my head around SQL's conditionals.
I'm looking to auto-generate emails when two criteria are met in my sql table.
If the value in column NextTestDate is not null AND today's date is equivalent to the next test date + 30 days THEN generate an email.
SELECT * FROM dbo.datmaintest
if NextTestDate is NOT NULL
AND DATEADD(day,30,DATEDIFF(day, 0, GETDATE())) = DATEADD(DAY, DATEDIFF(day, 0, NextTest), 0) then
BEGIN
use msdb
GO
EXEC sp_send_dbmail #profile_name='ControllerDB',
#recipients='test#test.com',
#subject='ITS ALIVE!',
#body='Time to grab lunch'
END
Think that you are looking for something like this.
IF EXISTS(
select *
FROM dbo.datmaintest
where DATEADD(day, 30, DATEDIFF(day, 0, GETDATE())) = NextTestDate
)
EXEC sp_send_dbmail #profile_name='ControllerDB'
, #recipients = 'test#test.com'
, #subject = 'ITS ALIVE!'
, #body = 'Time to grab lunch'
Something like this should get you started (I'm assuming that in the date logic portion, you meant to type 'NextTestDate' instead of 'NextTest'):
DECLARE #NextTestDate DATETIME
SET #NextTestDate = (SELECT TOP 1 NextTestDate FROM dbo.datmaintest)
--If the value is not null and today is equal to the value of 'next test date' + 30 days:
IF #NextTestDate IS NOT NULL
AND DATEDIFF(DAY, #NextTestDate, GETDATE()) = 30
BEGIN
use msdb
GO
EXEC sp_send_dbmail #profile_name='ControllerDB',
#recipients='test#test.com',
#subject='ITS ALIVE!',
#body='Time to grab lunch'
END
Basically, declare a variable and set it to the value from the database you want to check. Then, perform your check and if it is true, begin the email sending (and close it out with an END)