I need idea or a way to check if my schedule jobs work and insert the right data. I Have a function that call a stored procedure to count inserted data from today and insert into specific field.
like this
SET #male = (select count(idUser) from (select idUser from #tmpLog2 where sex = 1 AND CAST(catchTime as date) = #DATE group by idUser)u);
SET #female = (select count(idUser) from (select idUser from #tmpLog2 where sex = 0 AND CAST(catchTime as date) = #DATE group by idUser)u);
INSERT INTO CatchLog
(
[male],
[female]
)
VALUES
(
#male,
#female
)
the stored procedure works OK, but sometime when today have a lot of data, the result inserted 0 for male / female.
It's possible to have 0 data inserted when today data really no male / female.
but sometime it inserted 0 but there are male n female data. Anyone can help me to check, how to check if the data inserted true and give some report or insert into error Log if data inserted not true?
sorry for my bad english
I dont know what column are you using into Count() Function. But if you are using Nullable column into it then it may gives 0 even if record were added as column value is null.
Consider to use Primary key column or not Nullable column in count() argument.
Related
Lets say i have a small db table with only two fields. (MSSQL) Like this:
date (Date) daily_counter (Int)
-------------------------
2021-07-18 0
2021-07-18 1
2021-07-18 2
2021-07-19 0
I want to insert a new fifth row and insert value "2021-07-19" to the date field. And i want to know what the daily_counter is for my new row.
As you perhaps understand by the example, the daily_counter is supposed to auto increase, starting over each day.
So, since there is already a row with that date with the value 0 on the field daily_counter. I want to add 1 to daily_counter for my new row without sending the value 1 to the query.
How can i think when designing this type of table and data. Any hints appreciated. Thanks!
Kr
Gustav
UPDATE:
Ok, i think i got something that could work. The only downside would be when deleting and adding new rows, as the new id could be previosly used and deleted and added again.
A side from that i think i got something that i can use.
It might not be pretty now, but it looks like this.
It seems to work also when there is no row for the current day.
DECLARE #date DATE
SET #date = '2021-07-22'
DECLARE #daily_counter INT
SET #daily_counter = (SELECT MAX(daily_counter) from mytable where date = #date);
SET #daily_counter = #daily_counter + 1;
IF #daily_counter IS NULL
BEGIN
SET #daily_counter = 1;
END
INSERT INTO
mytable
(date, daily_counter)
OUTPUT #daily_counter
VALUES (#date, #daily_counter)
Thanks again for the help!
It's not possible to make the database do this automatically in the row itself. You must have a single counter across all dates (a SEQUENCE would be good for this).
What you can do is use the row_number() function to simulate this at the point where you query the data:
SELECT *, row_number() over (partition by [date] order by [date])
FROM ...
Unfortunately, this will still fail if you need to preserve the original position following deletes, but there's not a good way to do this right now in a database without triggers or application code.
SELECT * FROM
(SELECT IdNumber from LegacyTable
WHERE IdNumber != 0
ORDER BY IdNumber
OFFSET 0 ROWS
FETCH NEXT 10 ROWS ONLY
UNION
SELECT IdNumber FROM Customer
WHERE IdNumber != '0'
ORDER BY IdNumber
OFFSET 0 ROWS
FETCH NEXT 10 ROWS ONLY) as Customers
ORDER BY IdNumber
OFFSET 0 ROWS
FETCH NEXT 10 ROWS ONLY
I have two tables in my database. One Table contains legacy customer data and the other one is when a new customer signs up. I want to get all rows from BOTH tables but use offset/take for pagination.
If I don't paginate, the query works but it takes a VERY LONG TIME.
I am new to SQL Server but I've read that Pagination should help resolve the problem.
The above code gives me 10 records. However if I change the value of the OFFSET to 10, it ends up skipping rows.
If there is a better way, please let me know.
LegacyTable
CREATE TABLE LegacyTable (
IdNumber INT,
FirstName VARCHAR(40),
LastName VARCHAR(40)
)
CustomerTable
CREATE TABLE Customer (
IdNumber INT,
FirstName VARCHAR(40),
LastName VARCHAR(40)
)
The Legacy Table has 40 additional fields which was used by the mainframe application. I didnt list it here.
Your issue appears to be that you are using UNION instead of UNION ALL. This means that all the inner tables need to be fully evaluated for de-duplication before moving on to the outer query.
Putting an OFFSET on the inner tables also is going to mess up the results, for obvious reasons.
You could just put it on them, and get 10 rows for each table, but another option is to use UNION ALL, which can process rows without blocking.
I note that you are using != '0' on the second table, I would assume this is an int column, so you want to use a number here, not text:
SELECT * FROM
(SELECT IdNumber from LegacyTable
WHERE IdNumber != 0
UNION ALL
SELECT IdNumber FROM Customer
WHERE IdNumber != 0
) as Customers
ORDER BY IdNumber
OFFSET 0 ROWS
FETCH NEXT 10 ROWS ONLY
I have a table of 120 million rows. About 8 million of those rows are duplicates depending on what value/column I use to determine duplicates. For argument sake, I'm testing out the email column vs multiple columns to see what happens with my data.
The file is about 10GB, so I cannot simply add another table to the database because of the size limits of SQL Express. Instead, I thought I'd try to extract, truncate, insert using a temp table since I've been meaning to try that method out.
I know I can use CTE to remove the duplicates, but every single time I try to do that it takes forever and my system locks up. My solution is to do the following.
1.Extract all rows to tempdb
2.Sort by Min(id)
3.Truncate original table
4.Transfer new unique data from tempdb back to main table
5.Take the extra duplicates and trim to uniques using Delimit
6.Import the leftover rows back into the database.
My table looks like the following.
Name Gender Age Email ID
Jolly Female 28 jolly#jolly.com 1
Jolly Female 28 jolly#jolly.com 2
Jolly Female 28 jolly#jolly.com 3
Kate Female 36 kate#kate.com 4
Kate Female 36 kate#kate.com 5
Kate Female 36 kate#kate.com 6
Jack Male 46 jack#jack.com 7
Jack Male 46 jack#jack.com 8
Jack Male 46 jack#jack.com 9
My code
SET IDENTITY_INSERT test.dbo.contacts ON
GO
select name, gender, age, email, id into ##contacts
from test.dbo.contacts
WHERE id IN
(SELECT MIN(id) FROM test.dbo.contacts GROUP BY name)
TRUNCATE TABLE test.dbo.contacts
INSERT INTO test.dbo.contacts
SELECT name, gender, age, total_score, id
from ##students
SET IDENTITY_INSERT test.dbo.contactsOFF
GO
This code is almost working, except for the following error that I see.
"An explicit value for the identity column in table 'test.dbo.contacts' can only be specified when a column list is used and IDENTITY_INSERT is ON.
I have absolutely no idea why I keep seeing that message since I turned identity_insert on and off.
Can somebody please tell me what I'm missing in the code? And if anybody has another solution to keep unique rows I'd love to hear about it.
You said that your original problem was that " it takes forever and my system locks up".
The problem is the amount of time necessary for the operation and the lock escalation to table lock.
My suggestion is to break down the operation so that you delete less than 5000 rows at time.
I assume you have less than 5000 duplicates for each name.
You can read more about lock escalation here:
https://www.sqlpassion.at/archive/2014/02/25/lock-escalations/
About your problem (identity insert), your script contains at least two errors so I guess it's not the original one, so it hard to say why the original one fails.
use test;
if object_ID('dbo.contacts') is not null drop table dbo.contacts;
CREATE TABLE dbo.contacts
(
id int identity(1,1) primary key clustered,
name nvarchar(50),
gender varchar(15),
age tinyint,
email nvarchar(50),
TS Timestamp
)
INSERT INTO [dbo].[contacts]([name],[gender],[age],[email])
VALUES
('Jolly','Female',28,'jolly#jolly.com'),
('Jolly','Female',28,'jolly#jolly.com'),
('Jolly','Female',28,'jolly#jolly.com'),
('Kate','Female',36,'kate#kate.com'),
('Kate','Female',36,'kate#kate.com'),
('Kate','Female',36,'kate#kate.com'),
('Jack','Male',46,'jack#jack.com'),
('Jack','Male',46,'jack#jack.com'),
('Jack','Male',46,'jack#jack.com');
--for the purpose of the lock escalation, I assume you have less then 5.000 duplicates for each single name.
if object_ID('tempdb..#KillList') is not null drop table #KillList;
SELECT KL.*, C.TS
into #KillList
from
(
SELECT [name], min(ID) GoodID
from dbo.contacts
group by name
having count(*) > 1
) KL inner join
dbo.contacts C
ON KL.GoodID = C.id
--This has the purpose of testing concurrent updates on relevant rows
--UPDATE [dbo].[contacts] SET Age = 47 where ID=7;
--DELETE [dbo].[contacts] where ID=7;
while EXISTS (SELECT top 1 1 from #KillList)
BEGIN
DECLARE #id int;
DECLARE #name nvarchar(50);
DECLARE #TS binary(8);
SELECT top 1 #id=GoodID, #name=Name, #TS=TS from #KillList;
BEGIN TRAN
if exists (SELECT * from [dbo].[contacts] where id=#id and TS=#TS)
BEGIN
DELETE FROM C
from [dbo].[contacts] C
where id <> #id and Name = #name;
DELETE FROM #KillList where Name = #name;
END
ELSE
BEGIN
ROLLBACK TRAN;
RAISERROR('Concurrency error while deleting %s', 16, 1, #name);
RETURN;
END
commit TRAN;
END
SELECT * from [dbo].[contacts];
I wrote it this way, that you can see the sub results of each query.
The inner sql should not have *, instead use id.
delete from [contacts] where id in
(
select id from
(
select *, ROW_NUMBER() over (partition by name, gender, age, email order by id) as rowid from [contacts]
) rowstobedeleted where rowid>1
)
If this takes too long/makes much load, you can use SET ROWCOUNT to provide smaller chunks, but then you need to run it until nothing is delete anymore.
I think that you need something like this:
INSERT INTO test.dbo.contacts (idcol1,col2)
VALUES (value1,value2)
I am trying to constrain a SQL Server Database by a Start Date and End Date such that I can never double book a resource (i.e. no overlapping or duplicate reservations).
Assume my resources are numbered such that the table looks like
ResourceId, StartDate, EndDate, Status
So lets say I have resource #1. I want to make sure that I cannot have have the a reservation for 1/8/2017 thru 1/16/2017 and a separate reservation for 1/10/2017 - 1/18/2017 for the same resource.
A couple of more complications, a StartDate for a resource can be the same as the EndDate for the resource. So 1/8/1027 thru 1/16/2017 and 1/16/2017 thru 1/20/2017 is ok (i.e., one person can check in on the same day another person checkouts).
Furthermore, the Status field indicates whether the booking of the resource is Active or Cancelled. So we can ignore all cancelled reservations.
We have protected against these overlapping or double booking reservations in Code (Stored Procs and C#) when saving but we are hoping to add an extra layer of protection by adding a DB Contraint.
Is this possible in SQL Server ?
Thanks in Advance
You can use a CHECK constraint to make sure startdate is on or before EndDate easily enough:
CONSTRAINT [CK_Tablename_ValidDates] CHECK ([EndDate] >= [StartDate])
A constraint won't help with preventing an overlapping date range. You can instead use a TRIGGER to enforce this by creating a FOR INSERT, UPDATE trigger that rolls back the transaction if it detects a duplicate:
CREATE TRIGGER [TR_Tablename_NoOverlappingDates] FOR INSERT, UPDATE AS
IF EXISTS(SELECT * from inserted INNER JOIN [MyTable] ON blah blah blah ...) BEGIN
ROLLBACK TRANSACTION;
RAISERROR('hey, no overlapping date ranges here, buddy', 16, 1);
RETURN;
END
Another option is to create a indexed view that finds duplicates and put a unique constraint on that view that will be violated if more than 1 record exists. This is usually accomplished with a dummy table that has 2 rows cartesian joined to an aggregate view that selects the duplicate id-- thus one record with a duplicate would return two rows in the view with the same fake id value that has a unique index.
I've done both, I like the trigger approach better.
Drawing from this answer here: Date range overlapping check constraint.
First, check to make sure there are not existing overlaps:
select *
from dbo.Reservation as r
where exists (
select 1
from dbo.Reservation i
where i.PersonId = r.PersonId
and i.ReservationId != r.ReservationId
and isnull(i.EndDate,'20990101') > r.StartDate
and isnull(r.EndDate,'20990101') > i.StartDate
);
go
If it is all clear, then create your function.
There are a couple of different ways to write the function, e.g. we could skip the StartDate and EndDate and use something based only on ReservationId like the query above, but I will use this as the example:
create function dbo.udf_chk_Overlapping_StartDate_EndDate (
#ResourceId int
, #StartDate date
, #EndDate date
) returns bit as
begin;
declare #r bit = 1;
if not exists (
select 1
from dbo.Reservation as r
where r.ResourceId = #ResourceId
and isnull(#EndDate ,'20991231') > r.StartDate
and isnull(r.EndDate,'20991231') > #StartDate
and r.[Status] = 'Active'
group by r.ResourceId
having count(*)>1
)
set #r = 0;
return #r;
end;
go
Then add your constraint:
alter table dbo.Reservation
add constraint chk_Overlapping_StartDate_EndDate
check (dbo.udf_chk_Overlapping_StartDate_EndDate(ResourceId,StartDate,EndDate)=0);
go
Last: Test it.
I've written an Oracle DB Conversion Script that transfers Data from a previous singular table into a new DB with a main table and several child/reference/maintenance tables. Naturally, this more standardized layout (previous could have, say Bob/Storage Room/Ceiling as the [Location] value) has more fields than the old table and thus cannot be exactly converted over.
For the moment, I have inserted a record value (ex.) [NO_CONVERSION_DATA] into each of my child tables. For my main table, I need to set (ex.) [Color_ID] to 22, [Type_ID] to 57 since there is no explicit conversion for these new fields (annually, all of these records are updated, and after the next update all records will exist with proper field values whereupon the placeholder value/record [NO_CONVERSION_DATA] will be removed from the child tables).
I also similarly need to set [Status_Id] something like the following (not working):
INSERT INTO TABLE1 (STATUS_ID)
VALUES
-- Status was not set as Recycled, Disposed, etc. during Conversion
IF STATUS_ID IS NULL THEN
(CASE
-- [Owner] field has a value, set ID to 2 (Assigned)
WHEN RTRIM(LTRIM(OWNER)) IS NOT NULL THEN 2
-- [Owner] field has no value, set ID to 1 (Available)
WHEN RTRIM(LTRIM(OWNER)) IS NULL THEN 1
END as Status)
Can anyone more experienced with Oracle & PL/SQL assist with the syntax/layout for what I'm trying to do here?
Ok, I figured out how to set the 2 specific columns to the same value for all rows:
UPDATE TABLE1
SET COLOR_ID = 24;
UPDATE INV_ASSETSTEST
SET TYPE_ID = 20;
I'm still trying to figure out setting the STATUS_ID based upon the value in the [OWNER] field being NULL/NOT NULL. Coco's solution below looked good at first glace (regarding his comment, not the solution posted, itself), but the below causes each of my NON-NULLABLE columns to flag and the statement will not execute:
INSERT INTO TABLE1(STATUS_ID)
SELECT CASE
WHEN STATUS_ID IS NULL THEN
CASE
WHEN TRIM(OWNER) IS NULL THEN 1
WHEN TRIM(OWNER) IS NOT NULL THEN 2
END
END FROM TABLE1;
I've tried piecing a similar UPDATE statement together, but so far no luck.
Try with this
INSERT INTO TABLE1 (STATUS_ID)
VALUES
(
case
when TATUS_ID IS NULL THEN
(CASE
-- [Owner] field has a value, set ID to 2 (Assigned)
WHEN RTRIM(LTRIM(OWNER)) IS NOT NULL THEN 2
-- [Owner] field has no value, set ID to 1 (Available)
WHEN RTRIM(LTRIM(OWNER)) IS NULL THEN 1
END )
end);