I want to left join but only if it does not have the specific child.
Here is my current query:
SELECT "chatRoom"."id" as id,
"chatRoom"."name" as name,
"chatRoom"."type" as type,
"chatRoom"."description" as description,
"chatRoom"."thumbnail" as thumbnail,
"chatRoom"."status" as status,
chats.[unreadCount] as unreadCount
FROM "chat_room" "chatRoom"
LEFT JOIN "chat_room_participant" "participants" ON "participants"."chatRoomId"="chatRoom"."id"
LEFT JOIN (
SELECT "chatRoom"."id" AS "chatRoomId", COUNT(readBy.id) AS "unreadCount"
FROM "chat" "chat"
LEFT JOIN "chat_room" "chatRoom" ON "chatRoom"."id"="chat"."chatRoomId"
LEFT JOIN "chat_read_by_chat_room_participant" "chat_readBy" ON "chat_readBy"."chatId"="chat"."id"
LEFT JOIN "chat_room_participant" "readBy" ON "readBy"."id"="chat_readBy"."chatRoomParticipantId"
WHERE NOT(readBy.UserId IN ('ca774a5f-a04d-ec11-ae58-74d83e04f9d3'))
OR "readBy"."id" IS NULL
GROUP BY "chatRoom"."id"
) "chats" ON chats.chatRoomId = "chatRoom"."id"
WHERE ('ALL' = 'ALL' OR "chatRoom"."status" = 'ALL')
AND "chatRoom"."applicationId" = '4ac752e9-004c-ec11-ae53-74d83e04f9d3'
AND "participants"."userId" = 'A97D66C4-014C-EC11-AE53-74D83E04F9D3'
ORDER BY "chatRoom"."lastUpdate" DESC
In this query, it returns the correct value only if the readBy is null or contains 1 entry which equals to the given userId.
So here I have the userId. I want to get all the unread chats count from each chatRoom that has the user as a participant.
The schema is like this:
chatRoom has many chats
chatRoom has many participants
chats has many to many `readBy` (participants)
So there is a column automatically created for the many-to-many relation:
column: chat_read_by_chat_room_participant
Contains:
|chatId|chatRoomParticipantId|
In my query above, the left join will get any readBy from another user:
WHERE NOT(readBy.UserId IN ('ca774a5f-a04d-ec11-ae58-74d83e04f9d3')) OR "readBy"."id" IS NULL GROUP BY "chatRoom"."id") "chats" ON chats.chatRoomId = "chatRoom"."id"
which I do not want. This will return the entry if the user already read the chat, but another users have read as well. I want to return the entry only if the user has not read the chat.
How can I do this?
CREATION QUERY
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[chat](
[id] [uniqueidentifier] NOT NULL,
[sentAt] [bigint] NOT NULL,
[type] [nvarchar](255) NOT NULL,
[status] [nvarchar](255) NOT NULL,
[message] [nvarchar](max) NOT NULL,
[filePath] [nvarchar](max) NULL,
[chatRoomId] [uniqueidentifier] NOT NULL,
[senderId] [nvarchar](255) NOT NULL,
[userId] [uniqueidentifier] NULL,
CONSTRAINT [PK_9d0b2ba74336710fd31154738a5] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[chat] ADD CONSTRAINT [DF_9d0b2ba74336710fd31154738a5] DEFAULT (newsequentialid()) FOR [id]
GO
ALTER TABLE [dbo].[chat] WITH CHECK ADD CONSTRAINT [FK_52af74c7484586ef4bdfd8e4dbb] FOREIGN KEY([userId])
REFERENCES [dbo].[chat_room_participant] ([id])
GO
ALTER TABLE [dbo].[chat] CHECK CONSTRAINT [FK_52af74c7484586ef4bdfd8e4dbb]
GO
ALTER TABLE [dbo].[chat] WITH CHECK ADD CONSTRAINT [FK_e49029a11d5d42ae8a5dd9919a2] FOREIGN KEY([chatRoomId])
REFERENCES [dbo].[chat_room] ([id])
GO
ALTER TABLE [dbo].[chat] CHECK CONSTRAINT [FK_e49029a11d5d42ae8a5dd9919a2]
GO
CREATE TABLE [dbo].[chat_read_by_chat_room_participant](
[chatId] [uniqueidentifier] NOT NULL,
[chatRoomParticipantId] [uniqueidentifier] NOT NULL,
CONSTRAINT [PK_f3dd24628d4644dd6e79bcd03d1] PRIMARY KEY CLUSTERED
(
[chatId] ASC,
[chatRoomParticipantId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[chat_read_by_chat_room_participant] WITH CHECK ADD CONSTRAINT [FK_011624ccd7e7b0281ef629f2930] FOREIGN KEY([chatId])
REFERENCES [dbo].[chat] ([id])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[chat_read_by_chat_room_participant] CHECK CONSTRAINT [FK_011624ccd7e7b0281ef629f2930]
GO
ALTER TABLE [dbo].[chat_read_by_chat_room_participant] WITH CHECK ADD CONSTRAINT [FK_2e33e3de9d7c91d426c09a24810] FOREIGN KEY([chatRoomParticipantId])
REFERENCES [dbo].[chat_room_participant] ([id])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[chat_read_by_chat_room_participant] CHECK CONSTRAINT [FK_2e33e3de9d7c91d426c09a24810]
GO
CREATE TABLE [dbo].[chat_room](
[id] [uniqueidentifier] NOT NULL,
[name] [nvarchar](255) NULL,
[type] [nvarchar](255) NOT NULL,
[thumbnail] [nvarchar](max) NULL,
[description] [nvarchar](max) NULL,
[status] [nvarchar](255) NOT NULL,
[applicationId] [uniqueidentifier] NOT NULL,
[lastUpdate] [bigint] NULL,
CONSTRAINT [PK_8aa3a52cf74c96469f0ef9fbe3e] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[chat_room] ADD CONSTRAINT [DF_8aa3a52cf74c96469f0ef9fbe3e] DEFAULT (newsequentialid()) FOR [id]
GO
ALTER TABLE [dbo].[chat_room] WITH CHECK ADD CONSTRAINT [FK_2226638e6b7665ec0259d246b2b] FOREIGN KEY([applicationId])
REFERENCES [dbo].[application] ([id])
GO
ALTER TABLE [dbo].[chat_room] CHECK CONSTRAINT [FK_2226638e6b7665ec0259d246b2b]
GO
CREATE TABLE [dbo].[chat_room_participant](
[id] [uniqueidentifier] NOT NULL,
[joinedAt] [bigint] NOT NULL,
[privilege] [nvarchar](255) NOT NULL,
[chatRoomId] [uniqueidentifier] NOT NULL,
[userId] [uniqueidentifier] NOT NULL,
CONSTRAINT [PK_15913cf37a762fce4c8d6a32a42] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY],
CONSTRAINT [UQ_ae9630d66f6c5d12afd1a991fec] UNIQUE NONCLUSTERED
(
[chatRoomId] ASC,
[userId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[chat_room_participant] ADD CONSTRAINT [DF_15913cf37a762fce4c8d6a32a42] DEFAULT (newsequentialid()) FOR [id]
GO
ALTER TABLE [dbo].[chat_room_participant] WITH CHECK ADD CONSTRAINT [FK_9f718459ea81b5130f81980ca08] FOREIGN KEY([userId])
REFERENCES [dbo].[user] ([id])
GO
ALTER TABLE [dbo].[chat_room_participant] CHECK CONSTRAINT [FK_9f718459ea81b5130f81980ca08]
GO
ALTER TABLE [dbo].[chat_room_participant] WITH CHECK ADD CONSTRAINT [FK_fb664f48a4ec615ec5cce90a25d] FOREIGN KEY([chatRoomId])
REFERENCES [dbo].[chat_room] ([id])
GO
ALTER TABLE [dbo].[chat_room_participant] CHECK CONSTRAINT [FK_fb664f48a4ec615ec5cce90a25d]
GO
It looks like you just need a WHERE NOT EXISTS correlated subquery.
Don't be tempted to just use LEFT JOIN IS NULL syntax, it is generally less efficient, as it hides from the optimizer that you are doing an anti-join. Occasionally it can be useful though.
Further notes:
Don't quote column and table names unless you have to. And generally avoid names that need quoting.
Don't alias columns that don't need aliasing.
Choose short meaningful table aliases.
Basic formatting and good use of whitespace helps readability
Your LEFT JOIN chat_room_participant logically becomes an INNER JOIN because of the WHERE.
You don't need to re-join chat_room in the grouped subquery, you can just group by the join column.
You may want to use a grouped OUTER APPLY instead of that subquery. It is unlikely to get a different query plan, but it can be easier to read.
COUNT(SomeNotNullValue) is the same as COUNT(*)
('ALL' = 'ALL' OR cr.status = 'ALL') only makes sense if cr.Status is nullable, otherwise you would just use cr.status = 'ALL'. Even if it is nullable, you may as well use (cr.Status IS NULL OR cr.status = 'ALL')
SELECT
cr.id,
cr.name,
cr.type,
cr.description,
cr.thumbnail,
cr.status,
c.unreadCount
FROM chat_room cr
JOIN chat_room_participant p ON p.chatRoomId = cr.id
LEFT JOIN (
SELECT
c.chatRoomId AS chatRoomId,
COUNT(*) AS unreadCount
FROM chat c
WHERE NOT EXISTS (SELECT 1
FROM chat_read_by_chat_room_participant chat_readBy
JOIN chat_room_participant readBy ON readBy.id = chat_readBy.chatRoomParticipantId
WHERE chat_readBy.chatId = c.id
AND readBy.UserId IN ('ca774a5f-a04d-ec11-ae58-74d83e04f9d3')
)
GROUP BY c.chatRoomId
) c ON c.chatRoomId = cr.id
WHERE (cr.Status IS NULL OR cr.status = 'ALL')
AND cr.applicationId = '4ac752e9-004c-ec11-ae53-74d83e04f9d3'
AND p.userId = 'A97D66C4-014C-EC11-AE53-74D83E04F9D3'
ORDER BY
cr.lastUpdate DESC;
It seems that what you want is an anti-semi join. That is, a join to demonstrate that a row does not exist.
The two typical methods are...
main_table AS m
LEFT JOIN
other_table AS o
ON o.m_id = m.id
AND o.user = 'xyz'
WHERE
o.id IS NULL
Or...
main_table AS m
WHERE
NOT EXISTS (
SELECT *
FROM other_table AS o
WHERE o.m_id = m.id
AND o.user = 'xyz'
)
Exactly how to apply this to your example is unclear as you have cluttered your question with too many other details. (It is not a Minimal Verifiable Example.)
We have some SQL "normal" and graph tables, a script that syncs the information between them.
Everything is working ok from SSMS but when building the database project in Visual Studio using msbuild we get warnings (see code and warnings details below).
If we set TreatTSqlWarningsAsErrors to True these warnings become errors.
We don't want to ignore warnings but its unclear why we even get them.
Are these warnings correct?
Why are they showing in Visual Studio only and not in SSMS?
How can we resolve them without ignoring them?
The details:
We have the following two "normal" SQL tables:
CREATE TABLE [dbo].[Currency]
(
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](250) NOT NULL,
[UId] [uniqueidentifier] NOT NULL,
CONSTRAINT [PK_Currency]
PRIMARY KEY CLUSTERED ([Id] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON,
ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 90, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[Currency]
ADD CONSTRAINT [DF_Currency_UId] DEFAULT (NEWID()) FOR [UId]
GO
and
CREATE TABLE [dbo].[Portfolio]
(
[Id] [INT] IDENTITY(1,1) NOT NULL,
[Name] [NVARCHAR](250) NULL,
[CurrencyId] [INT] NOT NULL,
[UId] [UNIQUEIDENTIFIER] NOT NULL,
CONSTRAINT [PK_Portfolio]
PRIMARY KEY CLUSTERED ([Id] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON,
ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 90, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[Portfolio]
ADD CONSTRAINT [DF_Portfolio_UId] DEFAULT (NEWID()) FOR [UId]
GO
ALTER TABLE [dbo].[Portfolio] WITH CHECK
ADD CONSTRAINT [FK_Portfolio_Currency]
FOREIGN KEY([CurrencyId]) REFERENCES [dbo].[Currency] ([Id])
GO
ALTER TABLE [dbo].[Portfolio] CHECK CONSTRAINT [FK_Portfolio_Currency]
GO
and we created the following SQL graph schema together with two SQL node tables and one SQL edge:
CREATE SCHEMA [graph] ;
CREATE TABLE [graph].[Currency]
(
[Id] UNIQUEIDENTIFIER NOT NULL,
PRIMARY KEY CLUSTERED ([Id] ASC),
INDEX [GRAPH_UNIQUE_INDEX_4FFC60C0FCBE4843A7F4B9AB0729FF78] UNIQUE NONCLUSTERED ($node_id)
) AS NODE;
CREATE TABLE [graph].[Portfolio]
(
[Id] UNIQUEIDENTIFIER NOT NULL,
PRIMARY KEY CLUSTERED ([Id] ASC),
INDEX [GRAPH_UNIQUE_INDEX_F39F92BE8DD34DD791D6CA955DA0DA9A] UNIQUE NONCLUSTERED ($node_id)
) AS NODE;
CREATE TABLE [graph].[isOf]
(
[IsActive] BIT CONSTRAINT [DF_isOf_IsActive] DEFAULT ((1)) NOT NULL,
INDEX [GRAPH_UNIQUE_INDEX_AD8C5B40D277413580EDD943AC192869] UNIQUE NONCLUSTERED ($edge_id)
) AS EDGE;
CREATE UNIQUE NONCLUSTERED INDEX [UQ_FromTo]
ON [graph].[isOf] ($from_id, $to_id) ON [PRIMARY];
GO
A sync stored procedure which will populate the nodes and the edge from the "normal" tables that will be executed daily.
Attached parts of it:
PRINT ( 'Sync graph.[Portfolio]' );
INSERT INTO graph.Portfolio ( Id )
SELECT P.[UId]
FROM dbo.Portfolio P
WHERE NOT EXISTS ( SELECT 1
FROM graph.Portfolio G
WHERE G.Id = P.[UId] );
PRINT ( 'Sync graph.[Currency]' );
INSERT INTO graph.Currency ( Id )
SELECT C.[UId]
FROM dbo.Currency C
WHERE NOT EXISTS ( SELECT 1
FROM graph.Currency G
WHERE G.Id = C.[UId] );
PRINT('Sync Portfolio isOf Currency edge');
;
WITH UidCTE
AS ( SELECT P.[UId] AS PortfolioUid ,
C.[UId] AS CurrencyUid
FROM dbo.Portfolio P
JOIN dbo.Currency C ON C.Id = P.CurrencyId )
MERGE graph.isOf AS TGT
USING graph.[Portfolio] AS SourceFrom
JOIN UidCTE CTE ON SourceFrom.Id = CTE.PortfolioUid
JOIN graph.[Currency] AS SourceTo ON CTE.CurrencyUid = SourceTo.Id
ON MATCH(SourceFrom-(TGT)->SourceTo)
WHEN NOT MATCHED BY TARGET THEN
INSERT ( $from_id , $to_id )
VALUES ( SourceFrom.$node_id, SourceTo.$node_id)
WHEN MATCHED AND TGT.[IsActive] = 0 THEN
UPDATE SET TGT.[IsActive] = 1;
and the following script which updates the edge's IsActive column in case the relationship doesn't exist anymore in the "normal" tables(we don't want to delete it from the graph nodes/edges)
;WITH PortfolioIsOfCurrency
AS ( SELECT P.[UId] AS PortfolioUid ,
C.[UId] AS CurrencyUid
FROM dbo.Portfolio P
JOIN dbo.Currency C ON C.Id = P.CurrencyId ) ,
NodeCTE
AS ( SELECT P.$node_id AS FromNode ,
C.$node_id AS ToNode
FROM PortfolioIsOfCurrency CTE
JOIN graph.Portfolio P ON P.Id = CTE.PortfolioUid
JOIN graph.Currency C ON C.Id = CTE.CurrencyUid )
UPDATE ISOF
SET isOf.IsActive = 0
FROM graph.isOf ISOF
WHERE NOT EXISTS ( SELECT 1
FROM NodeCTE N
WHERE N.FromNode = ISOF.$from_id
AND N.ToNode = ISOF.$to_id );
We created a database project in VS 2019 with the following settings:
<DSP>Microsoft.Data.Tools.Schema.Sql.Sql150DatabaseSchemaProvider</DSP>
<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
<TreatWarningsAsErrors>False</TreatWarningsAsErrors>
<TreatTSqlWarningsAsErrors>False</TreatTSqlWarningsAsErrors>
From SSMS everything works perfect.
The problem is with the last syntax(UPDATE isof.active) I get the following warnings:
Warning SQL71502: Procedure: [graph].[SyncEngagementContentGraphs] has an unresolved reference to object [graph].[isOf].[N]. graph\Stored Procedures\SyncEngagementContentGraphs.sql 57
Warning SQL71502: Procedure: [graph].[SyncEngagementContentGraphs] has an unresolved reference to object [graph].[isOf].[N]. graph\Stored Procedures\SyncEngagementContentGraphs.sql 58
Warning SQL71509: The model already has an element that has the same name NodeCTE.$node_id. graph\Stored Procedures\SyncEngagementContentGraphs.sql 47
Warning SQL71509: The model already has an element that has the same name NodeCTE.$node_id. graph\Stored Procedures\SyncEngagementContentGraphs.sql 48
and if we set TreatTSqlWarningsAsErrors to True these warnings become errors and we cannot leave it like this (set to False) in the long run.
The solution is to define correctly the NodeCTE common table expression with columns names.
This seems to be mandatory in the case of a CTE with graph tables.
The correct query whithout warnings would be:
;WITH PortfolioIsOfCurrency
AS ( SELECT P.[UId] AS PortfolioUid ,
C.[UId] AS CurrencyUid
FROM dbo.Portfolio P
JOIN dbo.Currency C ON C.Id = P.CurrencyId ) ,
**NodeCTE(FromNode, ToNode)**
AS ( SELECT P.$node_id AS FromNode ,
C.$node_id AS ToNode
FROM PortfolioIsOfCurrency CTE
JOIN graph.Portfolio P ON P.Id = CTE.PortfolioUid
JOIN graph.Currency C ON C.Id = CTE.CurrencyUid )
UPDATE ISOF
SET isOf.IsActive = 0
FROM graph.isOf ISOF
WHERE NOT EXISTS ( SELECT 1
FROM NodeCTE N
WHERE N.FromNode = ISOF.$from_id
AND N.ToNode = ISOF.$to_id );
This change is needed just for the VS SqlProject warnings. Both versions work with no problems with SSMS/ADS.
My INSERT statement fails while it is trying to add a new record into an empty table (Attribute) (no record yet).
I am surprised by the error raised by the system:
Violation of UNIQUE KEY constraint 'CK_Attribute_Name_IDproject'. Cannot insert duplicate key in object 'dbo.Attribute'. The duplicate key value is (dummy, 55).
The creation script for this table looks like
CREATE TABLE [dbo].[Attribute](
[ID] [int] IDENTITY(1,1) NOT NULL,
[IDproject] [int] NOT NULL,
[IDtype] [int] NOT NULL,
[IDgroup] [int] NOT NULL,
[name] [varchar](50) NOT NULL,
[color] [int] NULL,
[protected] [tinyint] NULL,
[datemodified] [datetime] NOT NULL,
PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [CK_Attribute_Name_IDproject] UNIQUE NONCLUSTERED
(
[name] ASC,
[IDproject] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
I skiped foreign keys references and default values which does not seem of interest in this context.
The UNIQUE constraint applies to [name] and [IDproject].
When running the following statement
SELECT *
FROM [dbo].[Attribute]
GO
SELECT *
FROM [dbo].[Project]
GO
I get the results
(0 row(s) affected)
(2 row(s) affected)
The first result indicats the Attribute Table is empty
The second that there are 2 Projects
then running the following INSERT in table Attribute it failed with the above mentioned UNIQUE CONSTRAINT error
INSERT INTO [dbo].[Attribute] ([IDproject], [name], [IDtype], [IDgroup], [color], [protected], [datemodified])
SELECT DISTINCT
p.[ID],'dummy',t.[ID],g.[ID],-1,0,getdate()
FROM [dbo].[Project] p
INNER JOIN [dbo].[Group] g ON g.[name]='none' AND g.[IDproject] = p.[ID]
INNER JOIN [dbo].[AttributeType] t ON t.[format]='text' AND g.[IDproject] = p.[ID]
WHERE p.[name]='TESTPROJ'
GO
How can i get such an error on an empty table ?
I have found the solution myself: the derived SELECT returns 2 records with 'dummy' due to a duplicate INTO one of table, AttributeType, with which INNER JOIN is performed.
I have two tables: "Alarms" and "Devices" and a view "vwAlarms"
Alarms Table has 250K rows and Devices Table has only 50 rows.
vwAlarms is just join of two table.
my problems is when i add Top(x) and order by id desc to select * from vwAlarm it takes 10 seconds to execute query. however the same query runs quickly on table Alarm.
select * from Alarm --in milliseconds.
select * from vwAlarms --in milliseconds
select top (100) * from Alarms order by id desc --in milliseconds
select top (100) * from vwAlarms order by id desc --takes 10 seconds
Here is my view definition:
CREATE VIEW [dbo].[vwAlarms]
AS SELECT
dbo.Devices.Id ,
dbo.Devices.Name ,
dbo.Devices.PortsTagPrefix ,
dbo.Devices.ControlCenterNumber ,
dbo.Devices.AlarmNumber1 ,
dbo.Devices.AlarmNumber2 ,
dbo.Devices.SimCardNumber ,
dbo.Devices.StationNumber ,
dbo.Devices.SlaveId ,
dbo.Devices.TypeId ,
dbo.Devices.RegionId ,
dbo.Devices.EnquiryPassword ,
dbo.Devices.SetupPassword ,
dbo.Devices.ProtocolId ,
dbo.Devices.UploadedPacketsCount ,
dbo.Devices.LastPort ,
dbo.Devices.LastIp ,
dbo.Devices.IsForTesting ,
dbo.Devices.Latitude ,
dbo.Devices.Longitude ,
dbo.Devices.X ,
dbo.Devices.Y ,
dbo.Devices.MainSchematicId ,
dbo.Devices.MainTimeChartId ,
dbo.Devices.MainCategoryChartId ,
dbo.Alarms.Id ,
dbo.Alarms.DeviceId ,
dbo.Alarms.LogId ,
dbo.Alarms.PortId ,
dbo.Alarms.TypeId ,
dbo.Alarms.DateTime ,
dbo.Alarms.AcknowledgerId ,
dbo.Alarms.AcknowledgeDateTime ,
dbo.Alarms.Acknowledged ,
dbo.Alarms.PortValue
FROM Devices
INNER JOIN Alarms ON Devices.Id = Alarms.DeviceId
ORDER BY dbo.Alarms.Id DESC
Here is execution plan:
Warning message of of sort:
Alarm table Schema:
CREATE TABLE [dbo].[Alarms](
[Id] [int] IDENTITY(1,1) NOT NULL,
[DeviceId] [int] NOT NULL,
[LogId] [int] NOT NULL,
[PortId] [int] NOT NULL,
[TypeId] [int] NOT NULL,
[DateTime] [datetime2](0) NOT NULL,
[AcknowledgerId] [int] NULL,
[AcknowledgeDateTime] [datetime2](0) NULL,
[Acknowledged] [bit] NULL,
[PortValue] [numeric](19, 4) NULL,
CONSTRAINT [PK_Alarms] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[Alarms] WITH CHECK ADD CONSTRAINT [FK_Alarms_AlarmTypes] FOREIGN KEY([TypeId])
REFERENCES [dbo].[AlarmTypes] ([Id])
GO
ALTER TABLE [dbo].[Alarms] CHECK CONSTRAINT [FK_Alarms_AlarmTypes]
GO
ALTER TABLE [dbo].[Alarms] WITH CHECK ADD CONSTRAINT [FK_Alarms_Devices] FOREIGN KEY([DeviceId])
REFERENCES [dbo].[Devices] ([Id])
GO
ALTER TABLE [dbo].[Alarms] CHECK CONSTRAINT [FK_Alarms_Devices]
GO
ALTER TABLE [dbo].[Alarms] WITH CHECK ADD CONSTRAINT [FK_Alarms_ExtendedUsers] FOREIGN KEY([AcknowledgerId])
REFERENCES [dbo].[ExtendedUsers] ([Id])
GO
ALTER TABLE [dbo].[Alarms] CHECK CONSTRAINT [FK_Alarms_ExtendedUsers]
GO
ALTER TABLE [dbo].[Alarms] WITH CHECK ADD CONSTRAINT [FK_Alarms_Logs] FOREIGN KEY([LogId])
REFERENCES [dbo].[Logs] ([Id])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER
Is using the view a must? If not, you probably should first get your 100 alarms and then join with devices. That's the end result you want right?
It is not being smart about the sort.
Do you have the FK defined?
I would try all option of the table hints on the join to try and push that sort earlier.
Join Hints (Transact-SQL)
If table hints does not work I would try a Cross Apply.
I think the Cross Apply should be smart about sort.
But at the cost of not being as fast on the "join".
So it would be good at returning the first 1000 or 10,000 but bad at all.
SELECT t1.*, t2o.*
FROM t1
CROSS APPLY
(
SELECT *
FROM t2
WHERE t2.t1_id = t1.id
) t2o
SCENARIO
I need to select records from test_userData based on a 1-to-1 match from test_userCheck on the columns customer or account_info. The code below will create a mock-up of the tables and will populate with random data for the purpose of my question. Based on this code, it's looking for any records where test_userData.customer = 'Guerrero, Unity' or test_userData.account_info = 'XXXXXXXXXXXXXXXX0821', and should return three rows (confirmation_id = 6836985, 5502798, and 3046441)
PROBLEM
As it stands, the query returns what I need... however, my real userData table has almost 2 million records, and the userCheck table has about 10,000. The query takes about 7 seconds as it is and I think that's way too long. I'm also worried because the userData table will start to grow quickly (by tens of thousands of unique records a day), and I envision my current method becoming unmanageable.
QUESTION
Any ideas on how I can optimize this to scale with millions of records? The data resides on a shared SQL 2008 server with limited permissions.
--setup temporary testing tables
IF EXISTS
(
SELECT * FROM dbo.sysobjects
WHERE id = object_id(N'[dbo].[test_userData]')
AND OBJECTPROPERTY(id, N'IsUserTable') = 1
)
DROP TABLE [dbo].[test_userData]
GO
IF EXISTS
(
SELECT * FROM dbo.sysobjects
WHERE id = object_id(N'[dbo].[test_userCheck]')
AND OBJECTPROPERTY(id, N'IsUserTable') = 1
)
DROP TABLE [dbo].[test_userCheck]
GO
CREATE TABLE [dbo].[test_userData](
[id] [int] IDENTITY(1,1) NOT NULL,
[merchant_id] [int] NOT NULL,
[sales_date] [datetime] NOT NULL,
[confirmation_id] [int] NOT NULL,
[customer] [nvarchar](max) NOT NULL,
[total] [smallmoney] NOT NULL,
[account_info] [nvarchar](max) NOT NULL,
[email_address] [nvarchar](max) NOT NULL
CONSTRAINT [PK_test_userData] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[test_userCheck](
[confirmation_id] [int] NOT NULL,
[customer] [nvarchar](max) NOT NULL,
[total] [smallmoney] NOT NULL,
[account_info] [nvarchar](max) NOT NULL
CONSTRAINT [PK_test_userCheck] PRIMARY KEY CLUSTERED
(
[confirmation_id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
--insert some random user transactions
INSERT INTO [dbo].[test_userData] (merchant_id,sales_date,confirmation_id,customer,total,account_info,email_address) VALUES
('99','03/25/2010','3361424','Soto, Ahmed','936','XXXXXXXXXXXXXXXX8744','Donec.egestas#NullainterdumCurabitur.ca'),
('17','09/12/2010','6710165','Holcomb, Eden','1022','XXXXXXXXXXXXXXXX6367','Curabitur#dolortempus.org'),
('32','05/04/2010','4489509','Foster, Nasim','1463','XXXXXXXXXXXXXXXX7115','augue.eu.tellus#ullamcorperviverraMaecenas.ca'),
('95','01/02/2011','5384061','Browning, Owen','523','XXXXXXXXXXXXXXXX0576','sed.dictum.eleifend#accumsaninterdum.edu'),
('91','08/21/2010','6075234','Dawson, McKenzie','141','XXXXXXXXXXXXXXXX3580','dolor.sit.amet#etmagnis.org'),
('63','01/29/2010','1055619','Mathews, Keefe','1110','XXXXXXXXXXXXXXXX2682','ligula#Sednuncest.edu'),
('27','10/20/2010','1819662','Clarke, Briar','1474','XXXXXXXXXXXXXXXX7481','Donec.non.justo#malesuada.org'),
('82','03/05/2010','3184936','Holman, Dana','560','XXXXXXXXXXXXXXXX7080','Aenean.eget.magna#accumsan.edu'),
('24','06/11/2010','1007427','Kirk, Desiree','206','XXXXXXXXXXXXXXXX3681','parturient#at.com'),
('49','06/17/2010','6137066','Foley, Sopoline','1831','XXXXXXXXXXXXXXXX1718','ac.urna.Ut#pellentesqueafacilisis.org'),
('22','05/08/2010','3545367','Howell, Uriel','638','XXXXXXXXXXXXXXXX1945','ad.litora#arcuvelquam.ca'),
('5','10/25/2010','6836985','Little, Caryn','743','XXXXXXXXXXXXXXXX0821','Suspendisse.aliquet#auctor.org'),
('91','06/16/2010','6852582','Buckner, Chiquita','99','XXXXXXXXXXXXXXXX1533','tellus.sem#semvitaealiquam.edu'),
('63','06/12/2010','7930230','Nolan, Wyoming','1192','XXXXXXXXXXXXXXXX1291','Sed#diam.org'),
('32','02/01/2010','8407102','Cummings, Deacon','1315','XXXXXXXXXXXXXXXX4375','a.odio.semper#massaSuspendisseeleifend.ca'),
('75','06/29/2010','5502798','Guerrero, Unity','858','XXXXXXXXXXXXXXXX8000','eget#lectus.edu'),
('50','09/13/2010','8312525','Russo, Yvette','1680','XXXXXXXXXXXXXXXX2046','In.mi#eu.com'),
('11','04/13/2010','6204132','Small, Calista','426','XXXXXXXXXXXXXXXX0269','lacus#Cumsociisnatoque.org'),
('16','01/01/2011','7522507','Mosley, Thor','1459','XXXXXXXXXXXXXXXX8451','netus.et#Pellentesqueutipsum.com'),
('5','01/27/2010','1472120','Case, Kiona','1419','XXXXXXXXXXXXXXXX7097','Duis#duilectusrutrum.edu'),
('70','02/17/2010','1095935','Snyder, Tanner','1655','XXXXXXXXXXXXXXXX8556','metus.sit.amet#inconsequatenim.edu'),
('63','11/10/2010','3046441','Guerrero, Unity','629','XXXXXXXXXXXXXXXX0807','nonummy.ac.feugiat#Phasellusdapibus.org'),
('22','08/19/2010','5435100','Turner, Patrick','1133','XXXXXXXXXXXXXXXX6734','pede#Duis.edu'),
('96','10/05/2010','6381992','May, Dominic','1858','XXXXXXXXXXXXXXXX7227','hymenaeos#etcommodo.edu'),
('96','02/26/2010','8630748','Chandler, Olympia','1016','XXXXXXXXXXXXXXXX4001','sed.dui.Fusce#pellentesqueSed.com');
--insert a random fraud transaction to check against (based on customer and account_info only)
INSERT INTO [dbo].[test_userCheck] (confirmation_id, customer, total, account_info) VALUES
('2055015', 'Guerrero, Unity', '20.02', 'XXXXXXXXXXXXXXXX0821');
--get result, which is correct
SELECT a.confirmation_id, a.customer, a.total, a.account_info, a.email_address
FROM dbo.test_userData AS a RIGHT OUTER JOIN
dbo.test_userCheck AS b ON a.customer = b.customer OR a.account_info = b.account_info;
DROP TABLE [dbo].[test_userData];
DROP TABLE [dbo].[test_userCheck];
Create the appropriate index or indices. Just based on your question, I'd suggest two indices, one on test_userData.customer, and a second index on test_userData.account_info
Creating indexes would probably help, but have you considered another design that complies with normal forms. It would be better if you access the date through index on a integer column instead of string...