sql query for non-existing entries - sql-server

Question

I have inherited a website and its corresponding database (SQL Server). The website uses stored procedures to pull data from the database. One of these stored procedures contains a pivot and it the pivot is taking over 4 hours to run. This is currently unacceptable. I am looking for help in replacing the pivot with standard SQL queries because I assume that will be faster and provide better performance.
Here is the pivot in question:
SELECT *
FROM (
SELECT ac.AID
,ac.CatName AS t
,convert(INT, ac.Code) AS c
FROM categories AS ac
) AS s
Pivot(Sum(c) FOR t IN (
[tob]
,[ecit]
,[tobwcom]
,[rnorm]
,[raddict]
,[rpolicy]
,[ryouth]
,[rhealth]
,…
)) AS p;
And the results of the pivot
| AID | tob | ecit | tobwcom | rnorm |
|-----------|-----------|------------|---------------|-------------|
| 1 | 1 | NULL | NULL | 0 |
| 2 | 1 | NULL | NULL | 1 |
| 3 | 1 | NULL | NULL | 0 |
| 4 | 1 | NULL | NULL | 0 |
| 5 | 1 | NULL | NULL | 0 |
| 6 | 1 | NULL | NULL | 1 |
Here’s the source table categories and some sample data:
CREATE TABLE categories(
ArticleID INTEGER NOT NULL
,ThemeID INTEGER NOT NULL
,ThemeName VARCHAR(7) NOT NULL
,Code BIT NOT NULL
,CreatedTime VARCHAR(7) NOT NULL
);
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (1,1,'tob',1,'57:30.7');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (1,2,'ecig',1,'03:58.3');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (1,5,'rnorm',0,'42:56.5');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (2,1,'tob',1,'57:30.7');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (2,2,'ecig',0,'03:58.3');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (2,5,'rnorm',1,'42:56.5');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (2,6,'raddict',0,'42:59.8');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (3,1,'tob',1,'57:30.7');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (3,2,'ecig',0,'03:58.3');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (3,5,'rnorm',0,'42:56.5');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (21,1,'tob',1,'57:30.7');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (21,2,'ecig',0,'03:58.3');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (21,5,'rnorm',0,'42:56.5');
INSERT INTO categories(ArticleID,ThemeID,ThemeName,Code,CreatedTime) VALUES (21,6,'raddict',0,'42:59.8');
And here’s the table containing the category names – (mytable for now)
CREATE TABLE mytable(
CatID INTEGER NOT NULL PRIMARY KEY
,CatName VARCHAR(7) NOT NULL
,CreatedTime DATETIME NOT NULL
);
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (1,'tob','2015-03-12 10:07:54.173');
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (2,'ecig','2015-05-18 11:48:16.297');
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (4,'tobwcom','2015-06-19 11:12:01.537');
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (5,'rnorm','2015-06-22 14:24:02.317');
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (6,'raddict','2015-06-22 14:24:13.957');
INSERT INTO mytable(CatID,CatName,CreatedTime) VALUES (7,'ecit','2015-06-22 14:26:18.437');
What I need is a way to perform the pivot’s ability to find the non-existing data in categories. The output would be something like:
| AID | tob | ecit | tobwcom | rnorm |
|-----------|-----------|------------|---------------|-------------|
| 1 | 1 | NULL | NULL | 0 |
| 2 | 1 | NULL | NULL | 1 |
Or the list of AIDs and the CatNames that don’t have any values. Such as:
| AID | CatName |
|-----|---------|
| 1 | ecit |
| 1 | tobwcom |
| 2 | ecit |
| 2 | tobwcom |
I have tried
select distinct(AID) FROM [categories]
where [CatName] not in ( 'ecit', 'tobwcom')
but the results from this, the numbers don't seem to add up, however this could be an error on my part.

Answer 1

Not sure if it would be fast enough for such a huge table. But for that second expected result then something this could help to find the missing.
select a.ArticleID, c.CatName
from #myarticles a
cross join #mycategories c
left join categories ca on (ca.ArticleID = a.ArticleID and ca.ThemeID = c.CatID)
where ca.ArticleID is null;
A test can be found here
Note that this method benefits from a combined primary key index on (ArticleID, ThemeID)
As an alternative, the LEFT JOIN with a NULL check can be changed to a NOT EXISTS.
select a.ArticleID, c.CatName
from #myarticles a
join #mycategories c on c.CatID between 1 and 7
where NOT EXISTS
(
select 1
from categories ca
where ca.ArticleID = a.ArticleID
and ca.ThemeID = c.CatID
);