Statistics update different based on compatibility modes - sql-server

Question

I'm trying to understand why statistics are behaving differently with compatibility between 130-150 and 160.
Here are two test scenarios
Test scenarios 1:
USE master;
GO
IF EXISTS(SELECT * FROM sys.databases WHERE name = 'StatisticsUpdating')
BEGIN
ALTER DATABASE StatisticsUpdating SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
DROP DATABASE StatisticsUpdating;
END
GO
CREATE DATABASE StatisticsUpdating;
GO
ALTER DATABASE StatisticsUpdating SET
COMPATIBILITY_LEVEL = 160,
AUTO_UPDATE_STATISTICS ON,
AUTO_CREATE_STATISTICS ON;
GO
USE StatisticsUpdating;
CREATE TABLE StatMan
(
Id INT NOT NULL IDENTITY PRIMARY KEY,
StatName VARCHAR(100),
StatDescription VARCHAR(200)
) ON [PRIMARY];
GO
CREATE TABLE StatInformation
(
Id INT NOT NULL IDENTITY PRIMARY KEY,
StatManId INT NOT NULL,
StatLineItem VARCHAR(200)
) ON [PRIMARY];
GO
SET NOCOUNT ON;
INSERT INTO StatMan(StatName, StatDescription)
VALUES('Test3', 'Test Description');
GO 1000
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
GO
SET NOCOUNT ON;
INSERT INTO StatMan(StatName, StatDescription)
VALUES('Test3', 'Test Description');
GO 60000
SELECT 'DONE';
When viewing the modification_counter with the below query:
USE StatisticsUpdating;
SELECT s.object_id, sp.modification_counter, s.name,
CONCAT('USE ', DB_NAME(), ';', ' DBCC SHOW_STATISTICS(', QUOTENAME(OBJECT_NAME(s.object_id)), ', ', QUOTENAME(s.name), ');') AS ViewHistogram
FROM sys.stats s
OUTER APPLY sys.dm_db_stats_properties(s.object_id, s.stats_id) sp
WHERE s.object_id = OBJECT_ID('StatMan');
I get a modification_counter of 60,000, as expected.
After running the below query:
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
modification_counter is0 as expected.
Everything is working as expected so far.
Now running the same test but switching the COMPATIBILITY_LEVEL to either 130, 140, or 150.
When viewing the modification_counter with the below query:
USE StatisticsUpdating;
SELECT s.object_id, sp.modification_counter, s.name,
CONCAT('USE ', DB_NAME(), ';', ' DBCC SHOW_STATISTICS(', QUOTENAME(OBJECT_NAME(s.object_id)), ', ', QUOTENAME(s.name), ');') AS ViewHistogram
FROM sys.stats s
OUTER APPLY sys.dm_db_stats_properties(s.object_id, s.stats_id) sp
WHERE s.object_id = OBJECT_ID('StatMan');
I get a modification_counter of NULL, which is not expected.
When I run
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
Once again modification_counter is NULL, which is not expected.
So I'm really confused about why modification_counter is never getting any modifications, no matter how many rows are in the table. Even running
USE StatisticsUpdating;
DBCC SHOW_STATISTICS([StatMan], [PK__StatMan__3214EC078268D31F]);
Shows no stats.
Test scenarios 2:
USE master;
GO
IF EXISTS(SELECT * FROM sys.databases WHERE name = 'StatisticsUpdating')
BEGIN
ALTER DATABASE StatisticsUpdating SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
DROP DATABASE StatisticsUpdating;
END
GO
CREATE DATABASE StatisticsUpdating;
GO
ALTER DATABASE StatisticsUpdating SET
COMPATIBILITY_LEVEL = 160,
AUTO_UPDATE_STATISTICS ON,
AUTO_CREATE_STATISTICS ON;
GO
USE StatisticsUpdating;
CREATE TABLE StatMan
(
Id INT NOT NULL IDENTITY PRIMARY KEY,
StatName VARCHAR(100),
StatDescription VARCHAR(200)
) ON [PRIMARY];
GO
CREATE TABLE StatInformation
(
Id INT NOT NULL IDENTITY PRIMARY KEY,
StatManId INT NOT NULL,
StatLineItem VARCHAR(200)
) ON [PRIMARY];
GO
SET NOCOUNT ON;
INSERT INTO StatMan(StatName, StatDescription)
VALUES('Test3', 'Test Description');
INSERT INTO StatInformation(StatManId, StatLineItem)
SELECT Id, CONCAT('Test Item', CAST(Id AS CHAR(200)))
FROM StatMan
WHERE Id NOT IN
(
SELECT StatManId
FROM StatInformation
);
GO 1000
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
GO
SET NOCOUNT ON;
INSERT INTO StatMan(StatName, StatDescription)
VALUES('Test3', 'Test Description');
GO 60000
SELECT 'DONE';
When viewing the modification_counter with the below query:
USE StatisticsUpdating;
SELECT s.object_id, sp.modification_counter, s.name,
CONCAT('USE ', DB_NAME(), ';', ' DBCC SHOW_STATISTICS(', QUOTENAME(OBJECT_NAME(s.object_id)), ', ', QUOTENAME(s.name), ');') AS ViewHistogram
FROM sys.stats s
OUTER APPLY sys.dm_db_stats_properties(s.object_id, s.stats_id) sp
WHERE s.object_id = OBJECT_ID('StatMan');
I get a modification_counter of 60,498, which is expected.
When I run
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
modification_counter is 0, as expected.
Now running the same test but switching the COMPATIBILITY_LEVEL to either 130, 140, or 150.
When viewing the modification_counter with the below query:
USE StatisticsUpdating;
SELECT s.object_id, sp.modification_counter, s.name,
CONCAT('USE ', DB_NAME(), ';', ' DBCC SHOW_STATISTICS(', QUOTENAME(OBJECT_NAME(s.object_id)), ', ', QUOTENAME(s.name), ');') AS ViewHistogram
FROM sys.stats s
OUTER APPLY sys.dm_db_stats_properties(s.object_id, s.stats_id) sp
WHERE s.object_id = OBJECT_ID('StatMan');
I get a modification_counter of 60,498, which is expected. Go so far!
When I run
SELECT *
FROM StatMan sm
WHERE sm.Id = 200
OPTION(RECOMPILE);
It's still at 60,498! It didn't do a stat update even though I was expecting it to.
So now I try
SELECT *
FROM StatMan sm
JOIN StatInformation si ON sm.Id = si.StatManId
WHERE sm.Id = 200
OPTION(RECOMPILE);
modification_counter is still at 60,498! Once again, still didn't update stats.
Than I tried
SELECT *
FROM StatMan sm
JOIN StatInformation si ON sm.Id = si.StatManId
WHERE sm.Id = 200
OPTION(RECOMPILE);
modification_counter once again at 60,498! Once again, still didn't update stats.
So to get stats to update, I ended up having to run
INSERT INTO StatInformation(StatManId, StatLineItem)
SELECT TOP 101 Id, CONCAT('Test Item', CAST(Id AS CHAR(200)))
FROM StatMan
WHERE Id NOT IN
(
SELECT StatManId
FROM StatInformation
);
Where SELECT TOP 101 is the magic number for table StatInformation to get a stat update.
So I'm not exactly sure why it's behaving differently between 130-150 and 160. Especially when all Microsoft documents say compatibility level 130 and above should behave the same.
Does anyone know what may be causing this?