Automapper ProjectTo<> not working with Count() - sql-server

Question

I have an odd issue with AutoMapper (I'm using .NET core 3.1 and AutoMapper 10.1.1)
I'm doing a simple project to list and a simple projected count for total records:
var data = Db.Customers
.Skip((1 - 1) * 25)
.Take(25)
.ProjectTo<CustomerViewModel>(Mapper.ConfigurationProvider)
.ToList();
var count = Db.Customers
.ProjectTo<CustomerViewModel>(Mapper.ConfigurationProvider)
.Count();
The first line creates the expected SQL:
exec sp_executesql N'SELECT [c].[Code], [c].[Id], [c].[Name], [c].[Website], [s].Name
FROM [Customers] AS [c]
INNER JOIN [Status] AS [s] ON [s].id = [c].StatusId
ORDER BY (SELECT 1)
OFFSET #__p_0 ROWS FETCH NEXT #__p_1 ROWS ONLY',N'#__p_0 int,#__p_1 int',#__p_0=0,#__p_1=25
The second line, the Count(). Seems to ignore the projection entirely:
SELECT COUNT(*)
FROM [Customers] AS [c]
The result of this is that any customer with a null StatusId will be excluded from the first query but included in the count in the second. Which breaks paging.
I would have thought that project should create something like:
SELECT COUNT(*)
FROM [Customers] AS [c]
INNER JOIN [Status] AS [s] ON [s].id = [c].StatusId
Anyone know why the Count() is ignoring the ProjectTo<>?
Edit
Execution plan:
value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[Domain.Customer]).Select(dtoCustomer
=> new CustomerViewModel() { Code = dtoCustomer.Code, Id = dtoCustomer.Id, Name = dtoCustomer.Name, StatusName =
dtoCustomer.Status.Name, Website = dtoCustomer.Website})
Edit 2021/02/19
Mappings plan:
EF entities -
public class Customer
{
public Guid Id { get; private set; }
public string Name { get; private set; }
public string Code { get; private set; }
public string Website { get; private set; }
public CustomerStatus Status { get; private set; }
public Customer() { }
}
public class CustomerStatus
{
public Guid Id { get; private set; }
public string Name { get; private set; }
}
ViewModel -
public class CustomerViewModel
{
public Guid Id { get; set; }
public string Name { get; set; }
public string Code { get; set; }
public string Website { get; set; }
public string StatusName { get; set; }
}
Mapping -
CreateMap<Customer, CustomerViewModel>();
Edit 2021/02/20 - Manually Excluding Status
As pointed out in #atiyar answer you can manually exclude the status. This crosses me as a work around. My reasoning is this:
If you execute this query, as the very root query:
Db.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider)
You get:
exec sp_executesql N'SELECT TOP(#__p_0) [c].[Id], [c].[Name], [c0].[Name]
AS [StatusName]
FROM [Customers] AS [c]
INNER JOIN [CustomerStatus] AS [c0] ON [c].[StatusId] = [c0].[Id]',N'#__p_0
int',#__p_0=5
This shows automapper understands and can see that there is a needed relationship between Status and Customer. But when you apply the count mechanism:
Db.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider).Count()
Suddenly, the understood relationship between Status and Customer is lost.
SELECT COUNT(*)
FROM [Customers] AS [c]
In my experience with Linq each query step modifies the previous step in a predicable way. I would have expected the count to build on the first command and include the count as part of that.
Interestingly, if you execute this:
_context.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider).Take(int.MaxValue).Count()
Automapper applies the relationship and the result is what I would have expected:
exec sp_executesql N'SELECT COUNT(*)
FROM (
SELECT TOP(#__p_0) [c].[Id], [c].[Name], [c0].[Name] AS [Name0], [c0].[Id]
AS [Id0]
FROM [Customers] AS [c]
INNER JOIN [CustomerStatus] AS [c0] ON [c].[StatusId] = [c0].[Id]
) AS [t]',N'#__p_0 int',#__p_0=2147483647
Edit 2021/02/20 - Latest Version
Seems behaviour is the same in the latest version.
FYI: We have a scenario where records are imported on a regular basis from another application. We were hoping to use the inner join to exclude the records that don't have a matching record in another table. Then those records would be updated at a later point by the import process.
But from the application point of view it should always ignore those records hence the inner join and the status being mandatory. But we will have to manually exclude them (as per atiyar's solution) using the where to prevent paging from returning blown out page count numbers.
Edit 2021/02/20 - Further Digging
This does appear to be a design choice by the EF team and an optimisation. The assumption here is that if the relationship is non-null able. Then the join wont be included as a performance boost. The way around this is as suggested by #atiyar. Thanks for the help everyone #atiyar & #Lucian-Bargaoanu.

Answer 1

I have tested your code in .NET Core 3.1 with Entity Framework Core 3.1 and AutoMapper 10.1.1. And -
your first query generates a LEFT JOIN, not an INNER JOIN like you posted. So, the result from that query will not exclude any customer with a null StatusId. And, the generated SQL is same with ProjectTo<> and manual EF projection. I'd suggest to check your query and generated SQL again to make sure.
your second query generates the same SQL, the SQL you have posted, with ProjectTo<> and manual EF projection.
A solution for you :
If I understand correctly, you are trying to get -
a list of Customer, within the specified range, who has a related Status
the count of all such customers in your database.
Try the following -
Add a nullable foreign-key property in your Customer model -
public Guid? StatusId { get; set; }
This will help to simplify your queries and the SQL they generate.
To get your expected list, modify the first query as -
var viewModels = Db.Customers
.Skip((1 - 1) * 25)
.Take(25)
.Where(p => p.StatusId != null)
.ProjectTo<CustomerViewModel>(_Mapper.ConfigurationProvider)
.ToList();
It will generate the following SQL -
exec sp_executesql N'SELECT [t].[Code], [t].[Id], [t].[Name], [s].[Name] AS [StatusName], [t].[Website]
FROM (
SELECT [c].[Id], [c].[Code], [c].[Name], [c].[StatusId], [c].[Website]
FROM [Customers] AS [c]
ORDER BY (SELECT 1)
OFFSET #__p_0 ROWS FETCH NEXT #__p_1 ROWS ONLY
) AS [t]
LEFT JOIN [Statuses] AS [s] ON [t].[StatusId] = [s].[Id]
WHERE [t].[StatusId] IS NOT NULL',N'#__p_0 int,#__p_1 int',#__p_0=0,#__p_1=25
To get your expected count, modify the second query as -
var count = Db.Customers
.Where(p => p.StatusId != null)
.Count();
It will generate the following SQL -
SELECT COUNT(*)
FROM [Customers] AS [c]
WHERE [c].[StatusId] IS NOT NULL