A very strange case of DbUpdateConcurrencyException - sql-server

Today I've found and an exception in my code.
DbUpdateConcurrencyException
It was very confusing because there were no changes in business logic and there was no updating of entities. After some hours of trying to understand I decided to turn on SQL Server profiler.
And here is an example of issue that I found.
Lets look at Foo entity definition:
public class Foo
{
public Foo()
{
}
public Foo(int someId, DateTime createdAt, int x, int y)
{
SomeId = someId;
CreatedAt = createdAt;
X = x;
Y = y;
}
public int SomeId { get; private set; }
public DateTime CreatedAt { get; private set; }
public int X { get; private set; }
public int Y { get; private set; }
}
If you try to add an instance of Foo and save db context it will work.
using (var myDbContext = new MyDbContext())
{
DateTime now = DateTime.UtcNow;
var foo = new Foo(1, now, 2, 2);
myDbContext.Set<Foo>().Add(foo);
myDbContext.SaveChanges();
}
Below is code that is executed when you save changes.
exec sp_executesql N'INSERT [dbo].[Foo]([SomeId], [CreatedAt], [X], [Y])
VALUES (#0, #1, #2, #3)
',N'#0 int,#1 datetime2(7),#2 int,#3 int',#0=1,#1='2015-10-15 12:45:15.2580302',#2=2,#3=2
Now lets add a computed column.
public int Sum { get; private set; }
Bellow is code for creation column in migration.
Sql("alter table dbo.Foo add Sum as (X + Y)");
Update the database. Now the code throws DbUpdateConcurrencyException. And here is why. If we look in SQL Server profiler, we'll see the code above:
exec sp_executesql N'INSERT [dbo].[Foo]([SomeId], [CreatedAt], [X], [Y])
VALUES (#0, #1, #2, #3)
SELECT [Sum]
FROM [dbo].[Foo]
WHERE ##ROWCOUNT > 0 AND [SomeId] = #0 AND [CreatedAt] = #1',N'#0 int,#1 datetime2(7),#2 int,#3 int',#0=1,#1='2015-10-15 12:55:29.4479727',#2=2,#3=2
Now the query returns [Sum] column because it is computed. Insertion works well. But result set is empty. I think it causes DbUpdateConcurrencyException. The problem is in type of #1 variable. It is datetime2(7), but if you look in type of CreatedAt column you'll see that it is datetime. If you execute the script above, you’ll find a new row with '2015-10-15 12:55:29.447' in CreatedAt column (cast works fine). But query tries to find [Sum] of the row where CreatedAt equals to '2015-10-15 12:55:29.4479727'. Of cause the result set is empty.
So, you can fix the problem in two ways:
Change the precision of column value (for example without milliseconds).
Set manually the type of CreatedAt column in migration (datetime2(7)).
In my case I select first because I don't want to change database schema.
Here is the project to reproduce the problem.
PS: Sorry for my English:)

facing quite a similar case I use the following extension method
public static DateTime RoundedToSeconds(this DateTime dt) {
return new DateTime(dt.Ticks - (dt.Ticks % TimeSpan.TicksPerSecond), dt.Kind);
}
then your code should be
public Foo(int someId, DateTime createdAt, int x, int y)
{
SomeId = someId;
CreatedAt = createdAt.RoundedToSeconds();
X = x;
Y = y;
}
IMHO you may use a rounding to ms.

This issue doesn't repro with current builds of Entity Framework Core. The parameter type for the CreatedAt becomes of type DateTime2.
I think you missed mentioning that the key of the entity contains both SomeId and CreatedAt.

And here is an example of issue that I found.
Lets look at Foo entity definition:
public class Foo
{
public Foo()
{
}
public Foo(int someId, DateTime createdAt, int x, int y)
{
SomeId = someId;
CreatedAt = createdAt;
X = x;
Y = y;
}
public int SomeId { get; private set; }
public DateTime CreatedAt { get; private set; }
public int X { get; private set; }
public int Y { get; private set; }
}
If you try to add an instance of Foo and save db context it will work.
using (var myDbContext = new MyDbContext())
{
DateTime now = DateTime.UtcNow;
var foo = new Foo(1, now, 2, 2);
myDbContext.Set<Foo>().Add(foo);
myDbContext.SaveChanges();
}
Below is code that is executed when you save changes.
exec sp_executesql N'INSERT [dbo].[Foo]([SomeId], [CreatedAt], [X], [Y])
VALUES (#0, #1, #2, #3)
',N'#0 int,#1 datetime2(7),#2 int,#3 int',#0=1,#1='2015-10-15 12:45:15.2580302',#2=2,#3=2
Now lets add a computed column.
public int Sum { get; private set; }
Bellow is code for creation column in migration.
Sql("alter table dbo.Foo add Sum as (X + Y)");
Update the database. Now the code throws DbUpdateConcurrencyException. And here is why. If we look in SQL Server profiler, we'll see the code above:
exec sp_executesql N'INSERT [dbo].[Foo]([SomeId], [CreatedAt], [X], [Y])
VALUES (#0, #1, #2, #3)
SELECT [Sum]
FROM [dbo].[Foo]
WHERE ##ROWCOUNT > 0 AND [SomeId] = #0 AND [CreatedAt] = #1',N'#0 int,#1 datetime2(7),#2 int,#3 int',#0=1,#1='2015-10-15 12:55:29.4479727',#2=2,#3=2
Now the query returns [Sum] column because it is computed. Insertion works well. But result set is empty. I think it causes DbUpdateConcurrencyException. The problem is in type of #1 variable. It is datetime2(7), but if you look in type of CreatedAt column you'll see that it is datetime. If you execute the script above, you’ll find a new row with '2015-10-15 12:55:29.447' in CreatedAt column (cast works fine). But query tries to find [Sum] of the row where CreatedAt equals to '2015-10-15 12:55:29.4479727'. Of cause the result set is empty.
So, you can fix the problem in two ways:
Change the precision of column value (for example without milliseconds).
Set manually the type of CreatedAt column in migration (datetime2(7)).
In my case I select first because I don't want to change database schema.
Here is the project to reproduce the problem.
PS: Sorry for my English:)

Related

Automapper ProjectTo<> not working with Count()

I have an odd issue with AutoMapper (I'm using .NET core 3.1 and AutoMapper 10.1.1)
I'm doing a simple project to list and a simple projected count for total records:
var data = Db.Customers
.Skip((1 - 1) * 25)
.Take(25)
.ProjectTo<CustomerViewModel>(Mapper.ConfigurationProvider)
.ToList();
var count = Db.Customers
.ProjectTo<CustomerViewModel>(Mapper.ConfigurationProvider)
.Count();
The first line creates the expected SQL:
exec sp_executesql N'SELECT [c].[Code], [c].[Id], [c].[Name], [c].[Website], [s].Name
FROM [Customers] AS [c]
INNER JOIN [Status] AS [s] ON [s].id = [c].StatusId
ORDER BY (SELECT 1)
OFFSET #__p_0 ROWS FETCH NEXT #__p_1 ROWS ONLY',N'#__p_0 int,#__p_1 int',#__p_0=0,#__p_1=25
The second line, the Count(). Seems to ignore the projection entirely:
SELECT COUNT(*)
FROM [Customers] AS [c]
The result of this is that any customer with a null StatusId will be excluded from the first query but included in the count in the second. Which breaks paging.
I would have thought that project should create something like:
SELECT COUNT(*)
FROM [Customers] AS [c]
INNER JOIN [Status] AS [s] ON [s].id = [c].StatusId
Anyone know why the Count() is ignoring the ProjectTo<>?
Edit
Execution plan:
value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[Domain.Customer]).Select(dtoCustomer
=> new CustomerViewModel() { Code = dtoCustomer.Code, Id = dtoCustomer.Id, Name = dtoCustomer.Name, StatusName =
dtoCustomer.Status.Name, Website = dtoCustomer.Website})
Edit 2021/02/19
Mappings plan:
EF entities -
public class Customer
{
public Guid Id { get; private set; }
public string Name { get; private set; }
public string Code { get; private set; }
public string Website { get; private set; }
public CustomerStatus Status { get; private set; }
public Customer() { }
}
public class CustomerStatus
{
public Guid Id { get; private set; }
public string Name { get; private set; }
}
ViewModel -
public class CustomerViewModel
{
public Guid Id { get; set; }
public string Name { get; set; }
public string Code { get; set; }
public string Website { get; set; }
public string StatusName { get; set; }
}
Mapping -
CreateMap<Customer, CustomerViewModel>();
Edit 2021/02/20 - Manually Excluding Status
As pointed out in #atiyar answer you can manually exclude the status. This crosses me as a work around. My reasoning is this:
If you execute this query, as the very root query:
Db.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider)
You get:
exec sp_executesql N'SELECT TOP(#__p_0) [c].[Id], [c].[Name], [c0].[Name]
AS [StatusName]
FROM [Customers] AS [c]
INNER JOIN [CustomerStatus] AS [c0] ON [c].[StatusId] = [c0].[Id]',N'#__p_0
int',#__p_0=5
This shows automapper understands and can see that there is a needed relationship between Status and Customer. But when you apply the count mechanism:
Db.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider).Count()
Suddenly, the understood relationship between Status and Customer is lost.
SELECT COUNT(*)
FROM [Customers] AS [c]
In my experience with Linq each query step modifies the previous step in a predicable way. I would have expected the count to build on the first command and include the count as part of that.
Interestingly, if you execute this:
_context.Customers.ProjectTo<CustomerViewModel>(_mapper.ConfigurationProvider).Take(int.MaxValue).Count()
Automapper applies the relationship and the result is what I would have expected:
exec sp_executesql N'SELECT COUNT(*)
FROM (
SELECT TOP(#__p_0) [c].[Id], [c].[Name], [c0].[Name] AS [Name0], [c0].[Id]
AS [Id0]
FROM [Customers] AS [c]
INNER JOIN [CustomerStatus] AS [c0] ON [c].[StatusId] = [c0].[Id]
) AS [t]',N'#__p_0 int',#__p_0=2147483647
Edit 2021/02/20 - Latest Version
Seems behaviour is the same in the latest version.
FYI: We have a scenario where records are imported on a regular basis from another application. We were hoping to use the inner join to exclude the records that don't have a matching record in another table. Then those records would be updated at a later point by the import process.
But from the application point of view it should always ignore those records hence the inner join and the status being mandatory. But we will have to manually exclude them (as per atiyar's solution) using the where to prevent paging from returning blown out page count numbers.
Edit 2021/02/20 - Further Digging
This does appear to be a design choice by the EF team and an optimisation. The assumption here is that if the relationship is non-null able. Then the join wont be included as a performance boost. The way around this is as suggested by #atiyar. Thanks for the help everyone #atiyar & #Lucian-Bargaoanu.
I have tested your code in .NET Core 3.1 with Entity Framework Core 3.1 and AutoMapper 10.1.1. And -
your first query generates a LEFT JOIN, not an INNER JOIN like you posted. So, the result from that query will not exclude any customer with a null StatusId. And, the generated SQL is same with ProjectTo<> and manual EF projection. I'd suggest to check your query and generated SQL again to make sure.
your second query generates the same SQL, the SQL you have posted, with ProjectTo<> and manual EF projection.
A solution for you :
If I understand correctly, you are trying to get -
a list of Customer, within the specified range, who has a related Status
the count of all such customers in your database.
Try the following -
Add a nullable foreign-key property in your Customer model -
public Guid? StatusId { get; set; }
This will help to simplify your queries and the SQL they generate.
To get your expected list, modify the first query as -
var viewModels = Db.Customers
.Skip((1 - 1) * 25)
.Take(25)
.Where(p => p.StatusId != null)
.ProjectTo<CustomerViewModel>(_Mapper.ConfigurationProvider)
.ToList();
It will generate the following SQL -
exec sp_executesql N'SELECT [t].[Code], [t].[Id], [t].[Name], [s].[Name] AS [StatusName], [t].[Website]
FROM (
SELECT [c].[Id], [c].[Code], [c].[Name], [c].[StatusId], [c].[Website]
FROM [Customers] AS [c]
ORDER BY (SELECT 1)
OFFSET #__p_0 ROWS FETCH NEXT #__p_1 ROWS ONLY
) AS [t]
LEFT JOIN [Statuses] AS [s] ON [t].[StatusId] = [s].[Id]
WHERE [t].[StatusId] IS NOT NULL',N'#__p_0 int,#__p_1 int',#__p_0=0,#__p_1=25
To get your expected count, modify the second query as -
var count = Db.Customers
.Where(p => p.StatusId != null)
.Count();
It will generate the following SQL -
SELECT COUNT(*)
FROM [Customers] AS [c]
WHERE [c].[StatusId] IS NOT NULL

not mapped value doesn't return correct

I'm coding an ASP .NET Core Web API Project and have an issue about a not mapped prop in my model
Here's my model
public class Category
{
[Key]
public SqlHierarchyId NodeId { get; set; }
[Required, MaxLength(200)]
public string Name { get; set; }
[Required]
public int CategoryId { get; set; }
[NotMapped]
public int ChildCount { get; set; }
}
and the procedure
ALTER PROC [dbo].[GetCategoryChild](#categoryId int)
AS
BEGIN
DECLARE #CurrentNode hierarchyid
SELECT #CurrentNode = NodeId FROM Categories
WHERE CategoryId = #categoryId
SELECT *
,(select count(*) as cc from Categories where NodeId.GetAncestor(1) = ht.NodeId) as ChildCount
FROM Categories as ht
WHERE NodeId.GetAncestor(1) = #CurrentNode
END ;
when I execute my procedure in SQL server returns expected result
NodeId Name CategoryId ChildCount
------------------------------------------------
0x58 Books 19 2
0x8C Computers 25 1
but when I call it in Controller,all ChildCounts are 0
NodeId Name CategoryId ChildCount
------------------------------------------------
0x58 Books 19 0
0x8C Computers 25 0
and here's my controller
[HttpGet, Route("getChild/{categoryId}")]
public IEnumerable<Category> GetChild(int categoryId)
{
var childs = _db.Categories.
FromSql($"GetCategoryChild {categoryId}").ToList();
return childs;
}
Build a sql command and add the categoryId variable as a sqlcommand parameter. That’s how you should be calling sprocs from application code.
You can remove ChildCount property from Category model, and create a view model CategoryViewModel and map result from stored procedure to this view model:
var param = new SqlParameter("#categoryId", categoryId);
var result = dbContext.Database
.SqlQuery<CategoryViewModel>("dbo.GetCategoryChild #categoryId",param).ToList();
Please see this Link:
Execute stored procedure in entity Framework Core without expecting map to dbset

How can I implement this T-SQL in my stored procedure?

I'm new to SQL Server and want to implement this scenario. My stored procedure gets 8 input parameters from a C# web application, and checks all input has into the table. For that purpose I wrote this simple stored procedure:
CREATE PROCEDURE CheckValid
#p_bank varchar,
#p_pay_date varchar,
#p_bill_id varchar,
#p_payment_id varchar,
#p_ref_code varchar,
#p_branch varchar,
#p_channel_type varchar,
#p_send_date varchar
AS
BEGIN
SELECT
[p_bank], [p_pay_date], [p_bill_id], [p_payment_id],
[p_ref_code], [p_branch], [p_channel_type], [p_send_date]
FROM
[SAMPLE].[dbo].[MixedTable]
WHERE
[p_bank] = #p_bank
AND [p_pay_date] = #p_pay_date
AND [p_bill_id] = #p_bill_id
AND [p_payment_id] = #p_payment_id
AND [p_ref_code] = #p_ref_code
AND [p_branch] = #p_branch
AND [p_channel_type] = #p_channel_type
AND [p_send_date] = #p_send_date
END
But want to return to c# application this scenario, for example c# sends all field but when stored procedure select run for this purpose can not find data, for example p_bill_id not correct into the table for more explain in select query into where clause in the [p_bill_id]=#p_bill_id not trust and now want to return sp this :
p_bill_id,not found
and other example c# all variable correct but two field [p_channel_type] and [p_payment_id] not correct into where clause but other 6 field correct now SP return this:
[p_channel_type],not found
[p_payment_id],not found
Summary of question:
When data for passed parameter value is not found, I want it to return that corresponding column.
For example:
[p_channel_type],not found
[p_payment_id],not found
Note, varchar means varchar(1) so you should specify length for each argument explicitly like varchar(100)
CREATE PROCEDURE CheckValid
#p_bank varchar(<length>),
#p_pay_date varchar(<length>),
#p_bill_id varchar(<length>),
#p_payment_id varchar(<length>),
#p_ref_code varchar(<length>),
#p_branch varchar(<length>),
#p_channel_type varchar(<length>),
#p_send_date varchar(<length>)
AS
BEGIN
if not exists(select 1 from dbo.BankTable where p_bank = #p_bank)
begin
raiserror('Bank %s not found', 16, 1, #p_bank)
return
end
if not exists(select 1 from dbo.BillTable where p_bill_id = #p_bill_id)
begin
raiserror('Bill %s not found', 16, 1, #p_bill_id)
return
end
...
SELECT [p_bank],[p_pay_date],[p_bill_id],[p_payment_id],[p_ref_code],[p_branch],[p_channel_type],[p_send_date]
FROM [SAMPLE].[dbo].[MixedTable]
where [p_bank]=#p_bank and [p_pay_date]=#p_pay_date
and [p_bill_id]=#p_bill_id and [p_payment_id]=#p_payment_id
and [p_ref_code]=#p_ref_code and [p_branch]=#p_branch
and [p_channel_type]=#p_channel_type and [p_send_date]=#p_send_date
END
GO
Instead of creating stored procedure for this move "validation" logic to your c# application.
Database is just IO device and I think keeping "business logic" in IO device not a good approach.
// Class which represent your eight parameters
public class Data
{
public string Bank { get; set; }
public string PayDate { get; set; }
public string BillId { get; set; }
public string PaymentId { get; set; }
public string RefCode { get; set; }
public string Branch { get; set; }
public string ChannelType { get; set; }
public string SendDate { get; set; }
}
public class Validation
{
private Data _data;
public Validation(Data data)
{
_data = data;
}
public IEnumerable<string> Validate()
{
var columns = new KeyValuePair<string, string>[]
{
new KeyValuePair("p_bank", _data.Bank),
new KeyValuePair("p_pay_date", _data.PayDate),
new KeyValuePair("p_bill_id", _data.BillId),
new KeyValuePair("p_payment_id", _data.PaymentId),
new KeyValuePair("p_ref_code], _data.RefCode),
new KeyValuePair("p_branch", _data.Branch),
new KeyValuePair("p_channel_type", _data.ChannelType),
new KeyValuePair("p_send_date", _data.SendDate)
};
return columns.Where(pair => IsValueExists(pair.Key, pair.Value) == false);
}
private bool IsValueExists(string columnName, string value)
{
var query =
$"SELECT [{columnName}]
FROM [SAMPLE].[dbo].[MixedTable]
WHERE [{columnName}] = #value";
var parameter = new SqlParameter
{
ParameterName = "#value",
SqlDbType = SqlDbType.VarChar,
Value = _data.Bank
};
using (var connection = new SqlConnection(yourConnectionString))
using (var command = new SqlCommand(query, connection))
{
command.Parameters.Add(parameter);
connection.Open();
var value = command.ExecuteScalar();
return value != null; // null returned if no rows exists
}
}
}
Then you can use this method somewhere
var data = new Data { Bank = "BankName", RefCode = "SomeRefcode" } // put all values
var validation = new Validation(data);
var invalidValues = validation.Validate();
foreach(var invalidValue in invalidValues)
{
// Print or save column names where value is invalid
}

Does Dapper support c# 6 read-only properties in POCOs?

Given the following:
public class SomePoco {
public int IntValue { get; }
}
and
CREATE TABLE SomePocoStorage (IntValue INT NOT NULL)
and
INSERT SomePocoStorage VALUES (1), (274)
If I call
connection.Query<SomePoco>("SELECT * FROM SomePocoStorage")
does Dapper handle populating the IntValue field on the returned SomePoco instances?
Good question! It isn't a scenario I've targeted, but I'd be more than happy to take a look at what would be involved. Since we already do a lot of nasty reflection, this could still be viable. Probably better as a github issue, but I'll have a look.
Update - it does now (at the current time, via repo only - not deployed):
[Fact] // passes
public void GetOnlyProperties()
{
var obj = connection.QuerySingle<HazGetOnly>(
"select 42 as [Id], 'def' as [Name];");
obj.Id.IsEqualTo(42);
obj.Name.IsEqualTo("def");
}
class HazGetOnly
{
public int Id { get; }
public string Name { get; } = "abc";
}
No because there's no way for Dapper to set the value of the property if that property only has a getter.

Is there an OrmLite option for DateTime.SpecifyKind(DateTimeKind.Utc)?

Is there a way to specify that I want all of the DateTimes that OrmLite materializes to be set to UTC kind?
I store a lot of DateTimes in my database via stored procedures when a row is inserted:
insert [Comment] (
Body
, CreatedOn
) values (
#Body
, getutcdate()
);
When I retrieve the values via a select statement in ormlite, the datetimes come out in Unspecified kind (which is interpreted as the local timezone, I believe):
var comments = db.SqlList<Comment>("select * from [Comment] where ... ");
I would prefer not to set each DateTime object individually:
foreach (var comment in comments) {
comment.CreatedOn = DateTime.SpecifyKind(comment.CreatedOn, DateTimeKind.Utc);
}
I found this question, but I don't think it's quite what I'm asking for:
servicestack ormlite sqlite DateTime getting TimeZone adjustment on insert
Also found this pull request, but setting SqlServerOrmLiteDialectProvider.EnsureUtc(true) doesn't seem to do it either.
SqlServerOrmLiteDialectProvider.EnsureUtc(true) does work, there was something else going on with my test case that led me to believe that it didn't. Hopefully this will help someone else.
Here's some sample code:
model.cs
public class DateTimeTest {
[AutoIncrement]
public int Id { get; set; }
public DateTime CreatedOn { get; set; }
}
test.cs
var connectionString = "server=dblcl;database=flak;trusted_connection=true;";
var provider = new SqlServerOrmLiteDialectProvider();
provider.EnsureUtc(true);
var factory = new OrmLiteConnectionFactory(connectionString, provider);
var connection = factory.Open();
connection.CreateTable(true, typeof(DateTimeTest));
connection.ExecuteSql("insert DateTimeTest (CreatedOn) values (getutcdate())");
var results = connection.SqlList<DateTimeTest>("select * from DateTimeTest");
foreach(var result in results) {
Console.WriteLine("{0},{1},{2},{3},{4}", result.Id, result.CreatedOn, result.CreatedOn.Kind, result.CreatedOn.ToLocalTime(), result.CreatedOn.ToUniversalTime());
}
output
1,9/13/2013 5:19:12 PM,Utc,9/13/2013 10:19:12 AM,9/13/2013 5:19:12 PM

Resources