DB Intercept causing SQL Nested too deeply - sql-server

Question

I'm using a DB intercept in entity framework 6 (code first model) in order to implement a soft delete function, however intermittently I'm getting a exception thrown from SQL Server stating "Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries."
This is an example of the sql being generated, as you can see the SQL generated is pretty outrageous:
https://gist.github.com/junderhill/87caceac728809a8ca837b9d8b5189f3
The code for my EF Intercept is as follows:
public class SoftDeleteInterceptor : IDbCommandTreeInterceptor
{
public const string IsDeletedColumnName = "IsDeleted";
public void TreeCreated(DbCommandTreeInterceptionContext interceptionContext)
{
if (interceptionContext.OriginalResult.DataSpace != DataSpace.SSpace)
{
return;
}
var queryCommand = interceptionContext.Result as DbQueryCommandTree;
if (queryCommand != null)
{
interceptionContext.Result = HandleQueryCommand(queryCommand);
}
}
private static DbCommandTree HandleQueryCommand(DbQueryCommandTree queryCommand)
{
var newQuery = queryCommand.Query.Accept(new SoftDeleteQueryVisitor());
return new DbQueryCommandTree(
queryCommand.MetadataWorkspace,
queryCommand.DataSpace,
newQuery);
}
public class SoftDeleteQueryVisitor : DefaultExpressionVisitor
{
public override DbExpression Visit(DbScanExpression expression)
{
var table = (EntityType)expression.Target.ElementType;
if (table.Properties.All(p => p.Name != IsDeletedColumnName))
{
return base.Visit(expression);
}
var binding = expression.Bind();
return binding.Filter(
binding.VariableType
.Variable(binding.VariableName)
.Property(IsDeletedColumnName)
.NotEqual(DbExpression.FromBoolean(true)));
}
}

Answer 1

Update:
This did work correctly. However, something was causing significant slow-downs on our servers over time. Every time we would reset the IIS app pool, it would be fast, but it appears EF is somehow caching or leaking memory and slowing it down. We pulled the interceptors out temporarily.
We hit this issue at work, and after a serious amount of time, here's a solution (has yet to be extensively tested; I'll update this with any updates if needed):
To start at the root, in SoftDeleteQueryVisitor, if you need to add the filter, you return return binding.Filter(...), which modifies the query. This in turn triggers TreeCreated. This walks the tree, finds the DbScanExpression again, and re-adds the filter. The obvious solution is to not add the filter if it already has it, then.
Looking at the structure of the tree, we find that the filter is a parent node of the scan:
+ queryCommand {DbQueryCommandTree
|_Parameters
| |_p__linq__0 : Edm.Guid
|_Query : Collection{Record['IDGUID'=Edm.Guid, 'Name'=Edm.String, 'FooField'=Edm.String, 'BarField'=Edm.Boolean, 'Group'=Edm.String, 'IsDeleted'=Edm.Boolean]}
|_Project
|_Input : 'Limit1'
| |_Limit
| |_Filter
| | |_Input : 'Extent1'
| | | |_Filter
| | | |_Input : 'Var_8'
| | | | |_Scan : CodeFirstDatabase.LeaderboardSuite
| | | |_Predicate
| | | |_
| | | |_Var(Var_8).IsDeleted
| | | |_<>
| | | |_True
| | |_Predicate
| | |_
| | |_Var(Extent1).IDGUID
| | |_=
| | |_#p__linq__0
| |_2
|_Projection
|_NewInstance : Record['IDGUID'=Edm.Guid, 'Name'=Edm.String, 'FooField'=Edm.String, 'BarField'=Edm.Boolean, 'Group'=Edm.String, 'IsDeleted'=Edm.Boolean]
|_Column : 'IDGUID'
| |_Var(Limit1).IDGUID
|_Column : 'Name'
| |_Var(Limit1).Name
|_Column : 'FooField'
| |_Var(Limit1).FooField
|_Column : 'BarField'
| |_Var(Limit1).BarField
|_Column : 'Group'
| |_Var(Limit1).Group
|_Column : 'IsDeleted'
|_Var(Limit1).IsDeleted} System.Data.Entity.Core.Common.CommandTrees.DbQueryCommandTree
However, once we realize we own both the visitor and the visitor's lifetime, we realize we can store state:
var newQuery = queryCommand.Query.Accept(new SoftDeleteQueryVisitor());
We also know that the Filter node will be hit before the Scan node because it is a parent of the Scan node. At this point, we have to consider two possibilities: There is no where clause on the query, and there is a where clause on the query.
At this point, I just walked the tree, determined if the query predicate used the IsDeleted field in an Equals or NotEquals clause, and didn't add the node there if so. I didn't bother checking the Or or Xor clauses because these are incredibly unlikely cases. They are easy to add, though, if you find you need them.
public class SoftDeleteQueryVisitor : DefaultExpressionVisitor
{
private readonly List<DbScanExpression> _scanExpressions = new List<DbScanExpression>();
public override DbExpression Visit(DbScanExpression expression)
{
var table = (EntityType)expression.Target.ElementType;
if (table.Properties.All(p => p.Name != InterceptorConstants.IsDeletedColumnName))
{
return base.Visit(expression);
}
if (_scanExpressions.Contains(expression))
{
return base.Visit(expression);
}
var binding = expression.Bind();
return binding.Filter(
binding.VariableType
.Variable(binding.VariableName)
.Property(InterceptorConstants.IsDeletedColumnName)
.NotEqual(DbExpression.FromBoolean(true)));
}
private bool HasField(DbScanExpression expression)
{
var table = (EntityType)expression.Target.ElementType;
return table.Properties.Any(p => p.Name == InterceptorConstants.IsDeletedColumnName);
}
private bool HasExpression(DbExpression predicate)
{
if (predicate is DbAndExpression andExpression)
{
return HasExpression(andExpression.Left) || HasExpression(andExpression.Right);
}
if (predicate is DbComparisonExpression comparisonExpression &&
(comparisonExpression.ExpressionKind == DbExpressionKind.NotEquals ||
comparisonExpression.ExpressionKind == DbExpressionKind.Equals))
{
if (comparisonExpression.Right is DbPropertyExpression rightPropertyExpression &&
rightPropertyExpression.Property.Name == InterceptorConstants.IsDeletedColumnName ||
comparisonExpression.Left is DbPropertyExpression leftPropertyExpression &&
leftPropertyExpression.Property.Name == InterceptorConstants.IsDeletedColumnName)
{
return true;
}
}
return false;
}
public override DbExpression Visit(DbFilterExpression expression)
{
if (expression.Input.Expression is DbScanExpression scanExpression)
{
if (HasField(scanExpression))
{
if (HasExpression(expression.Predicate))
{
_scanExpressions.Add(scanExpression);
}
}
}
return base.Visit(expression);
}
}