I want to retrieve data from a database from a specific column sorted by the column, plus distinct. The data is also fetched distinctly, but not sorted. My query looks like this:
return context.TableName.OrderBy(l => l.ColumnName).Select(p => p.ColumnName)
.Distinct().ToList();
Distinct itself discards ordering. It is because of realisation and it depends on Database. So do ordering after Distinct.
return context.TableName
.Select(p => p.ColumnName)
.Distinct()
.OrderBy(l => l)
.ToList();
Related
The goal is to have a single query that will be generated by the EF and MSSQL will execute it in one go. Having the current implementation, everything works correctly, but not optimal. To be more specific, looking at the SQL Server Profiler logs, it makes additional exec sp_executesql queries per each company to fetch data (in example below, it would be Products).
Say, we have selected product ids.
List<int> selectedProductIds = { 1, 2, 3 };
We filter over a collection of Companies to get only those companies that have ALL selected products.
And a query where we dynamically extend it as many as we need, thankfully to IQueryable interface.
Imagine x of type Company and it contains a collection of Products.
if (selectedProductIds.Count > 0)
{
query = query.Where(x => selectedProductIds.All(id => x.Products.Select(p => p.ProductId).Contains(id)));
}
Is there any way to rewrite the predicate using LINQ? I know I can make a dynamic SQL query myself anytime, but I am trying to understand what I miss in terms of EF/LINQ. Thanks!
The version of Entity Framework Core is 2.1.
UPD:
Company products are unique and never duplicated within a company entity. No need to make distinct.
Try the following query:
if (selectedProductIds.Count > 0)
{
query = query.Where(x => x.Products
.Where(p => selectedProductIds.Contains(p.ProductId))
.Count() == selectedProductIds.Count
);
}
I am trying to construct a query using EF Core 3.1 across multiple tables, and it's simply not working.
I'll try to explain with some dummy examples.
Assume my SQL tables all have the following defined within the SQL Server DB:
Primary Keys (composite if needed)
Foreign Keys
The dummy Entities are as follows:
Town
Inhabitant
InhabitantCar
Car
Manufacturer
Mechanic
Foreign Key relationships:
Inhabitant -> Town
InhabitantCar -> Inhabitant and -> Car
Car -> Manufacturer and -> Mechanic
The Entities all have their Navigation Properties set, and I've set up the Primary and Foreign keys in the DB Context.
The vast majority of these tables have a bit field "Enabled", so rows can be disabled without deleting them
So the query I'm trying to write is similar to the following:
var data = await context.Town.AsNoTracking()
.Where(t => t.TownName == request.TownName)
.Include(t => t.Inhabitants.Where(i => i.Name == request.InhabitantName && i.Enabled)
.ThenInclude(i => i.InhabitantCar.Where(ic => ic.Enabled))
.ThenInclude(ic => ic.Cars.Where(c => c.Enabled))
.ThenInclude(c => c.Manufacturer.Where(m => m.Enabled))
.Include(t => t.Inhabitants.Where(i => i.Name == request.InhabitantName && i.Enabled)
.ThenInclude(i => i.InhabitantCar.Where(ic => ic.Enabled))
.ThenInclude(ic => ic.Cars.Where(c => c.Enabled))
.ThenInclude(c => c.Mechanic.Where(m => m.Enabled && m.Name == request.AllowedMechanic))
.ToListAsync().ConfigureAwait(false);
So to summarise, I want to know what cars are being driven by a "John Smith" who lives in "London" that are serviced by "MechanicsAreUs".
This seems quite long-winded to me, and that may be where my problem lies.
Anyhow, quite a few of the .WHERE clauses on the latter ThenIncludes just won't compile. Removing them one-by-one until it compiles gives me:
var data = await context.Town.AsNoTracking()
.Where(t => t.TownName == request.TownName)
.Include(t => t.Inhabitants.Where(i => i.Name == request.InhabitantName && i.Enabled)
.ThenInclude(i => i.InhabitantCar.Where(ic => ic.Enabled))
.ThenInclude(ic => ic.Cars)
.ThenInclude(c => c.Manufacturer)
.Include(t => t.Inhabitants.Where(i => i.Name == request.InhabitantName && i.Enabled)
.ThenInclude(i => i.InhabitantCar.Where(ic => ic.Enabled))
.ThenInclude(ic => ic.Cars)
.ThenInclude(c => c.Mechanic)
.ToListAsync().ConfigureAwait(false);
So, as written, it's going to bring back disabled entries, and I'm not specifying the mechanic. However, when I run it, I get the exception:
System.InvalidOperationException: Lambda expression used inside
Include is not valid.
I've spent ages going through the various Microsoft Examples, but I'm not finding any examples that seems to be of this level of complexity. It's only a handful of inner-joins. Something that could be accomplished in a stored procedure within minutes. Just that I want to do this using Entity Framework.
You cannot filter .Include(...) eager loads - it's all or nothing. As David Browne stated in a comment to your question, you should use query filters if you want to exclude records based on their Enabled flags, e.g.:
modelBuilder.Entity<Car>()
.HasQueryFilter(c => c.Enabled);
I seems you are interested in Car entities so let's restructure the query make that the focus:
var query = context.Cars;
You want cars associated with an Inhabitant with a specific name that is associated with a specific Town but is also serviced by a specific Mechanic, so let's filter by that criteria:
query = query.Where( c =>
c.InhabitantCar.Inhabitant.Name == request.InhabitantName
&& c.InhabitantCar.Inhabitant.Town.TownName == request.TownName
&& c.Mechanic == request.AllowedMechanic );
This query will now return the Car entities you desire, so let's now configure the eager loads:
query = query.Include( c => c.Manufacturer )
.Include( c => c.Mechanic )
.Include( c => c.InhabitantCar )
.ThenInclude( ic => ic.Inhabitant )
.ThenInclude( i => i.Town );
Give that a shot.
One recommendation was to use Query Filters.
The idea behind this was great - in my DB Context file I could add a common set of filters, e.g.
builder.Entity<Town>()
.HasQueryFilter(a => a.Enabled);
builder.Entity<Car>()
.HasQueryFilter(a => a.Enabled);
builder.Entity<Manufacturer>()
.HasQueryFilter(a => a.Enabled);
And that would be included in every query generated by my Service file - the Developers need not care.
However, when I analysed the resultant SQL, I found that my code was littered with multiple sub-queries, e.g.
Inner Join (Select...Where ...Enabled = 1)
On removing these centralized Query Filters and adding this to my WHERE clause in the LINQ statement resulted in a far more efficient query.
Entities reflect the data state. You cannot filter what related data you want, it's all or nothing. A town doesn't just have "enabled" inhabitants, it has inhabitants, just some of them are enabled and others aren't. That you don't want to view disabled ones or irrelevant ones, that is a view's concern, not the data model.
You can use Select to populate a model structure that is suited to your view. This can flatten out the joining tables, and load just the enabled records you want to see, plus streamline the fields your view needs rather than exposing everything about your domain. You can leverage AutoMapper to fill in the view models /w ProjectTo.
I read the cookbook, but I can not figure out how to combine in a single query a matching() and a orWhere().
Example: I have Photo that belongs from Album. Both have the active field. So I'm trying to write a findInactive() method. A "inactive" photo has the active field as false or matching an album that has the active fields as false.
Something like this:
public function findInactive(Query $query, array $options)
{
$query->matching('Albums', function ($q) {
return $q->where(['Albums.active' => false]);
})
->orWhere(['Photos.active' => false])
->enableAutoFields(true);
return $query;
}
But that does not work:
'SELECT [...] FROM photos Photos INNER JOIN photos_albums Albums ON (Albums.active = :c0 AND Albums.id = (Photos.album_id)) WHERE Photos.active = :c1'
How to do? Thanks.
EDIT
Maybe a possible solution is usecontain():
$query->contain(['Albums => ['fields' => ['active']]])
->where(['Photos.active' => false])
->orWhere(['Albums.active' => false]);
But is it not possible to use matching() or innerJoinWith()?
Add the conditions to the main query
matching() or innerJoinWith() with conditions is the wrong way to go, as the conditions are being addded to the INNER joins ON clause, which will cause the Photo row to be excluded in case the Albums.active condition doesn't match.
If you want to only receive photos that belong to an album, then you want to use matching() or innerJoinWith(), but you'll have to add the conditions to the main query instead, ie:
$query
->innerJoinWith('Albums')
->where(['Albums.active' => false])
->orWhere(['Photos.active' => false])
// ...
In case a photo doesn't have to belong to an album, or it's not important whether it does, you could use either leftJoin(), leftJoinWith(), or even contain().
The latter however may use the INNER joinStrategy and/or the select strategy (which uses a separate query), so you'd need to take care of ensuring that it uses LEFT and join instead. Using containments however is usually only advised if you actually want to contain something, and given that your finder seems to be supposed to just filter things, I'd say go with leftJoinWith() instead:
$query
->leftJoinWith('Albums')
->where(['Albums.active' => false])
->orWhere(['Photos.active' => false])
// ...
See also
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Filtering by Associated Data Via Matching And Joins
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Retrieving Associated Data
Cookbook > Database Access & ORM > Associations - Linking Tables Together > BelongsTo Associations
I want to optimize this code to minimum.
Dictionary<long, string> developmentRegions = objectset
.Where(f => f.IsDeleted == false && f.IsApproved == true)
.OrderBy(o => o.Name)
.ToDictionary(a => a.Id, a => a.Name);
Can anyone optimize this code
Your problem probaly is the order of query resul.
Try on SQL Server create a index on Name field, for take a fester result on order.
CREATE INDEX ixName ON TableWhereNameIsField (NameOfOrderField);
Assuming You really need all those names loaded from database, there is little You can do. You can remove the OrderBy, as Dictionary does not guarantee order anyway.
Lets say I have a model, Article that has a large amount of columns and the database contains more than 100,000 rows. If I do something like var articles = db.Articles.ToList() it is retrieving the entire article model for each article in the database and holding it in memory right?
So if I am populating a table that only shows the date of the entry and it's title is there a way to only retrieve just these columns from the database using the entity framework, and would it be more efficient?
According to this,
There is a cost required to track returned objects in the object
context. Detecting changes to objects and ensuring that multiple
requests for the same logical entity return the same object instance
requires that objects be attached to an ObjectContext instance. If you
do not plan to make updates or deletes to objects and do not require
identity management , consider using the NoTracking merge options when
you execute queries.
it looks like I should use NoTracking since the data isn't being changed or deleted, only displayed. So my query now becomes var articles = db.Articles.AsNoTracking().ToList(). Are there other things I should do to make this more efficient?
Another question I have is that according to this answer, using .Contains(...) will cause a large performance drop when dealing with a large database. What is the recommended method to use to search through the entries in a large database?
It's called a projection and just translates into a SELECT column1, column2, ... in SQL:
var result = db.Articles
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
Instead of a => new { ... } (creates a list of "anonymous" objects) you can also use a named helper class (or "view model"): a => new MyViewModel { ... } that contains only the selected properties (but you can't use a => new Article { ... } as an entity itself).
For such a projection you don't need AsNoTracking() because projected data are not tracked anyway, only full entity objects are tracked.
Instead of using Contains the more common way is to use Where like:
var date = DateTime.Now.AddYears(-1);
var result = db.Articles
.Where(a => date <= a.Date)
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
This would select only the articles that are not older than a year. The Where is just translated into a SQL WHERE statement and the filter is performed in the database (which is as fast as the SQL query is, depending on table size and proper indexing, etc.). Only the result of this filter is loaded into memory.
Edit
Refering to your comment below:
Don't confuse IEnumerable<T>.Contains(T t) with string.Contains(string subString). The answer you have linked in your question talks about the first version of Contains. If you want to search for articles that have the string "keyword" in the text body you need the second Contains version:
string keyword = "Entity Framework";
var result = db.Articles
.Where(a => a.Body.Contains(keyword))
.Select(a => new
{
Date = a.Date,
Title = a.Title
})
.ToList();
This will translate into something like WHERE Body like N'%Entity Framework%' in SQL. The answer about the poor performance of Contains doesn't apply to this version of Contains at all.