I have the following entities (I'll show the properties I'm working with because I don't want to make it larger than needed):
PROPERTY: Where a property can be child of another one and has a 1-1 relationship with GeoLocation and can have multiple Multimedia and Operation
public partial class Property
{
public Property()
{
InverseParent = new HashSet<Property>();
Multimedia = new HashSet<Multimedia>();
Operation = new HashSet<Operation>();
}
public long Id { get; set; }
public string GeneratedTitle { get; set; }
public string Url { get; set; }
public DateTime? DatePublished { get; set; }
public byte StatusCode { get; set; }
public byte Domain { get; set; }
public long? ParentId { get; set; }
public virtual Property Parent { get; set; }
public virtual GeoLocation GeoLocation { get; set; }
public virtual ICollection<Property> InverseParent { get; set; }
public virtual ICollection<Multimedia> Multimedia { get; set; }
public virtual ICollection<Operation> Operation { get; set; }
}
GEOLOCATION: As mentioned, it has a 1-1 relationship with Property
public partial class GeoLocation
{
public int Id { get; set; }
public double? Latitude { get; set; }
public double? Longitude { get; set; }
public long? PropertyId { get; set; }
public virtual Property Property { get; set; }
}
MULTIMEDIA: it can hold multiple Images, with different sizes, for a single Property. The detail here is that Order specifies the order of the images to be shown in the client application, but it doesn't start always with 1. There're some cases where a Property has Multimedia files that starts with 3 or x.
public partial class Multimedia
{
public long Id { get; set; }
public long? Order { get; set; }
public string Resize360x266 { get; set; }
public long? PropertyId { get; set; }
public virtual Property Property { get; set; }
}
OPERATIONS: defines all the operations a Property can have, using OperationType to name this operation. (rent, sell, etc.)
public partial class Operation
{
public Operation()
{
Price = new HashSet<Price>();
}
public long Id { get; set; }
public long? OperationTypeId { get; set; }
public long? PropertyId { get; set; }
public virtual OperationType OperationType { get; set; }
public virtual Property Property { get; set; }
public virtual ICollection<Price> Price { get; set; }
}
public partial class OperationType
{
public OperationType()
{
Operation = new HashSet<Operation>();
}
public long Id { get; set; }
public string Name { get; set; }
public virtual ICollection<Operation> Operation { get; set; }
}
PRICE: defines the price for each operation and the currency type. (i.e.: A property can have the rent option - Operation - for X amount in USD currency, but another price registered for the same Operation in case of use another currency type )
public partial class Price
{
public long Id { get; set; }
public float? Amount { get; set; }
public string CurrencyCode { get; set; }
public long? OperationId { get; set; }
public virtual Operation Operation { get; set; }
}
Said that, I want to get all the records (actually are about 40K-50K), but only for a few properties. As I mentioned before, the Multimedia table can have a lot of records for each Property, but I only need the first one with the smaller Order value and sorted by DatePublished. After that, I need to convert the result into MapMarker object, which is as follows:
public class MapMarker : EstateBase
{
public long Price { get; set; }
public int Category { get; set; }
public List<Tuple<string, string, string>> Prices { get; set; }
}
In order to achieve this, I made the following:
public async Task<IEnumerable<MapMarker>> GetGeolocatedPropertiesAsync(int quantity)
{
var properties = await GetAllProperties().AsNoTracking()
.Include(g => g.GeoLocation)
.Include(m => m.Multimedia)
.Include(p => p.Operation).ThenInclude(o => o.Price)
.Include(p => p.Operation).ThenInclude(o => o.OperationType)
.Where(p => p.GeoLocation != null
&& !string.IsNullOrEmpty(p.GeoLocation.Address)
&& p.GeoLocation.Longitude != null
&& p.GeoLocation.Latitude != null
&& p.StatusCode == (byte)StatusCode.Online
&& p.Operation.Count > 0)
.OrderByDescending(p => p.ModificationDate)
.Take(quantity)
.Select(p => new {
p.Id,
p.Url,
p.GeneratedTitle,
p.GeoLocation.Address,
p.GeoLocation.Latitude,
p.GeoLocation.Longitude,
p.Domain,
p.Operation,
p.Multimedia.OrderBy(m => m.Order).FirstOrDefault().Resize360x266
})
.ToListAsync();
var mapMarkers = new List<MapMarker>();
try
{
foreach (var property in properties)
{
var mapMarker = new MapMarker();
mapMarker.Id = property.Id.ToString();
mapMarker.Url = property.Url;
mapMarker.Title = property.GeneratedTitle ?? string.Empty;
mapMarker.Address = property.Address ?? string.Empty;
mapMarker.Latitude = property.Latitude.ToString() ?? string.Empty;
mapMarker.Longitude = property.Longitude.ToString() ?? string.Empty;
mapMarker.Domain = ((Domain)Enum.ToObject(typeof(Domain), property.Domain)).ToString();
mapMarker.Image = property.Resize360x266 ?? string.Empty;
mapMarker.Prices = new List<Tuple<string, string, string>>();
foreach (var operation in property.Operation)
{
foreach (var price in operation.Price)
{
var singlePrice = new Tuple<string, string, string>(operation.OperationType.Name, price.CurrencyCode, price.Amount.ToString());
mapMarker.Prices.Add(singlePrice);
}
}
mapMarkers.Add(mapMarker);
}
}
catch (Exception ex)
{
throw;
}
return mapMarkers;
}
but the results take more than 14 minutes and this method could be called multiple times in a minute. I want to optimize it to return the results in the less time possible. I alreay tried removing ToListAsync(), but in the foreach loop it takes a lot of time too, and that makes all the sense.
So, what do you think I can do here?
Thanks in advance.
UPDATE:
Here is GetAllProperties() method, I forgot to include this one.
private IQueryable<Property> GetAllProperties()
{
return _dbContext.Property.AsQueryable();
}
And the SQL query that Entity Framework is making against SQL Server:
SELECT [p].[Id], [p].[Url], [p].[GeneratedTitle], [g].[Address], [g].[Latitude], [g].[Longitude], [p].[Domain], (
SELECT TOP(1) [m].[Resize360x266]
FROM [Multimedia] AS [m]
WHERE [p].[Id] = [m].[PropertyId]
ORDER BY [m].[Order]), [t].[Id], [t].[CreationDate], [t].[ModificationDate], [t].[OperationTypeId], [t].[PropertyId], [t].[Id0], [t].[CreationDate0], [t].[ModificationDate0], [t].[Name], [t].[Id1], [t].[Amount], [t].[CreationDate1], [t].[CurrencyCode], [t].[ModificationDate1], [t].[OperationId]
FROM [Property] AS [p]
LEFT JOIN [GeoLocation] AS [g] ON [p].[Id] = [g].[PropertyId]
LEFT JOIN (
SELECT [o].[Id], [o].[CreationDate], [o].[ModificationDate], [o].[OperationTypeId], [o].[PropertyId], [o0].[Id] AS [Id0], [o0].[CreationDate] AS [CreationDate0], [o0].[ModificationDate] AS [ModificationDate0], [o0].[Name], [p0].[Id] AS [Id1], [p0].[Amount], [p0].[CreationDate] AS [CreationDate1], [p0].[CurrencyCode], [p0].[ModificationDate] AS [ModificationDate1], [p0].[OperationId]
FROM [Operation] AS [o]
LEFT JOIN [OperationType] AS [o0] ON [o].[OperationTypeId] = [o0].[Id]
LEFT JOIN [Price] AS [p0] ON [o].[Id] = [p0].[OperationId]
) AS [t] ON [p].[Id] = [t].[PropertyId]
WHERE (((([g].[Id] IS NOT NULL AND ([g].[Address] IS NOT NULL AND (([g].[Address] <> N'') OR [g].[Address] IS NULL))) AND [g].[Longitude] IS NOT NULL) AND [g].[Latitude] IS NOT NULL) AND ([p].[StatusCode] = CAST(1 AS tinyint))) AND ((
SELECT COUNT(*)
FROM [Operation] AS [o1]
WHERE [p].[Id] = [o1].[PropertyId]) > 0)
ORDER BY [p].[ModificationDate] DESC, [p].[Id], [t].[Id], [t].[Id1]
UPDATE 2: As #Igor mentioned, this is the link of the Execution Plan Result:
https://www.brentozar.com/pastetheplan/?id=BJNz9KdQI
Ok, a few things that should help. #1. .Include() and .Select() should in general be treated mutually exclusive.
You are selecting:
p.Id,
p.Url,
p.GeneratedTitle,
p.GeoLocation.Address,
p.GeoLocation.Latitude,
p.GeoLocation.Longitude,
p.Domain,
p.Operation,
p.Multimedia.OrderBy(m => m.Order).FirstOrDefault().Resize360x266
but then in your foreach loop accessing Price and OperationType entities off it.
Edit Updated the example for the collection of operation. (Whups)
Instead I would recommend:
p.Id,
p.Url,
p.GeneratedTitle,
p.GeoLocation.Address,
p.GeoLocation.Latitude,
p.GeoLocation.Longitude,
p.Domain,
Operations = p.Operation.Select( o => new
{
OperationTypeName = o.OperationType.Name,
o.Price.Amount,
o.Price.CurrencyCode
}).ToList(),
p.Multimedia.OrderBy(m => m.Order).FirstOrDefault().Resize360x266
Then adjust your foreach logic to use the returned properties rather than a returned entity and related entity values.
Loading 40-50k records with something like that image field (MultiMedia) is potentially always going to be problematic. Why do you need to load all 50k in one go?
This looks like something that would put markers on a map. Solutions like this should consider applying a radius filter at the very least to get markers within a reasonable radius of a given center point on a map, or if loading a larger area (zoomed out map) calculating regions and filtering data by region or getting a count falling in that region and loading/rendering the locations in batches of 100 or so rather than potentially waiting for all locations to load. Something to consider.
Related
I have two classes in my database which are defined as classes, fed into entity and then called from the API
The full method is below for the calls. The first call works fine, the second throws the exception
public async Task<ActionResult<List<QuizForms>>> GetQuiz([FromQuery]string id)
{
var form = await _context.QuizForms.Where(t=>t.QuizId == id).ToListAsync();
if (form == null)
{
return NotFound();
}
var elem = new List<Element>();
foreach(var e in form)
{
var data = await _context.Element.Where(t => t.ElementId == e.ElementId).ToListAsync();
elem.AddRange(data);
e.Element.AddRange(elem);
}
return form;
}
When the var data line is hit, an excception is thrown
Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'QuizFormsFormId'.
It looks like the name of the class and column name are being concatenated and the used as the query parameter.
The two classes look like this
public class QuizForms
{
[Key]
public int FormId { get; set; }
public string QuizId { get; set; } = "";
#nullable enable
public string? Title { get; set; }
public int? ElementId { get; set; }
public List<Element>? Element { get; set; }
public int? PreviousId { get; set; }
public int? NextId { get; set; }
#nullable disable
}
and
public class Element
{
[Key]
public int Id { get; set; }
public int ElementId { get; set; }
#nullable enable
public int? MathsId { get; set; }
public int? QuestionId { get; set; }
public int? InformationId { get; set; }
public int? AnswerId { get; set; }
#nullable disable
public string QuizId { get; set; } = "";
}
Is it because I'm not using Id for the primary key or do I need to do something else so the class and property aren't concatented like this?
On EF core have Two tables(Page, Group) both have many to many relations with junction table GroupPage. Want to get all pages data with junction table related data based on groupId as like bellow.
If you construct your EF relation correctly you should not have a GroupPage entity.
See Entity Framework Database First many-to-many on how to construct your EF EDM correctly.
Once you have your EDM correctly mapped, you should have the classes
public class Page
{
public int Id { get; set; }
public ICollection<Group> Groups { get; set; }
...
}
public class Group
{
public int Id { get; set; }
public ICollection<Page> Pages { get; set; }
...
}
Then you just need to do the following
public IQueryable<Page> GetPages(int groupId)
{
return from group in _context.Groups
where group.Id == groupId
from page in group.Pages
select page;
}
The following syntax is self-descriptive. Here are the entities structure and Page Dto.
public class Page
{
public int Id { get; set; }
public ICollection<Group> Groups { get; set; }
...
}
public class Group
{
public int Id { get; set; }
public ICollection<Page> Pages { get; set; }
...
}
public class PageGroup
{
public int PageId { get; set; }
public Page Page { get; set; }
public int GroupId { get; set; }
public Group Group { get; set; }
}
public class PagesDto
{
public string Name { get; set; }
public int GroupId { get; set; }
public int PageId { get; set; }
public string Description { get; set; }
public string Tab { get; set; }
public string Module { get; set; }
public bool? IsActive { get; set; }
public bool? IsDefault { get; set; }
public PagesDto()
{
IsActive = false;
IsDefault = false;
}
}
Following function help us to get group related pages information.
public async Task<List<PagesDto>> GetAllPagesByGroupId(int selectedGroupId)
{
//get all pages
var pages = await _pagesRepository.GetAll().Select(p => new PagesDto {
PageId = p.Id,
Name = p.Name,
GroupId = 0
}).ToListAsync();
//get group ralated pages
var selectedGroupPageIds = _groupPagesRepository
.GetAll()
.Where(p => p.GroupId == selectedGroupId)
.Select(p => p.PageId);
//update page information base on group related pages info.
foreach (var item in pages.Where(p=>selectedGroupPageIds.Contains(p.PageId)))
{
item.GroupId = selectedGroupId;
}
return pages;
}
I am using EF Core and I tried to create a one-to-one relationship between three tables (Car, ElectricCar and PetrolCar)
public class Car
{
public int Id { get; set; }
public string RegistrationNumber { get; set; }
public ElectricCar Company { get; set; }
public PetrolCar Trust { get; set; }
}
public class ElectricCar
{
public int ElectricCarId { get; set; }
public double BatteryCapacityWattage{ get; set; }
public int CarId { get; set; }
public Car Car { get; set; }
}
public class PetrolCar
{
public int PetrolCarId { get; set; }
public double TankCapacity { get; set; }
public int CarId { get; set; }
public Car Car { get; set; }
}
public partial class CarDbContext : Microsoft.EntityFrameworkCore.DbContext
{
public CarDbContext()
{
}
public CarDbContext(DbContextOptions<CarDbContext> options)
: base(options)
{
}
public DbSet<ElectricCar> ElectricCar { get; set; }
public DbSet<Car> Car { get; set; }
public DbSet<PetrolCar> PetrolCar { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (!optionsBuilder.IsConfigured)
{
optionsBuilder.UseSqlServer("Server=DESKTOP-PC\\SQLLOCAL;Database=OneToOneEFCoreCar;Trusted_Connection=True;");
}
}
}
and the code that inserts the data:
CarDbContext context = new CarDbContext();
context.Car.Add(new Car
{
RegistrationNumber = "EL123",
Company = new ElectricCar() { BatteryCapacityWattage = 2000 }
});
context.Car.Add(new Car
{
RegistrationNumber = "PETR123",
Trust = new PetrolCar() { TankCapacity = 50 }
});
context.SaveChanges();
That works without any issue and creates the following data
When I go to the PetrolCar I insert a new row with CarId = 1 and it accepts it without giving any error although that CarId is used in the ElectricCar table as CarId.
Is there any way to restrict this?
If you're entirely set on keeping your object models / data structure the same as it is above then a unique constraint across the two tables isnt really natively achievable.
One possible in code solution (though its not particularly clean, so I would suggest restructuring your data over this, though that seems to be something you would like to avoid) is to override the SaveChanges method.
something along the lines of:
public override SaveChanges()
{
var petrolCars = ChangeTracker.Entries().Where(e is PetrolCar).ToList();
foreach(var pCar in petrolCars)
{
if(query the database for electric cars to see if car id exists)
{
do some sort of error processing and avoid saving;
}
}
base.SaveChanges();
}
it does mean creating a context class that inherits from the default context, though it adds a lot of flexibility in terms of doing something like this (obviously you would want to handle the other cases too of cars having the same id in the other direction)
I am using Orc.FluentValidation and I have:
[ValidatorDescription(nameof(Customer), ValidationResultType.Error,
Orc.FluentValidation.ValidationType.BusinessRule)]
public class CustomerBusinessRuleValidator : AbstractValidator<Customer>
{
public CustomerBusinessRuleValidator()
{
RuleFor(x => x.Addresses).Must(x => x != null && x.Count > 0 && x.Any(add => add.IsCurrent))
.WithMessage("Customer object is required to have at least 1 current address.");
}
}
CustomerAddress
public class CustomerAddress : Entity
{
[DomainSignature] public Address Address { get; set; }
[DomainSignature] public Lookup AddressType { get; set; }
[DomainSignature] public bool IsCurrent { get; set; }
}
Customer
public class Customer : Entity
{
[DomainSignature]
public string Code { get; set; }
public Gender Gender { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string MiddleName { get; set; }
public DateTime DateOfBirth { get; set; }
public Lookup PlaceOfBirth { get; set; }
public string PhoneNumber { get; set; }
public string Email { get; set; }
public ICollection<CustomerAddress> Addresses { get; set; }
public Lookup Occupation { get; set; }
public IdDocument Id1 { get; set; }
public IdDocument Id2 { get; set; }
}
On the View even after a CustomerAddress with IsCurrent = true is added for the Customer, the message still shows. Also, I am not sure why some field-bound controls show the error and others not _this is not a field validation rule.
Is there like a method call to be done after adding the CustomerAddress to the Addresses collection?
i.imgur.com/eecAFuJ.png
Make sure you raise a property changed on the whole collection for error validation (e.g. RaisePropertyChanged(nameof(MyCollection)), otherwise the UI can't update the validation results.
Also, I am not sure why some field-bound controls show the error and
others not _this is not a field validation rule.
This is probably because of the default styles you are using. For most of the controls, Orchestra creates an error template (decorator), but not every control had this. We've been working on adding these last week, so I recommend to try out the latest alpha of Orchestra & Orc.Controls.
Also make sure to set ValidateOnDataErrors and NotifyOnValidationErrors on the binding to show the validation in the UI.
in my Winform Application, I have Suppliers, Customers, Transport Companies. They are similar as they are basically some kind of Contacts, however they do different slightly in term of available fields.
For example Suppliers need have StartDate and EndDate fields. And currently even though Suppliers and Customers could have more than one contact person\entity, but we are not going do that in these release, but the Transport companies will have more than one contact person\entity and addresses. At the same time, the Supplier and Customer do require PO Address and Delivery Address, and two phone numbers just in case.
Currently in my Code First Entities, I have Suppliers, Customers and Transport Companies each contains a PrimaryContact which is a Contact Type, and for each Contact type, I have a ICollection of Address and Phone which in turn store one or more than one address and phone information. The difference is that Transport Companies will have a collection of Contact in addition of PrimaryContact.
As my understanding, even I have the freedom of design the DB/Entity by myself, there is not always the case that Objects in BLL is exactly mapping of the DB structure underneath.
So the idea is in my BLL layer, I will translate the data from Supplier to BOSupplier to Presentation Layer, and will doing translation to Supplier when get data back from Presentation Layer to DAL. Because in my Presentation Layer, the Supplier will looks like:
public class BOSupplier
{
// Primery key
public int ID { get; set; }
public string Name { get; set; }
public string TaxNumber { get; set; }
public bool InActive { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
public string BankAccountNumber { get; set; }
public string BankAccountName { get; set; }
// Property related to Contact Table
public string FirstName { get; set; }
public string LastName { get; set; }
public string EmailAddress { get; set; }
public string SkypeName { get; set; }
// Proterty related to Address Table
// PO address Info
public string POAddressLine { get; set; }
public string POCity { get; set; }
public string PORegion { get; set; }
public string POCountry { get; set; }
public string POPostCode { get; set; }
// Delivery AddressLine
public string DelAddressLine { get; set; }
public string DelCity { get; set; }
public string DelRegion { get; set; }
public string DelCountry { get; set; }
public string DelPostCode { get; set; }
// Proterties related to Phone table
public string PhoneNumber1 { get; set; }
public string PhoneNumber2 { get; set; }
}
}
But in my DAL Layer, my Supplier will looks like this:
public class Supplier
{
// Primery key
public int ID { get; set; }
public string Name { get; set; }
public virtual Contact PrimaryContact { get; set; }
public string TaxNumber { get; set; }
public bool InActive { get; set; }
public DateTime? StartDate { get; set; }
public DateTime? EndDate { get; set; }
public string BankAccountNumber { get; set; }
public string BankAccountName { get; set; }
}
Then when I am actually writing code for BLL classes to manage my intermediate BOSupplier object and List which didn't actually mapping back an Entity to DB side. It seems a lots low level code just to transfer/map fields from two slightly different BOSupplier and Supplier, like this:
public static IEnumerable<BOSupplier> GetBOSuppliers()
{
var suppliers = dbContext.Suppliers;
BOSupplier currentSupplier;
foreach (Supplier supplier in suppliers)
{
currentSupplier = new BOSupplier()
{
ID = supplier.ID,
Name = supplier.Name,
Code = supplier.Code,
FirstName = supplier.PrimaryContact.FirstName,
TaxNumber = supplier.TaxNumber
};
// PO Address
Address poAddress = supplier.PrimaryContact.Addresses
.FirstOrDefault<Address>(a => a.AddressTypeValue == (int)AddressTypes.Postal);
if (poAddress != null)
{
currentSupplier.POAddressLine = poAddress.AddressLine1;
currentSupplier.POCity = poAddress.City;
currentSupplier.POCountry = poAddress.Country;
}
// Delivery Address
Address delAddress = supplier.PrimaryContact.Addresses
.FirstOrDefault<Address>(a => a.AddressTypeValue == (int)AddressTypes.Delivery);
if (delAddress != null)
{
currentSupplier.DelAddressLine = delAddress.AddressLine1;
currentSupplier.DelCity = delAddress.City;
currentSupplier.DelCountry = delAddress.Country;
}
// ToDo:
// There is probably more to think about how we want map multi phone numbers into limited two phone numbers
if (supplier.PrimaryContact.Phones.Count > 0)
{
foreach (Phone phone in supplier.PrimaryContact.Phones)
{
if (phone.PhoneType == PhoneTypes.Default)
{
currentSupplier.PhoneNumber1 = phone.PhoneNumber;
}
else
{
currentSupplier.PhoneNumber2 = phone.PhoneNumber;
}
}
}
this.boSupplierList.Add(currentSupplier);
}
return boSupplierList;
}
I am keep thinking: "Maybe my Entity Model should be simpler, or there is some better way of doing what I am trying to?". So please, from your experience, tell me that my Entity model are on over-complex side, or I just need some better way of mapping from BOSuppier to Supplier or some other thoughts.
Your entity model is not complex according to your description of the domain. You can use AutoMapper to map your Supplier to BOSupplier. Here is an example of flattening object graph using AutoMapper.
I see a problem in your GetBOSuppliers(). It uses lazy loading when you access PrimaryContact and Addresses. To avoid the multiple round trips to database you can eager load them as follows.
var suppliers = dbContext.Suppliers.Include(s => s.PrimaryContact.Addresses);