So I'm using the C# nuget wrapper around Azure Search. My problem is I have a index of products:
public class ProductDocument
{
[System.ComponentModel.DataAnnotations.Key]
public string Key { get; set; }
[IsSearchable]
public string Sku { get; set; }
[IsSearchable]
public string Name { get; set; }
[IsSearchable]
public string FullDescription { get; set; }
[IsSearchable]
public List<CustomerSkuDocument> CustomerSkus { get; set; }
}
public class CustomerSkuDocument
{
[IsSearchable]
public int AccountId { get; set; }
[IsSearchable]
public string Sku { get; set; }
}
Example data would be:
new Product() { Key= 100,Name="Nail 101",Sku = "CCCCCCCC", CustomerSkus = new List<ProductCustomerSku>()
{
new ProductCustomerSku() {AccountId = 222, CustomerSku = "BBBB"},
new ProductCustomerSku() {AccountId = 333, CustomerSku = "EEEEEEE"}
}
So the problem is around CustomerSkuDocument.
When I Search I need to pass the AccountId in as well as the search term, however the AccountId is only used for when searching the ProductCustomerSkus.
Basically an Account can have different customer skus but it's only associated to that account - I don't want a separate index per account.
So my call would be something like /AccountId=222&term=BBBB which would find the match.
However /AccountId=333&term=BBBB would not find a match.
So I'm calling it like:
SearchParameters sp = new SearchParameters();
sp.SearchMode = SearchMode.Any;
sp.QueryType = QueryType.Full;
DocumentSearchResult<ProductDocument> results =
productIndexClient.Documents.Search<ProductDocument>(term, sp);
Where term is the normal search term, tried it with adding the AccountId but it doesn't work.
Azure Search does not support repeating data structures nested under a property of the outer document. We're working on this (see https://feedback.azure.com/forums/263029-azure-search/suggestions/6670910-modelling-complex-types-in-indexes), but we still have some work to do before we can release that.
Given that, the example you're showing is not probably indexing the nested parts. Can you post the search index definition you're using? While we work in direct support for complex types, you can see your options for approach here: https://learn.microsoft.com/en-us/azure/search/search-howto-complex-data-types
From the above you'll arribe at a index structure that will also guide your query options. If all you need is equality, perhaps you can simply include the accountId and the SKU in the same field and use a collection field so you can have multiple instances. For your query you would issue a search query that requires the accountId and has the rest as optional keywords.
Related
I've got a relatively basic model - Users and Tags. There is a fixed list of Tags. A User can have multiple Tags and a Tag can be used by multiple users.
I had gone with structure below and finding performance issues when returning results.
public class User
{
public string Id {get; set;}
public virtual List<UserTag> UserTags {get; set}
}
public class UserTag
{
public string UserId { get; set; }
public User User { get; set; }
public int TagId { get; set; }
public Tag Tag{ get; set; }
}
public class Tag
{
[Key]
public int TagId { get; set; }
public string Name { get; set; }
public virtual List<UserTag> UserTags { get; set; }
}
I have the following query which is takings a long time (several seconds):
var x = db.Users.Include(u => u.UserTags).ThenInclude(u => u.Trait).ToList<User>();
I have tried writing it as such, which has improved the time, however it is still taking too long:
db.UserTags.Load();
db.Tags.Load();
var x = db.Users.ToList<User>();
Is there any other way to speed this up? Running a query directly in SQL SMS is almost instant (e.g.
select * from Users u left outer join UserTags t on t.UserId = u.Id)
In terms of data rows, it is apx Tags: 100, UserTags:50,000, Users: 5,000
First you can check how EF translates your request to SQL Server - therefore use the "SQL Server Profiler"
Then you could use the genereated query to check if there might be an missing index which speeds up the query
You also can try to write a Join instead of ThenInclude and see how the query then behaves
best regards
Jimmy
I come from Django, and with Django's ORM when defining a model I can set options, for example:
class MyModel(Model):
CHOICES = (...) # List of choices here
# it has tu be a 2-tuple list
# but that's is not the point right now.
choice = IntegerField(choices=CHOICES) # Set the choices.
Now, when MyModel is used to create an instance, the attribute(property) choice can hold only values defined in CHOICES.
There is some like this for EntityFrameworkCore-2.0?
It is something like this possible?
class MyEntity
{
[choices=EnumDefiningChoices]
property enum choices { get; set; }
}
A property can have a single defined value, an int, string, etc in order to be stored in a database.
An approach is to use 2 tables, one for your entity and one for the choices.
MyEntity:
public class MyEntity
{
public int Id { get; set; }
// all the other properties
public int ChoiceId { get; set; } // Here is the refference for the choice
public virtual Choice Choice { get; set; }
}
Choice:
public class Choice
{
public int Id { get; set; }
public string Name { get; set; }
}
As you can see the ChoiceId will allow only values from the Choice table.
However if you want to use an Enum there is no "blocking" mechanism using EF, this must be implemented at the application level.
Indeed, there is a way!
With EF > 6 you can do something like this:
public enum Option
{
option_1 = 1,
option_2,
option_3
}
class SomeEntity
{
public Option Option { get; set; }
}
And you can get the enum option name with:
var option_name = Option.GetName(typeof(Option), some_entity.option_2);
Console.WriteLine(option_name);
References
Working with Enumerated Values in Entity Framework
Enum.GetName Method (Type, Object)
I am using EF6 and SQL Server 2014.
I have something like the following simplified model:
public class Order
{
public int Id { get; set; }
public DateTime Created { get; set; }
public IEnumerable<OrderItem> OrderItems { get; set; }
}
public class OrderItem
{
public int Id { get; set; }
public int OrderId { get; set; }
public Order Order { get; set; }
public string ItemName { get; set; }
public int OrderItemNumber { get; set; }
}
The Id properties are standard auto-increment primary keys.
The OrderItem.OrderItemNumber property uniquely identifies an OrderItem within the context of a given Order.
When a new OrderItem row is inserted it needs to be assigned the next OrderItemNumber for the parent Order, something like
SELECT COALESCE(MAX(OrderItemNumber),0) FROM dbo.OrderItems WHERE OrderId = 2
Is there a means using EF6/SQL Server to auto-assign the OrderItemNumber at the point a row is inserted?
Although this proved an interesting learning experience on EF support for triggers (see https://github.com/NickStrupat/EntityFramework.Triggers) I eventually decided to re-model.
In this case my question was a result of incorrect modelling. The OrderItemNumber property was redundant so I removed it; the sequence the OrderItems were created can be inferred from the Id value, and there was no real requirement to allow re-sequencing.
I store animals in an SQL Server database table created by Entity Framework Code First. This is the POCO for it:
[Table("Animal")]
public class Animal
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int ID { get; set; }
public string SomeData { get; set; }
public byte[] OtherData { get; set; }
public int ExternalSourceId { get; set; }
public string ExternalSourceAnimalId { get; set; }
}
Some animals are regularly updated by an external source (there are a few sources but a specific animal is updated from maximum 1 source). Also when an external source has a new animal it needs to be inserted. Some animals are maintained locally, these have no external source.
An external source have the Animal identified by the ExternalSourceAnimalId. These are unique for one source but there are chances of 2 sources using overlapping IDs, so ExternalSourceId is also there to make the 2 of them a combined natural key.
So I get a huge list of animals from an external source and I need to insert or update them in the local database depending on the existence of the specific external key in our datebase.
This is my current solution on doing this (in a class inheriting form SharpRepository):
public void InsertOrUpdateAnimal(ExternalAnimal exAnimal, int externalSourceId)
{
var animal = DbSet.SingleOrDefault(o => o.ExternalSourceId == externalSourceId && o.ExternalSourceAnimalId == exAnimal.Id);
if (animal != null)
{
CopyData(exAnimal, animal); // copies the properties from exAnimal to animal
this.Update(animal);
}
else
{
animal = new Animal();
CopyData(exAnimal, animal);
this.Add(animal);
}
}
Since this is a bulk operation and takes quite some time I was wondering if there is a faster solution for this. For example if there is a way to upsert in a single database operation.
I've started to play with Dapper.Net, and am really loving it so far - however, I have run into one problem.
Say that I have a POCO class like:
public class Person
{
public string FirstName { get; set; }
public string LastName { get; set; }
public string FullName { get { return FirstName + " " + LastName; } }
}
Now, using Dapper.Net and the Dapper.Net extensions, I want to simply load all instances of that data type from the DB by doing this:
string connectionString = CloudConfigurationManager.GetSetting("DBConnection");
using (SqlConnection cn = new SqlConnection(connectionString))
{
cn.Open();
IEnumerable<Types.Person> entities = cn.GetList<Types.Person>();
var forceMe = entities.ToList();
}
This works fine in the Linq setup, but when it hits the line with the .ToList(), which forces the evaluation, it blows up with "invalid column names" on FullName. Thinking that it might respect the Entity Framework DataAnnotations stuff for NotMapped, I tried adding a NotMapped attribute (after adding EF 5 to the project). This didn't work.
So, the question is, how do I tell Dapper.Net that a column isn't to be expected from the DB? Is this a problem with the extensions, trying to map a DB column for everything it sees in the model POCO? Do I need to revert to writing SQL, and explicitly ask for the columns that I want only, or is there a way to get an equivalent to NotMapped on the column?
I think the only way to to ignore certain properties from being mapped is to implement an auto class mapper, where you can specify your custom field mappings. For example:
public class CustomMapper : ClassMapper<Foo>
{
public CustomMapper()
{
Table("FooTable");
Map(f => f.Id).Column("FooId").Key(KeyType.Identity);
Map(f => f.DateOfBirth).Column("BirthDate");
Map(f => f.FirstName).Column("First");
Map(f => f.LastName).Column("Last");
Map(f => f.FullName).Ignore();
Map(f => f.Calculated).ReadOnly();
}
}
public class Foo
{
public int Id { get; set; }
public DateTime DateOfBirth { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string FullName
{
get { return string.Format("{0} {1}", FirstName, LastName); }
}
}
In the above example, FullName is being ignored.
The auto-mapper also allows you to adjust table names, in case your POCO class names do not match table names.
Also, keep in mind that you must keep your custom maps in the same assembly as your POCO classes. The library uses reflection to find custom maps and it only scans one assembly.
Hope this helps,
Good luck