I have an SQLite database, which contains one table named "Main". Each record of this table contains only two fields: ID (integer, primary key) and name (string). There are 100 records in the database.
Using Entity Framework Power Tools I've created the Code First model from the existing database. The model is rather simple:
// MainMap.cs
public class MainMap : EntityTypeConfiguration<Main>
{
public MainMap()
{
// Primary Key
this.HasKey(t => t.ID);
// Properties
this.Property(t => t.name)
.IsRequired()
.HasMaxLength(50);
// Table & Column Mappings
this.ToTable("Main");
this.Property(t => t.ID).HasColumnName("ID");
this.Property(t => t.name).HasColumnName("name");
}
}
// Main.cs
public partial class Main
{
public long ID { get; set; }
public string name { get; set; }
}
// mainContext.cs
public partial class mainContext : DbContext
{
static mainContext()
{
Database.SetInitializer<mainContext>(null);
}
public mainContext()
: base("Name=mainContext")
{
}
public DbSet<Main> Mains { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new MainMap());
}
}
Now I'm trying to get the records from the database:
mainContext context = new mainContext();
context.Mains.Load();
Now I can use context.Mains.Local with a comfort for different purposes (actually, I bind it to ListView's ItemsSource).
The problem is that context.Main.Load() line executes for about 2.7 seconds. I think, it is too much time as for retrieving about 100 records from a simple database. Although, I'm a newcomer to databases, so, maybe I'm not right and 2.7 seconds is rather suitable period of time. My CPU is Intel i3-3220 (2x3.30 GHz), Entity Framework's version is 6.0.
Maybe, my Code First model is poor, or maybe EF doesn't provide high performance, or maybe there is no need to call Load() method to obtain records (but if I don't call it, context.Mains.Local is empty).
So, how can I increase the performance of getting the records from the database?
Any help and hints will be appreciated.
i ran some tests with both SQLite and SQL Server. on my laptop (corei7 2630QM 2.00GHZ & win7 64bit) the load time for both was ~1.5sec.
then i tried to warm it up with something like
context.Database.Exists();
and the time reduced to ~700ms for both.
i used "Prefer 32-bit" and "Optimize code" options in build tab of the project properties. these options produced best results.
try these and see if the load time changes.
Related
I have some models like those below:
public class Mutant
{
public long Id { get; set; }
...
// Relations
public long OriginalCodeId { get; set; }
public virtual OriginalCode OriginalCode { get; set; }
public int DifficultyLevelId { get; set; }
public virtual DifficultyLevel DifficultyLevel { get; set; }
}
and
public class OriginalCode
{
public long Id { get; set; }
...
// Relations
public virtual List<Mutant> Mutants { get; set; }
public virtual List<OriginalCodeInputParameter> OriginalCodeInputParameters { get; set; }
}
and in the OnModelCreating of DBContext I made the relations like these:
modelBuilder.Entity<Mutant>()
.HasOne(m => m.OriginalCode)
.WithMany(oc => oc.Mutants)
.HasForeignKey(m => m.OriginalCodeId)
.OnDelete(Microsoft.EntityFrameworkCore.Metadata.DeleteBehavior.Restrict);
modelBuilder.Entity<Mutant>()
.HasOne(m => m.DifficultyLevel)
.WithMany(dl => dl.Mutants)
.HasForeignKey(m => m.DifficultyLevelId)
.OnDelete(Microsoft.EntityFrameworkCore.Metadata.DeleteBehavior.Restrict);
now when I request for Mutants, the OriginalCode is null:
but as soon as I request for OriginalCodes like below:
then the OriginalCode field of the mutants will be not null:
What is the reason and how could I fix it?
The reason is explained in the Loading Related Data section of the EF Core documentation.
The first behavior is because EF Core currently does not support lazy loading, so normally you'll get null for navigation properties until you specifically load them via eager or explicit loading. However, the Eager loading section contains the following:
Tip
Entity Framework Core will automatically fix-up navigation properties to any other entities that were previously loaded into the context instance. So even if you don't explicitly include the data for a navigation property, the property may still be populated if some or all of the related entities were previously loaded.
which explains why the navigation property is not null in the second case.
Now, I'm not sure which of the two behaviors do you want to fix, so will try to address both.
The first behavior can be "fixed" by using one of the currently available methods for loading related data, for instance eager loading:
var mutants = db.Mutants.Include(m => m.OriginalCode).ToList();
The second behavior is "by design" and cannot be controlled. If you want to avoid it, make sure to use fresh new DbContext instance just for executing a single query to retrieve the data needed, or use no tracking query.
Update: Starting with v2.1, EF Core supports Lazy Loading. However it's not enabled by default, so in order to utilize it one should mark all navigation properties virtual, install Microsoft.EntityFrameworkCore.Proxies and enable it via UseLazyLoadingProxies call, or utilize Lazy-loading without proxies - both explained with examples in the EF Core documentation.
Using Package Manager Console install Microsoft.EntityFrameworkCore.Proxies
install-package Microsoft.EntityFrameworkCore.Proxies
And then in your Context class add .UseLazyLoadingProxies():
namespace SomeAPI.EFModels
{
public partial class SomeContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (!optionsBuilder.IsConfigured)
{
optionsBuilder
.UseLazyLoadingProxies()
.UseSqlServer(connectionString);
}
}
}
}
My question: what determines the speed(performance) of calling DbContext.SaveChanges() method? And is it a bad practice to put all the DbSets in a single DbContext?
I have a c#/WPF/MS SQL Server/Entity Framework Core project, which is actually for my company's wholesale business.
I implemented a single DbContext which contains dozens of DbSet's, each of which, of course, represents a table in the database. There are about 10 major tables representing orders, order details, customers, products, etc, and each of the major DbSet/tables contains about 50,000 to 150,000 records in it. The problem is when DbContext.SaveChanges method is called, it takes over 9,000ms(9 sec) to execute! I put ALL of the DbSets in the same DbContext. Is this a bad habit and the cause for slow speed?
For a test, I created a separate DbContext and put only one DbSet in it. The DbSet has about 100,000 records, but calling SaveChanges for that took about 500ms, which was a significant improvement.
Given my situation, what is the best practice for database performances? Please help.
public class MyDbContext : DbContext
{
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseLazyLoadingProxies().UseSqlServer(DbConn.GetConnStr());
base.OnConfiguring(optionsBuilder);
}
public DbSet<Order> Orders { get; set; } // This has 100k+ records.
public DbSet<OrderDetail> OrderDetails { get; set; } // This has 150k+ records.
public DbSet<Ship> Ships { get; set; } // 100k+ records
public DbSet<ShipDetail> ShipDetails { get; set; } // 150k+ records
public DbSet<Customer> Customers { get; set; } // 100k records
public DbSet<Product> Products { get; set; } // 10k+ records
public DbSet<ProductStock> ProductStocks { get; set; }
public DbSet<ProductPrice> ProductPrices { get; set; }
public DbSet<PriceType> PriceTypes { get; set; }
public DbSet<Claim> Claims { get; set; }
public DbSet<Carrier> Carriers { get; set; }
public DbSet<Channel> Channels { get; set; }
public DbSet<Import> Imports { get; set; }
public DbSet<ImportDetail> ImportDetails { get; set; }
}
No, quite the opposite. You should encapsulate one database per dbContext extended class in your app. If it is just one db ( or rather one schema ) then you should not split the class at all.
Instead make a partial class and define different dbSets within domain-like-files that form up a concrete class.
The speed is based on the changes made x items loaded ( rly abstract... ).
The more changes, the more the rows that you are affecting/loading, the harder stuff get.
Biggest hit for you would be the sql updates. When you want to manage very big datasets, skip loading them into memory at all. Work with .FromSqlRaw and do everything at the db level returning the minimum that you need.
For example mass updates is a great case for this.
Also care for the case that you are loading uneeded objects ( relations that you are not using )
Credit to Gert Arnold, rantri, and MKougiouris for your replies and comments. All of you are dead right. Here's what I've figured. As all of you mentioned, the problem was not the fact that a single DbContext has all tables in it. The problem was that I was using and passing around a single "instance" of my DbContext across multiple operations throughout the lifetime of running the appplication. This should NEVER be done with a DbContext.
I figured that a DbContext is supposed to be instantiated for a single unit-of-work or a single operation and then dispose the instance as soon as the operation is over. I was reading ALL of the DataSets into the DbContext and querying as much as possible with the single DbContext instance. This is a guarantee for slow performance.
I said it took 9 seconds(9,000ms) to persist changes to the DB by calling SaveChanges. Now it takes 250ms(0.25 sec) to get the same job done. Hope my comment helps for anyone with the same issue.
I am migrating a legacy database to a new database which we need to access and "manage" (as oxymoronic as it might sound) primarily through Entity Framework Code-First.
We are using MS SQL Server 2014.
The legacy database contained some tables with computed columns. Typical GUID and DateTime stuff.
Technically speaking, these columns did not have a computed column specification, but rather where given a default value with NEWID() and GETDATE()
We all know that it is very easy to configure the DbContext to deal with those properties as follows:
modelBuilder.Entity<Foo>()
.Property(t => t.Guid)
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.Computed);
modelBuilder.Entity<Bar>()
.Property(t => t.DTS)
.HasDatabaseGeneratedOption(DatabaseGeneratedOption.Computed);
The above would instruct the Entity Framework to ignore submitting any supplied values for such properties during INSERTs and UPDATEs.
But now we need to allow for import of legacy records and maintain the OLD values, including the PRIMARY KEY, which is marked as IDENTITY
This means we would have to set the Id, Guid and DTS properties to DatabaseGeneratedOption.None while inserting those records.
For the case of Id, we would have to somehow execute SET IDENTITY_INSERT ... ON/OFF within the connection session.
And we want to do this
importing process via Code-First as well.
If I modify the model and "temporarily" and set those properties to DatabaseGeneratedOption.None after the database has been created, we would get the typical:
The model backing the context has changed since the database was created. Consider using Code First Migrations to update the database.
I understand that we could generate an empty coded-migration with -IgnoreChanges so as to "establish" this latest version of the context, but this wouldn't be an acceptable strategy as we would have to be run empty migrations back-and-forth solely for this purpose.
Half an answer:
We have considered giving these properties nullable types, i.e.
public class Foo
{
...
public Guid? Guid { get; set; }
}
public class Bar
{
...
public DateTime? DTS { get; set; }
}
While caring about the default values in an initial DbMigration:
CreateTable(
"dbo.Foos",
c => new
{
Id = c.Int(nullable: false, identity: true),
Guid = c.Guid(nullable: false, defaultValueSql: "NEWID()"),
})
.PrimaryKey(t => t.Id);
CreateTable(
"dbo.Bars",
c => new
{
Id = c.Int(nullable: false, identity: true),
DTS = c.Guid(nullable: false, defaultValueSql: "GETDATE()"),
})
.PrimaryKey(t => t.Id);
The Question:
But the question remains: Is there a way to switch between DatabaseGeneratedOption.Identity, DatabaseGeneratedOption.Computed and DatabaseGeneratedOption.None at runtime?
At the very least, how could we turn DatabaseGeneratedOption.Identity on/off at runtime?
A certain amount of the configuration of the context is always going to be dependent on the runtime environment - for example, proxy generation and validation. As such, runtime configuration of the Entity Framework DbContext is something I leverage quite heavily.
Although I've never used this approach to switch the configuration of the context on a per use-case basis, I see no reason why this would not work.
In its simplest form, this can be achieved by having a set of EntityTypeConfiguration classes for each environment. Each configuration set is then wired to the DbContext on a per-environment basis. Again, in its simplest form this could be achieved by having a DbContext type per environment. In your case, this would be per use-case.
Less naively, I usually encapsulate the configuration of the context in an environment-specific unit of work. For example, the unit of work for an Asp.Net environment has an underlying DbContext configured to delegate validation to the web framework, as well as to turn off proxy generation to prevent serialisation issues. I imagine this approach would have similar usefulness to your problem.
For example (using brute force code):
// Foo Configuration which enforces computed columns
public class FooConfiguration : EntityTypeConfiguration<Foo>
{
public FooConfiguration()
{
Property(p => p.DateTime).HasDatabaseGeneratedOption(DatabaseGeneratedOption.Computed);
Property(p => p.Guid).HasDatabaseGeneratedOption(DatabaseGeneratedOption.Computed);
}
}
// Foo configuration that allows computed columns to be overridden
public class FooConfiguration2 : EntityTypeConfiguration<Foo>
{
public FooConfiguration2()
{
Property(p => p.DateTime).HasDatabaseGeneratedOption(DatabaseGeneratedOption.None);
Property(p => p.Guid).HasDatabaseGeneratedOption(DatabaseGeneratedOption.None);
}
}
// DbContext that enforces computed columns
public class MyContext : DbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new FooConfiguration());
}
}
// DbContext that allows computed columns to be overridden
public class MyContext2 : DbContext
{
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Configurations.Add(new FooConfiguration2());
}
}
This can obviously be tidied up - we usually use a combination of factory and strategy patterns to encapsulate the creation of a runtime specific context. In combination with a DI container this allows the correct set up configuration classes to be injected on a per-environment basis.
Example usage:
[Fact]
public void CanConfigureContextAtRuntime()
{
// Enforce computed columns
using (var context = new EfContext())
{
var foo1 = new Foo();
context.Foos.Add(foo1);
context.SaveChanges();
}
// Allow overridden computed columns
using (var context = new EfContext2())
{
var foo2 = new Foo { DateTime = DateTime.Now.AddYears(-3) };
context.Foos.Add(foo2);
context.SaveChanges();
}
// etc
}
Ok I’m at a loss, being new to breeze I’m still learning the ropes. My project uses the hot towel template for AngularJs and breeze from John Papa.
He's what I’m trying to achieve: I have a master\slave tables in my database. An "Agency" has many people it can "Notify". Here are the EF classes for the server side:
public class Agency {
public Agency() {
this.Notifies = new HashSet<Notify>();
}
public long Id { get; set; }
[Required, MaxLength(50)]
public string Name { get; set; }
<<removed unneeded details>>
public bool Active { get; set; }
public virtual ICollection<Notify> Notifies { get; set; }
}
public class Notify
{
public long Id { get; set; }
public long? AgencyId { get; set; }
public string Name { get; set; }
<<removed unneeded details>>
public virtual Agency Agency { get; set; }
}
Now the Maps:
public class AgencyMaps : EntityTypeConfiguration<Agency>
{
internal AgencyMaps()
{
HasKey(x => x.Id);
}
}
public class NotifyMap : EntityTypeConfiguration<Notify>
{
internal NotifyMap()
{
HasKey(x => x.Id);
HasOptional(x => x.Agency)
.WithMany(p => p.Notifies)
.HasForeignKey(i => i.AgencyId);
}
}
Now on the client side I use breeze to create new entities like this:
// create a new entity
function create() {
return manager.createEntity(entityName);
}
// create a new notify entity
function createNotify(){
return manager.createEntity(entityNameNotify);
}
Then there are two scenarios I need to achieve:
- First is where I retrieve an existing agency and add additional
people to notify
- Second is where I create a new agency and add people
to notify
Both fail in the same place.
Note: I’m using SQL server and my Id fields are bigint (long) at this point in time.
I’m retrieving the “Agency” entity and placing it in a variable called “vm.agency”. “vm.agency” has a navigation called “notifies” with an entity type of “Notify”. So when I want to create and add a new person I’m calling this function:
function addNotifyRec(){
if (vm.agency !== undefined){
var notifyRec = datacontext.agency.createNotify(); //<<< fails here
notifyRec.agencyId = vm.agency.id;
notifyRec.name = vm.notify.name;
<<removed unneeded details>>
vm.agency.notifies.push(notifyRec);
logSuccess(“New person to notify added”);
}
else
{ logError(“Agency is undefined”); }
}
As soon as the createNotify() is called I get the “Ids can not be autogenerated for entities with multipart keys” error.
So I’m stuck. It seems to me this is a pretty common scenario. I am obviously not understanding the breeze framework well enough to implement this. If you can point me in the right directions I’d appreciate your help.
UPDATE 4/9/2014
I'm thinking i could eliminate this issue altogether if i switch over to guid id and generate them client side. Is this correct thinking?
What's interesting here is that Breeze thinks that Notify.Id and Notify.AgencyId are multi part primary keys but they are actually not. Id is a PK and AgencyId is an FK. The only thing that I can think of is try removing the EntityTypeConfiguration for both Agency and Notify, specifically the part where it specifies HasKey and HasForeignKey. This Fluent API configuration shouldn't be required as EF will match your configuration by convention instead.
I took a different approach on working around my issue. Since i have the luxury to change out the id types, i swapped out the bigint ids to Uuid types and removed the auto generation of the ids in sql. Now i'm just creating my own ids using breeze.core.getUuid() when a new record is created. Not sure this is the most efficient way to work around the issue, but it seems to be working fine.
After I read one question in attached link, I got a sense of how to set DateCreated and DateModified columns in Entity Framework and use it in my application. In the old SQL way though, the trigger way is more popular because is more secure from DBA point of view.
So any advice on which way is the best practice? should it be set in entity framework for the purpose of application integrity? or should use trigger as it make more sense from data security point of view? Or is there a way to compose trigger in entity framework? Thanks.
EF CodeFirst: Rails-style created and modified columns
BTW, even though it doesn't matter much, I am building this app using ASP.NET MVC C#.
Opinion: Triggers are like hidden behaviour, unless you go looking for them you usually won't realise they are there. I also like to keep the DB as 'dumb' as possible when using EF, since I'm using EF so my team wont need to maintain SQL code.
For my solution (mix of ASP.NET WebForms and MVC in C# with Business Logic in another project that also contains the DataContext):
I recently had a similar issue, and although for my situation it was more complex (DatabaseFirst, so required a custom TT file), the solution is mostly the same.
I created an interface:
public interface ITrackableEntity
{
DateTime CreatedDateTime { get; set; }
int CreatedUserID { get; set; }
DateTime ModifiedDateTime { get; set; }
int ModifiedUserID { get; set; }
}
Then I just implemented that interface on any entities I needed to (because my solution was DatabaseFirst, I updated the TT file to check if the table had those four columns, and if so added the interface to the output).
UPDATE: here's my changes to the TT file, where I updated the EntityClassOpening() method:
public string EntityClassOpening(EntityType entity)
{
var trackableEntityPropNames = new string[] { "CreatedUserID", "CreatedDateTime", "ModifiedUserID", "ModifiedDateTime" };
var propNames = entity.Properties.Select(p => p.Name);
var isTrackable = trackableEntityPropNames.All(s => propNames.Contains(s));
var inherits = new List<string>();
if (!String.IsNullOrEmpty(_typeMapper.GetTypeName(entity.BaseType)))
{
inherits.Add(_typeMapper.GetTypeName(entity.BaseType));
}
if (isTrackable)
{
inherits.Add("ITrackableEntity");
}
return string.Format(
CultureInfo.InvariantCulture,
"{0} {1}partial class {2}{3}",
Accessibility.ForType(entity),
_code.SpaceAfter(_code.AbstractOption(entity)),
_code.Escape(entity),
_code.StringBefore(" : ", String.Join(", ", inherits)));
}
The only thing left was to add the following to my partial DataContext class:
public override int SaveChanges()
{
// fix trackable entities
var trackables = ChangeTracker.Entries<ITrackableEntity>();
if (trackables != null)
{
// added
foreach (var item in trackables.Where(t => t.State == EntityState.Added))
{
item.Entity.CreatedDateTime = System.DateTime.Now;
item.Entity.CreatedUserID = _userID;
item.Entity.ModifiedDateTime = System.DateTime.Now;
item.Entity.ModifiedUserID = _userID;
}
// modified
foreach (var item in trackables.Where(t => t.State == EntityState.Modified))
{
item.Entity.ModifiedDateTime = System.DateTime.Now;
item.Entity.ModifiedUserID = _userID;
}
}
return base.SaveChanges();
}
Note that I saved the current user ID in a private field on the DataContext class each time I created it.
As for DateCreated, I would just add a default constraint on that column set to SYSDATETIME() that takes effect when inserting a new row into the table.
For DateModified, personally, I would probably use triggers on those tables.
In my opinion, the trigger approach:
makes it easier; I don't have to worry about and remember every time I save an entity to set that DateModified
makes it "safer" in that it will also apply the DateModified if someone finds a way around my application to modify data in the database directly (using e.g. Access or Excel or something).
Entity Framework 6 has interceptors which can be used to set created and modified. I wrote an article how to do it: http://marisks.net/2016/02/27/entity-framework-soft-delete-and-automatic-created-modified-dates/
I agree with marc_s - much safer to have the trigger(s) in the database. In my company's databases, I require each field to have a Date_Modified, Date_Created field, and I even have a utility function to automatically create the necessary triggers.
When using with Entity Framework, I found I needed to use the [DatabaseGenerated] annotation with my POCO classes:
[Column(TypeName = "datetime2")]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
public DateTime? Date_Modified { get; set; }
[Column(TypeName = "datetime2")]
[DatabaseGenerated(DatabaseGeneratedOption.Computed)]
public DateTime? Date_Created { get; set; }
I was attempting to use stored procedure mapping on an entity, and EF was creating #Date_Modified, #Date_Created parameters on my insert/update sprocs getting the error
Procedure or function has too many arguments specified.
Most of the examples show using [NotMapped], which will allow select/insert to work, but then those fields will not show up when that entity is loaded!
Alternately you can just make sure any sprocs contain the #Date_Modified, #Date_Created parameters, but this goes against the design of using triggers in the first place.