I know the reason for the exception (SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM.) is a non nullable DateTime field in a Entity and so Nhibernate wants to save a smaller DateTime value than MSSQL accepts.
The Problem ist that there are far to many entities in the project to find the right DateTime field.
The exception occurs after an SaveOrUpdate() but is not triggered by the entity i want to save but any other entity which was loaded in the current session and now is affected by the flush().
How can i find out which field really is responsible for the exception?
If you cast the exception to a SqlTypeException, that will expose the Data collection. Normally there is a single Key and a single Value in the collection. The value is the SQL that was attempted to be executed. By examining the DML you can then see what table was being acted upon. Hopefully that table is narrow enough to make determining the offending column trivial.
Here's some simple code I use to spit out the Key and Value of the exception.
catch (SqlTypeException e)
{
foreach(var key in e.Data.Keys)
{
System.Console.Write("Key is " + key.ToString());
}
foreach(var value in e.Data.Values)
{
Console.WriteLine("Value is "+value.ToString());
}
}
Have you tried forcing NHib to output the generated sql and reviewing that for the rogue DateTime? It'd be easier if you were using something like NHProfiler (I don't work for them, just a satisfied customer), but really all that's doing for you is showing/isolating the sql anyway, which you can do from the output window with a little extra effort. The trick will be if it's a really deep save, then there could potentially be a lot of sql to read through, but chances are you'll be able to spot it pretty quickly.
You can create a class that implements both IPreUpdateEventListener and IPreInsertEventListener as follows:
public class InsertUpdateListener : IPreInsertEventListener, IPreUpdateEventListener {
public bool OnPreInsert(PreInsertEvent #event) {
CheckDateTimeWithinSqlRange(#event.Persister, #event.State);
return false;
}
public bool OnPreUpdate(PreUpdateEvent #event) {
CheckDateTimeWithinSqlRange(#event.Persister, #event.State);
return false;
}
private static void CheckDateTimeWithinSqlRange(IEntityPersister persister, IReadOnlyList<object> state) {
var rgnMin = System.Data.SqlTypes.SqlDateTime.MinValue.Value;
// There is a small but relevant difference between DateTime.MaxValue and SqlDateTime.MaxValue.
// DateTime.MaxValue is bigger than SqlDateTime.MaxValue but still within the valid range of
// values for SQL Server. Therefore we test against DateTime.MaxValue and not against
// SqlDateTime.MaxValue. [Manfred, 04jul2017]
//var rgnMax = System.Data.SqlTypes.SqlDateTime.MaxValue.Value;
var rgnMax = DateTime.MaxValue;
for (var i = 0; i < state.Count; i++) {
if (state[i] != null
&& state[i] is DateTime) {
var value = (DateTime)state[i];
if (value < rgnMin /*|| value > rgnMax*/) { // we don't check max as SQL Server is happy with DateTime.MaxValue [Manfred, 04jul2017]
throw new ArgumentOutOfRangeException(persister.PropertyNames[i], value,
$"Property '{persister.PropertyNames[i]}' for class '{persister.EntityName}' must be between {rgnMin:s} and {rgnMax:s} but was {value:s}");
}
}
}
}
}
You also need to then register this event handler when you configure the session factory. Add an instance to Configuration.EventListeners.PreUpdateEventListeners and to Configuration.EventListeners.PreInsertEventListeners and then use the Configuration object when creating NHibernate's session factory.
What this does is this: Every time NHibernate inserts or updates an entity it will call OnPreInsert() or OnPreUpdate() respectively. Each of these methods in turn calls CheckDateTimeWithinSqlRange().
CheckDateTimeWithinSqlRange() iterates over all property values of the entity, ie the object, that is being saved. If the property value is not null it then checks if it is of type DateTime. If that is the case it checks that it is not less than SqlDateTime.MinValue.Value (note the additional .Value to avoid exceptions). There is no need to check against SqlDateTime.MaxValue.Value if you are using SQL Server 2012 or later. They will happily accept even DateTime.MaxValue which is a few time ticks greater than SqlDateTime.MaxValue.Value.
If the value is outside of the allowed range this code will then throw an ArgumentOutOfRangeException with an appropriate message that includes the names of the class (entity) and property causing the problem as well as the actual value that was passed in. The message is similar to the equivalent SqlServerException for the SqlDateTime overflow exception but will make it easier to pinpoint the problem.
A couple of things to consider. Obviously this does not come for free. You will incur a runtime overhead as this logic consumes CPU. Depending on your scenario this may not be a problem. If it is, you can also consider optimizing the code given in this example to make it faster. One option could perhaps be to use caching to avoid the loop for the same class. Another option could be to use it only in test and development environments. For production you could then rely that the rest of the system operates correctly and the values will always be within valid range.
Also, be aware that this code introduces a dependency on SQL Server. NHibernate is typically used to avoid dependencies like this. Other database servers that are supported by NHibernate may have a different range of allowed values for datetime. Again, there are options for resolving this as well, e.g. by using different boundaries depending on SQL dialect.
Happy coding!
Related
Is there any way to define the SQL conversion component for additional functions to Linq2Entities.
For example:
myQuery.Where(entity => entity.Contains('foo', SearchFlags.All))
Ideally I am looking for something that doesn't require editing and building a new version the EntityFramework.dll directly. Is there any way to allow extension methods to entity framework that can support SQL generation.
So far I have a template which would represent the method I need to replace for LINQ to Entities:
public static bool Contains(this object source, string searchTerms, SearchFlags flags)
{
return true;
}
Of course this causes the error:
LINQ to Entities does not recognize the method 'Boolean
CONTAINS(System.Object, System.String, SearchFlags)' method, and this method
cannot be translated into a store expression.
To be clear, I don't want to do:
myQuery.AsEnumerable().Where(entity => entity.Contains('foo', SearchFlags.All))
Because I want to be able to execute code in SQL space and not return all the entities manually.
I also cannot use the .ToString() of the IQueryable and execute it manually because I need Entity Framework to populate the objects from several .Include joins.
I don't understand your Q clearly. However if your problem is that you can't use your own methods or other linq to objects method, just use .AsEnumerable() and do your other jobs through linq to objects, not L2E:
myQuery.AsEnumerable().Where(entity => entity.Contains('foo', SearchFlags.All))
And if you need to use your myQuery several times somewhere else, first load it to memory, then use it as many as you want:
var myQuery = from e in context.myEntities
select d;
myQuery.Load();
// ...
var myOtherQuery = from d in context.myEntities.Local
select d;
// Now any L2O method is supported...
I ended up doing the following (which works but is very far from perfect):
All my entities inherit from an IEntity which defines long Id { get; set; }
I then added a redundant restriction
context.myEntities.Where(entity => entity.Id != 0) this is
redundant since the identity starts at 1, but Linq2Entities doesn't
know that.
I then call .ToString() on the IQueryable after I have done all
my other queries, since it is of type DBQuery<Entity> it returns
the SQL Command Text, I do a simple replace with my query restriction.
In order to get all the .Include(...) to work I actually execute
two different sql commands. There is no other more pretty way to tap into this because of query execution plan caching causes issues otherwise (even when disabled).
As a result my code looks like this:
public IQueryable<IEntity> MyNewFunction(IQueryable<IEntity> myQueryable, string queryRestriction)
{
string rawSQL = myQueryable.Select(entity => entity.Id).ToString().Replace("[Extent1].Id <> 0", queryRestriction);
List<long> ids = // now execute rawSQL, get the list of ids;
return myQuerable.Where(entity => ids.Contains(entity.Id));
}
In short, other than manually executing the SQL or running a similar SQL command and appending the restriction using the existing commands the only way to write your own methods to Linq-to-Entities is to manually alter and build your own EntityFramework.dll from the EF6 source.
Is it ever a good idea to work directly with the context? For example, say I have a database of customers and a user can search them by name, display a list, choose one, then edit that customer's properties.
It seems I should use the context to get a list of customers (mapped to POCOs or CustomerViewModels) and then immediately close the context. Then, when the user selects one of the CustomerViewModels in the list the customer properties section of the UI populates.
Next they can change the name, type, website address, company size, etc. Upon hitting a save button, I then open a new context, use the ID from the CustomerViewModel to retrieve that customer record, and update each of its properties. Finally, I call SaveChanges() and close the context. This is a LOT OF WORK.
My question is why not just work directly with the context leaving it open throughout? I have read using the same context with a long lifetime scope is very bad and will inevitably cause problems. My assumption is if the application will only be used by ONE person I can leave the context open and do everything. However, if there will be many users, I want to maintain a concise unit of work and thus open and close the context on a per request basis.
Any suggestions? Thanks.
#PGallagher - Thanks for the thorough answer.
#Brice - your input is helpful as well
However, #Manos D. the 'epitome of redundant code' comment concerns me a bit. Let me go through an example. Lets say I'm storing customers in a database and one of my customer properties is CommunicationMethod.
[Flags]
public enum CommunicationMethod
{
None = 0,
Print = 1,
Email = 2,
Fax = 4
}
The UI for my manage customers page in WPF will contain three check boxes under the customer communication method (Print, Email, Fax). I can't bind each checkbox to that enum, it doesn't make sense. Also, what if the user clicked that customer, gets up and goes to lunch... the context sits there for hours which is bad. Instead, this is my thought process.
End user chooses a customer from the list. I new up a context, find that customer and return a CustomerViewModel, then the context is closed (I've left repositories out for simplicity here).
using(MyContext ctx = new MyContext())
{
CurrentCustomerVM = new CustomerViewModel(ctx.Customers.Find(customerId));
}
Now the user can check/uncheck the Print, Email, Fax buttons as they are bound to three bool properties in the CustomerViewModel, which also has a Save() method. Here goes.
public class CustomerViewModel : ViewModelBase
{
Customer _customer;
public CustomerViewModel(Customer customer)
{
_customer = customer;
}
public bool CommunicateViaEmail
{
get { return _customer.CommunicationMethod.HasFlag(CommunicationMethod.Email); }
set
{
if (value == _customer.CommunicationMethod.HasFlag(CommunicationMethod.Email)) return;
if (value)
_customer.CommunicationMethod |= CommunicationMethod.Email;
else
_customer.CommunicationMethod &= ~CommunicationMethod.Email;
}
}
public bool CommunicateViaFax
{
get { return _customer.CommunicationMethod.HasFlag(CommunicationMethod.Fax); }
set
{
if (value == _customer.CommunicationMethod.HasFlag(CommunicationMethod.Fax)) return;
if (value)
_customer.CommunicationMethod |= CommunicationMethod.Fax;
else
_customer.CommunicationMethod &= ~CommunicationMethod.Fax;
}
}
public bool CommunicateViaPrint
{
get { return _customer.CommunicateViaPrint.HasFlag(CommunicationMethod.Print); }
set
{
if (value == _customer.CommunicateViaPrint.HasFlag(CommunicationMethod.Print)) return;
if (value)
_customer.CommunicateViaPrint |= CommunicationMethod.Print;
else
_customer.CommunicateViaPrint &= ~CommunicationMethod.Print;
}
}
public void Save()
{
using (MyContext ctx = new MyContext())
{
var toUpdate = ctx.Customers.Find(_customer.Id);
toUpdate.CommunicateViaEmail = _customer.CommunicateViaEmail;
toUpdate.CommunicateViaFax = _customer.CommunicateViaFax;
toUpdate.CommunicateViaPrint = _customer.CommunicateViaPrint;
ctx.SaveChanges();
}
}
}
Do you see anything wrong with this?
It is OK to use a long-running context; you just need to be aware of the implications.
A context represents a unit of work. Whenever you call SaveChanges, all the pending changes to the entities being tracked will be saved to the database. Because of this, you'll need to scope each context to what makes sense. For example, if you have a tab to manage customers and another to manage products, you might use one context for each so that when a users clicks save on the customer tab, all of the changes they made to products are not also saved.
Having a lot of entities tracked by a context could also slow down DetectChanges. One way to mitigate this is by using change tracking proxies.
Since the time between loading an entity and saving that entity could be quite long, the chance of hitting an optimistic concurrency exception is greater than with short-lived contexts. These exceptions occur when an entity is changed externally between loading and saving it. Handling these exceptions is pretty straightforward, but it's still something to be aware of.
One cool thing you can do with long-lived contexts in WPF is bind to the DbSet.Local property (e.g. context.Customers.Local). this is an ObservableCollection that contains all of the tracked entities that are not marked for deletion.
Hopefully this gives you a bit more information to help you decide which approach to help.
Microsoft Reference:
http://msdn.microsoft.com/en-gb/library/cc853327.aspx
They say;
Limit the scope of the ObjectContext
In most cases, you should create
an ObjectContext instance within a using statement (Using…End Using in
Visual Basic).
This can increase performance by ensuring that the
resources associated with the object context are disposed
automatically when the code exits the statement block.
However, when
controls are bound to objects managed by the object context, the
ObjectContext instance should be maintained as long as the binding is
needed and disposed of manually.
For more information, see Managing Resources in Object Services (Entity Framework). http://msdn.microsoft.com/en-gb/library/bb896325.aspx
Which says;
In a long-running object context, you must ensure that the context is
disposed when it is no longer required.
StackOverflow Reference:
This StackOverflow question also has some useful answers...
Entity Framework Best Practices In Business Logic?
Where a few have suggested that you promote your context to a higher level and reference it from here, thus keeping only one single Context.
My ten pence worth:
Wrapping the Context in a Using Statement, allows the Garbage Collector to clean up the resources, and prevents memory leaks.
Obviously in simple apps, this isn't much of a problem, however, if you have multiple screens, all using alot of data, you could end up in trouble, unless you are certain to Dispose your Context correctly.
Hence I have employed a similar method to the one you have mentioned, where I've added an AddOrUpdate Method to each of my Repositories, where I pass in my New or Modified Entity, and Update or Add it depending upon whether it exists.
Updating Entity Properties:
Regarding updating properties however, I've used a simple function which uses reflection to copy all the properties from one Entity to Another;
Public Shared Function CopyProperties(Of sourceType As {Class, New}, targetType As {Class, New})(ByVal source As sourceType, ByVal target As targetType) As targetType
Dim sourceProperties() As PropertyInfo = source.GetType().GetProperties()
Dim targetProperties() As PropertyInfo = GetType(targetType).GetProperties()
For Each sourceProp As PropertyInfo In sourceProperties
For Each targetProp As PropertyInfo In targetProperties
If sourceProp.Name <> targetProp.Name Then Continue For
' Only try to set property when able to read the source and write the target
'
' *** Note: We are checking for Entity Types by Checking for the PropertyType to Start with either a Collection or a Member of the Context Namespace!
'
If sourceProp.CanRead And _
targetProp.CanWrite Then
' We want to leave System types alone
If sourceProp.PropertyType.FullName.StartsWith("System.Collections") Or (sourceProp.PropertyType.IsClass And _
sourceProp.PropertyType.FullName.StartsWith("System.Collections")) Or sourceProp.PropertyType.FullName.StartsWith("MyContextNameSpace.") Then
'
' Do Not Store
'
Else
Try
targetProp.SetValue(target, sourceProp.GetValue(source, Nothing), Nothing)
Catch ex As Exception
End Try
End If
End If
Exit For
Next
Next
Return target
End Function
Where I do something like;
dbColour = Classes.clsHelpers.CopyProperties(Of Colour, Colour)(RecordToSave, dbColour)
This reduces the amount of code I need to write for each Repository of course!
The context is not permanently connected to the database. It is essentially an in-memory cache of records you have loaded from disk. It will only request records from the database when you request a record it has not previously loaded, if you force it to refresh or when you're saving your changes back to disk.
Opening a context, grabbing a record, closing the context and then copying modified properties to an object from a brand new context is the epitomy of redundant code. You are supposed to leave the original context alone and use that to do SaveChanges().
If you're looking to deal with concurrency issues you should do a google search about "handling concurrency" for your version of entity framework.
As an example I have found this.
Edit in response to comment:
So from what I understand you need a subset of the columns of a record to be overridden with new values while the rest is unaffected? If so, yes, you'll need to manually update these few columns on a "new" object.
I was under the impression that you were talking about a form that reflects all the fields of the customer object and is meant to provide edit access to the entire customer record. In this case there's no point to using a new context and painstakingly copying all properties one by one, because the end result (all data overridden with form values regardless of age) will be the same.
This should be simple, but I haven't found a way yet...
I have a legacy database with name fields that are stored as CHAR(50). When this is bound to a TextBox with a Max Length of 50, you cannot insert.
How can I make the EF trim these values or at least map to RTrim(Column)?
I've tried using value converters, but the round trip causes issues with back spacing and spaces getting deleted between words.
Note that I only want to trim some fields, not all.
We are using SQL Server 2000 as the database. Soon to move to SQL 2008.
Thanks!
Entity framework is able to map only to table directly. You can also map to view or custom DB query but in such case your entity will became readonly unless you also map Insert, Delete and Update operations to stored procedures.
I think the problem you describes is related to ANSI PADDING behavior. It can be turned on but:
It is not recommended. In future version of SQL server it will be considered as error.
Must be configured before you create a column
You must handle trimming in the application. You can for example modify T4 template (if you use them) to trim string properties. Not sure how it works with WPF but you probably can inherit text box and override Text property to trim values.
Another way is handling ObjectMaterialized event on ObjectContext and manually trimming text properties but it can slow down your execution of your queries.
There's no way to do this with EF and SQL Server that I have found. I solved it with an extension method on IEnumerable<T> that calls TrimEnd() on each string property:
public static IEnumerable<TEntity> Trim<TEntity>(this IEnumerable<TEntity> collection)
{
Type type = typeof(TEntity);
IEnumerable<PropertyDescriptor> properties = TypeDescriptor.GetProperties(type).Cast<PropertyDescriptor>()
.Where(p => p.PropertyType == typeof(string));
foreach (TEntity entity in collection)
{
foreach (PropertyDescriptor property in properties)
{
string value = (string) property.GetValue(entity);
if (!String.IsNullOrEmpty(value))
{
value = value.TrimEnd();
property.SetValue(entity, value);
}
}
}
return collection;
}
Just make sure you call it after EF has retrieved the entities from the database. For example, after ToList():
public IEnumerable<Country> FetchCountries()
{
return _context.Set<Country>().ToList().Trim();
}
Have a look at the available attributes for your Database Connection String. I had a similar issue with Sybase Advantage database and solved with it's TrimTrailingSpaces attribute. Your database may support something similar.
Data Source=\\serverx\volumex\path\db.add;User ID=user;Password=pass;ServerType=REMOTE;TrimTrailingSpaces=TRUE;
http://www.connectionstrings.com/
Time and again I find myself creating a database utility class which has multiple functions which all do almost the same thing but treat the result set slightly differently.
For example, consider a Java class which has many functions which all look like this:
public void doSomeDatabaseOperation() {
Connection con = DriverManager.getConnection("jdbc:mydriver", "user", "pass");
try {
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("SELECT whatever FROM table"); // query will be different each time
while (rs.next()) {
// handle result set - differently each time
}
} catch (Exception e) {
// handle
} finally {
con.close();
}
}
Now imagine a class with 20 of these functions.
As you can see, tons of boilerplate (opening a connection, try-finally block), and the only thing that changes would be the query and the way you handle the result set. This type of code occurs in many languages (considering you're not using an ORM).
How do you manage your DB utility classes so as to reduce code duplication? What does a typical DB utility class look like in your language/framework?
The way I have done in one of my project is that I followed what Spring does with JDBC template and came up with a Query framework. Basically create a common class which can take select statement or pl/sql calls and bind parameters. If the query returns resultset, also pass the Rowmapper. This rowmapper object will be called by the framework to convert each row into an object of any kind.
Example -
Query execute = new Query("{any select or pl/sql}",
// Inputs and Outputs are for bind variables.
new SQL.Inputs(Integer.class, ...),
// Outputs is only meaningful for PL/SQL since the
// ResultSetMetaData should be used to obtain queried columns.
new SQL.Outputs(String.class));
If you want the rowmapper -
Query execute = new Query("{any select or pl/sql}",
// Inputs and Outputs are for bind variables.
new SQL.Inputs(Integer.class, ...),
// Outputs is only meaningful for PL/SQL since the
// ResultSetMetaData should be used to obtain queried columns.
new SQL.Outputs(String.class), new RowMapper() {
public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
Actor actor = new Actor();
actor.setFirstName(rs.getString("first_name"));
actor.setSurname(rs.getString("surname"));
return actor;
});
Finally a Row class is the output which will have list of objects if you have passed the RowMapper -
for (Row r : execute.query(conn, id)) {
// Handle the rows
}
You can go fancy and use Templates so that type safety is guaranteed.
Sounds like you could make use of a Template Method pattern here. That would allow you to define the common steps (and default implementations of them, where applicable) that all subclasses will take to perform the action. Then subclasses need only override the steps which differ: SQL query, DB-field-to-object-field mapping, etc.
When using .net, the Data Access Application Block is in fairly widespread use to provide support for the following:
The [data access] application block
was designed to achieve the following
goals:
Encapsulate the logic used to perform
the most common data access tasks.
Eliminate common coding errors, such
as failing to close connections.
Relieve developers of the need to
write duplicated code for common data
access tasks.
Reduce the need for
custom code.
Incorporate best
practices for data access, as
described in the .NET Data Access
Architecture Guide.
Ensure that, as
far as possible, the application block
functions work with different types of
databases.
Ensure that applications
written for one type of database are,
in terms of data access, the same as
applications written for another type
of database.
There are plenty of examples and tutorials of usage too: a google search will find msdn.microsoft, 4guysfromrolla.com, codersource.com and others.
SQLServer int field. Value sometimes null.
DataAdapter fills dataset OK and can display data in DatagridView OK.
When trying to retrieve the data programmatically from the dataset the Dataset field retrieval code throws a StronglyTypedException error.
[global::System.Diagnostics.DebuggerNonUserCodeAttribute()]
public int curr_reading {
get {
try {
return ((int)(this[this.tableHistory.curr_readingColumn]));
}
catch (global::System.InvalidCastException e) {
throw new global::System.Data.StrongTypingException("The value for column \'curr_reading\' in table \'History\' is DBNull.", e);
}
Got past this by checking for DBNull in the get accessor and returning null but...
When the dataset structure is modified (Still developing) my changes (unsurprisingly) are gone.
What is the best way to handle this situation?
It seems I am stuck with dealing with it at the dataset level.
Is there some sort of attribute that can tell the auto code generator to leave the changes in place?
In the typed dataset designer there is the nullvalue property.
By default its value is throw exception (hence your generated code)
You can set it to the desired default value.. ie. 0.
Then it will return 0 instead of an exception. (other code is generated)
VS2008 :This works directly in the dataset designer.
VS2005 : It only works for strings in the designer but you can directly edit the XSD an set the property msprop:nullValue="0"
Leave the auto-generated code alone. There's no way to "intercept" it getting generated so any changes you do make are guaranteed to get blown away sooner or later.
.NET (well at least the .NET 2.0 system.data bits) will not convert from DBNull into anything else. This sucks but you can't do anything about it.
Write an extension method called ToNullable() or similar: it can do this:
.
public static Nullable<T> ToNullable(this object x){
if(x == DBNull.Value)
return default(T); // return null thing
else
return (T)x;
}
then you can do
int? thing = DataRow["column"].ToNullable<int>();
The dataset will have a boolean property to indicate null.
int curr_reading = ( Iscurr_readingColumnNull) ?
<default_value> : row.curr_readingColumn;
If memory serves, you need to mark the row as being edited - using .BeginEdit() or similar - then make your edits and save the row, probably using .EndEdit() or similar. You may want to do a little bit of reading into these methods (they may be on the DataSet, DataTable or DataRow) - my memory is a little hazy.
Hope this helps at least a little bit.
if(row["curr_reading"] is DBNull){
}else{
row.curr_reading;
}