I have a db table with several DateTime fields with null values. These are mapped to nullable DateTimes in my class.
If I try to perform an update with Dapper, within my data layer:
using (IDbConnection cnn = new SqlConnection(DB.getConString()))
{
cnn.Open();
return cnn.Execute((this.OptionID == 0 ? _insertSQL : _updateSQL), this);
}
I get a SqlDateTimeOverflow exception (because the DateTime field is '01/01/0001 00:00:00' rather than null.
Is the only way around this to specify each parameter individually and switch the value to null like this:
using (IDbConnection cnn = new SqlConnection(DB.getConString()))
{
cnn.Open();
return cnn.Execute("UPDATE MyTable SET MyDateField = #MyDateField", new {MyDateField = (MyDateField.HasValue? MyDateField : Null), etc etc... );
I have about 50 fields in the table so this would be quite a bit of code, plus there is an INSERT method to update similarly too. Is there an easier syntax I am missing?
The issue here is that 01/01/0001 00:00:00 is not a "null value"; if you had used DateTime? I suspect it would have worked fine. However, I also have to recognize that DateTime.MinValue has often (mainly due to .NET 1.1 lacking nullable structs) been used to represent a null value. My preferred suggestion here would be to simply use DateTime?. The min-value map is a bit complicated as we might also consider whether that should be automatically mapped instead to the sql min-value (January 1, 1753).
Re the update statement - maybe add an extension method to map between min-value and null?
public static DateTime? NullIfZero(this DateTime when) {
return when == DateTime.MinValue ? (DateTime?)null : when;
}
and use:
new { MyDateField = MyDateField.NullIfZero() }
but again, if MyDateField was DateTime?, you could just use:
new { MyDateField }
Related
I've declared a nullable DateTime? NextUpdateproperty in my model and database.
I can update the DateTime value fine on my DB as it allows null for this field.
But when I try to get the value of NextUpdate field from the database using SQL Data Reader it bombs out because the value of NextUpdate is null.
I did try to init the NextUpdate value if it is null using the following assignment but the error is still thrown telling me that field is null:
NextUpdate = dataReader.GetDateTime(dataReader.GetOrdinal("NextUpdate")) != null ? dataReader.GetDateTime(dataReader.GetOrdinal("NextUpdate")) : DateTime.MinValue,
Error:
Data is Null. This method or property cannot be called on Null values - at System.Data.SqlClient.SqlBuffer.get_DateTime()
Question:
Is there a short method of reading back and initializing a nullable **DateTime?** value?
Code:
using (IDataReader dataReader = db.ExecuteReader(dbCommand))
{
if (dataReader.Read())
{
esc = new Escalation
{
NextUpdate = dataReader.GetDateTime(dataReader.GetOrdinal("NextUpdate")),
RootCause = dataReader["RootCause"] != null ? dataReader["EM"].ToString() : ""
};
}
}
Property in Model:
public DateTime? NextUpdate { get; set; }
Compare the value against DBNull.Value instead of null:
NextUpdate = dataReader["NextUpdate"].Equals(DBNull.Value) ? (DateTime?)null : (DateTime?)dataReader["NextUpdate"]
(assuming that the NextUpdate member is a DateTime?)
Try following expression:
RootCause = (dataReader["RootCause"] is DBNull) ? (DateTime?)null : (DateTime?)dataReader["RootCause"] ;
I'm using QueryMultiple which returns a GridReader.
Since I don't know how much data I'm gonna read, I'm looping over the reader with the stop condition of IsConsumed:
using (var reader = conn.QueryMultiple(mySql)) {
while(!reader.IsConsumed) {
reader.Read<...>
}
}
However, I'm always getting an ObjectDisposedException on the last read. The value of IsConsumed is still false.
I've tried to pass DynamicParameters to the query with the intention of getting a callback (which seems to be useful via IParameterCallbacks), but I couldn't patch it together.
I would really rather not to have such an expected exception in the code. Thanks for any help.
I'm using SQL Server, my provider is System.Data.SqlClient in .NET 4.5, Dapper version 1.40.0.0
A failing test for example:
[TestMethod]
public void QueryMultipleWithCursor()
{
const string sql = #"
DECLARE #CurrentDate DATE
DECLARE DatesCursor CURSOR LOCAL FOR
SELECT DISTINCT DataDate FROM Data_Table ORDER BY DataDate
OPEN DatesCursor
FETCH NEXT FROM DatesCursor INTO #CurrentDate
WHILE ##FETCH_STATUS = 0 BEGIN
SELECT DISTINCT
DataDate AS Date1,
DataDate AS Date2
FROM Data_Table
WHERE DataDate=#CurrentDate
FETCH NEXT FROM DatesCursor INTO #CurrentDate
END
CLOSE DatesCursor
DEALLOCATE DatesCursor";
using (var conn = _database.GetConnection())
{
var reader = conn.QueryMultiple(sql);
while (!reader.IsConsumed)
{
reader.Read<DateTime, DateTime, DateTime>(
(date1, date2) => date1,
splitOn: "Date2").ToList();
}
}
}
I'm getting a NullReferenceException with the following stack:
at Dapper.SqlMapper.GridReader.NextResult() in D:\Dev\dapper-dot-net\Dapper NET40\SqlMapper.cs:line 4440
at Dapper.SqlMapper.GridReader.<MultiReadInternal>d__9`8.System.IDisposable.Dispose()
at Dapper.SqlMapper.GridReader.<MultiReadInternal>d__9`8.MoveNext() in D:\Dev\dapper-dot-net\Dapper NET40\SqlMapper.cs:line 4309
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Dapper.SqlMapper.GridReader.Read[TFirst,TSecond,TReturn](Func`3 func, String splitOn, Boolean buffered) in D:\Dev\dapper-dot-net\Dapper NET40\SqlMapper.cs:line 4330
at Project.MyTests.QueryMultipleWithCursor() in C:\Project\MyTests.cs:line 171
Result Message:
Test method Project.MyTests.QueryMultipleWithCursor threw exception:
System.NullReferenceException: Object reference not set to an instance of an object.
I've pushed the following, which passes on SQL Server / SqlConnection; so it can work:
[Fact]
public void SO35554284_QueryMultipleUntilConsumed()
{
using (var reader = connection.QueryMultiple(
"select 1 as Id; select 2 as Id; select 3 as Id;"))
{
List<HazNameId> items = new List<HazNameId>();
while (!reader.IsConsumed)
{
items.AddRange(reader.Read<HazNameId>());
}
items.Count.IsEqualTo(3);
items[0].Id.IsEqualTo(1);
items[1].Id.IsEqualTo(2);
items[2].Id.IsEqualTo(3);
}
}
I wonder if the issue here is a problem with a specific ADO.NET provider. You might want to specify exactly:
what backend RDBMS / etc you are using (SQL Server? Oracle? Postgresql? ...?)
what ADO.NET provider you are using
what runtime (.NET what.what? core-clr?) / OS you are using
what exact library version you are using (the above is against the source code, which is most similar to 1.50.0-beta8)
I am running into the same issue with Dapper and I am using the version 1.42.0 and SQL Server 2012 as the back end. Upon debugging I found that this issue is happening only when we try to create multiple objects using the Dapper's splitOn option on the last result set.
I have submitted a new issue on GitHub
https://github.com/StackExchange/dapper-dot-net/issues/469
Well it seems to be an issue with Dapper implementation, for the mean time I'm using both Dapper and SqlDataReader, which is more reliable:
public static SqlMapper.GridReader QueryMultipleStoredProcedure(this IDbConnection dbConnection, string spName, object parameters, out SqlDataReader sqlDataReader)
{
var gridReader = dbConnection.QueryMultiple(spName, new DynamicParameters(parameters), commandType: CommandType.StoredProcedure);
sqlDataReader = typeof (SqlMapper.GridReader).GetInstanceField<SqlDataReader>(gridReader, "reader");
return gridReader;
}
private static T GetInstanceField<T>(this Type type, object instance, string fieldName)
{
var bindFlags = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;
var field = type.GetField(fieldName, bindFlags);
return (T) field?.GetValue(instance);
}
And then I can use sqlDataReader.HasRows
In my simplified example i have an object with following properties:
Name (string)
BirthDateTimeStamp (datetime)
I need to ba able to build dynamic queries in the following way
var predicate = "Name = #0";
var values = new object[]{"Ed"};
myIQueryableDataSource.Where(predicate, values)
This work good. Now i want to compare my datetime
var predicate = "BirthDateTimeStamp >= #0";
var values = new object[]{someDateTime};
This works good also. But what i actually want to do when comparing the datetimes and this issue shows itself better when doing the equals is comparing just on date.
var predicate = "BirthDateTimeStamp.Date >= #0";
This is not possible since the Date property is not recognized by EF to SQL server
var predicate = "System.Data.Entity.DbFunctions.TruncateTime(BirthDateTimeStamp) >= #0";
This is also not working since i can only access my object properties in the predicate.
How can i solve this in this way so the predicate stays in the string format. This code is just a part of a big existing parser for my queries and connat be completely rewritten.
See https://stackoverflow.com/a/26451213/525788
System.Linq.Dynamic is parsing the expression that you give as C# but does not recognize the class DbFunctions. However you can patch in DbFunctions as a predefined type:
var type = typeof( DynamicQueryable ).Assembly.GetType( "System.Linq.Dynamic.ExpressionParser" );
FieldInfo field = type.GetField( "predefinedTypes", BindingFlags.Static | BindingFlags.NonPublic );
Type[] predefinedTypes = (Type[])field.GetValue( null );
Array.Resize( ref predefinedTypes, predefinedTypes.Length + 1 );
predefinedTypes[ predefinedTypes.Length - 1 ] = typeof( DbFunctions );
field.SetValue( null, predefinedTypes );
Then you can use
var predicate = "DbFunctions.TruncateTime(BirthDateTimeStamp) >= #0";
Answer given by #RockResolve did work, but is kind a hack. Linq Dynamics provide functionality to add custom functions
public class CustomTypeProvider: IDynamicLinkCustomTypeProvider
{
public HashSet<Type> GetCustomTypes()
{
HashSet<Type> types = new HashSet<Type>();
// adding custom types
types.Add(typeof(DbFunctions));
return types;
}
}
// use below line to add this to linq
System.Linq.Dynamics.GlobalConfig.CustomTypeProvider = new CustomTypeProvier();
I have mvc application. In action I have Dictionary<string,int>. The Key is ID and Value is sortOrderNumber. I want to create stored procedure that will be get key(id) find this record in database and save orderNumber column by value from Dictionary. I want to call stored procedure once time and pass data to it, instead of calling many times for updating data.
Have you any ideas?
Thanks!
The accepted answer of using a TVP is generally correct, but needs some clarification based on the amount of data being passed in. Using a DataTable is fine (not to mention quick and easy) for smaller sets of data, but for larger sets it does not scale given that it duplicates the dataset by placing it in the DataTable simply for the means of passing it to SQL Server. So, for larger sets of data there is an option to stream the contents of any custom collection. The only real requirement is that you need to define the structure in terms of SqlDb types and iterate through the collection, both of which are fairly trivial steps.
A simplistic overview of the minimal structure is shown below, which is an adaptation of the answer I posted on How can I insert 10 million records in the shortest time possible?, which deals with importing data from a file and is hence slightly different as the data is not currently in memory. As you can see from the code below, this setup is not overly complicated yet highly flexible as well as efficient and scalable.
SQL object # 1: Define the structure
-- First: You need a User-Defined Table Type
CREATE TYPE dbo.IDsAndOrderNumbers AS TABLE
(
ID NVARCHAR(4000) NOT NULL,
SortOrderNumber INT NOT NULL
);
GO
SQL object # 2: Use the structure
-- Second: Use the UDTT as an input param to an import proc.
-- Hence "Tabled-Valued Parameter" (TVP)
CREATE PROCEDURE dbo.ImportData (
#ImportTable dbo.IDsAndOrderNumbers READONLY
)
AS
SET NOCOUNT ON;
-- maybe clear out the table first?
TRUNCATE TABLE SchemaName.TableName;
INSERT INTO SchemaName.TableName (ID, SortOrderNumber)
SELECT tmp.ID,
tmp.SortOrderNumber
FROM #ImportTable tmp;
-- OR --
some other T-SQL
-- optional return data
SELECT #NumUpdates AS [RowsUpdated],
#NumInserts AS [RowsInserted];
GO
C# code, Part 1: Define the iterator/sender
using System.Collections;
using System.Data;
using System.Data.SqlClient;
using System.IO;
using Microsoft.SqlServer.Server;
private static IEnumerable<SqlDataRecord> SendRows(Dictionary<string,int> RowData)
{
SqlMetaData[] _TvpSchema = new SqlMetaData[] {
new SqlMetaData("ID", SqlDbType.NVarChar, 4000),
new SqlMetaData("SortOrderNumber", SqlDbType.Int)
};
SqlDataRecord _DataRecord = new SqlDataRecord(_TvpSchema);
StreamReader _FileReader = null;
// read a row, send a row
foreach (KeyValuePair<string,int> _CurrentRow in RowData)
{
// You shouldn't need to call "_DataRecord = new SqlDataRecord" as
// SQL Server already received the row when "yield return" was called.
// Unlike BCP and BULK INSERT, you have the option here to create an
// object, do manipulation(s) / validation(s) on the object, then pass
// the object to the DB or discard via "continue" if invalid.
_DataRecord.SetString(0, _CurrentRow.ID);
_DataRecord.SetInt32(1, _CurrentRow.sortOrderNumber);
yield return _DataRecord;
}
}
C# code, Part 2: Use the iterator/sender
public static void LoadData(Dictionary<string,int> MyCollection)
{
SqlConnection _Connection = new SqlConnection("{connection string}");
SqlCommand _Command = new SqlCommand("ImportData", _Connection);
SqlDataReader _Reader = null; // only needed if getting data back from proc call
SqlParameter _TVParam = new SqlParameter();
_TVParam.ParameterName = "#ImportTable";
// _TVParam.TypeName = "IDsAndOrderNumbers"; //optional for CommandType.StoredProcedure
_TVParam.SqlDbType = SqlDbType.Structured;
_TVParam.Value = SendRows(MyCollection); // method return value is streamed data
_Command.Parameters.Add(_TVParam);
_Command.CommandType = CommandType.StoredProcedure;
try
{
_Connection.Open();
// Either send the data and move on with life:
_Command.ExecuteNonQuery();
// OR, to get data back from a SELECT or OUTPUT clause:
SqlDataReader _Reader = _Command.ExecuteReader();
{
Do something with _Reader: If using INSERT or MERGE in the Stored Proc, use an
OUTPUT clause to return INSERTED.[RowNum], INSERTED.[ID] (where [RowNum] is an
IDENTITY), then fill a new Dictionary<string, int>(ID, RowNumber) from
_Reader.GetString(0) and _Reader.GetInt32(1). Return that instead of void.
}
}
finally
{
_Reader.Dispose(); // optional; needed if getting data back from proc call
_Command.Dispose();
_Connection.Dispose();
}
}
Using Table Valued parameters is really not that complex.
given this SQL:
CREATE TYPE MyTableType as TABLE (ID nvarchar(25),OrderNumber int)
CREATE PROCEDURE MyTableProc (#myTable MyTableType READONLY)
AS
BEGIN
SELECT * from #myTable
END
this will show how relatively easy it is, it just selects out the values you sent in for demo purposes. I am sure you can easily abstract this away in your case.
using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
namespace TVPSample
{
class Program
{
static void Main(string[] args)
{
//setup some data
var dict = new Dictionary<string, int>();
for (int x = 0; x < 10; x++)
{
dict.Add(x.ToString(),x+100);
}
//convert to DataTable
var dt = ConvertToDataTable(dict);
using (SqlConnection conn = new SqlConnection("[Your Connection String here]"))
{
conn.Open();
using (SqlCommand comm = new SqlCommand("MyTableProc",conn))
{
comm.CommandType=CommandType.StoredProcedure;
var param = comm.Parameters.AddWithValue("myTable", dt);
//this is the most important part:
param.SqlDbType = SqlDbType.Structured;
var reader = comm.ExecuteReader(); //or NonQuery, etc.
while (reader.Read())
{
Console.WriteLine("{0} {1}", reader["ID"], reader["OrderNumber"]);
}
}
}
}
//I am sure there is a more elegant way of doing this.
private static DataTable ConvertToDataTable(Dictionary<string, int> dict)
{
var dt = new DataTable();
dt.Columns.Add("ID",typeof(string));
dt.Columns.Add("OrderNumber", typeof(Int32));
foreach (var pair in dict)
{
var row = dt.NewRow();
row["ID"] = pair.Key;
row["OrderNumber"] = pair.Value;
dt.Rows.Add(row);
}
return dt;
}
}
}
Produces
0 100
1 101
2 102
3 103
4 104
5 105
6 106
7 107
8 108
9 109
Stored procedures do not support arrays as inputs. Googling gives a couple of hacks using XML or comma separated strings, but those are hacks.
A more SQLish way to do this is to create a temporary table (named e.g. #Orders) and insert all the data into that one. Then you can call the sp, using the same open Sql Connection and insie the SP use the #Orders table to read the values.
Another solution is to use Table-Valued Parameters but that requires some more SQL to setup so I think it is probably easier to use the temp table approach.
Is there a way to specify that I want all of the DateTimes that OrmLite materializes to be set to UTC kind?
I store a lot of DateTimes in my database via stored procedures when a row is inserted:
insert [Comment] (
Body
, CreatedOn
) values (
#Body
, getutcdate()
);
When I retrieve the values via a select statement in ormlite, the datetimes come out in Unspecified kind (which is interpreted as the local timezone, I believe):
var comments = db.SqlList<Comment>("select * from [Comment] where ... ");
I would prefer not to set each DateTime object individually:
foreach (var comment in comments) {
comment.CreatedOn = DateTime.SpecifyKind(comment.CreatedOn, DateTimeKind.Utc);
}
I found this question, but I don't think it's quite what I'm asking for:
servicestack ormlite sqlite DateTime getting TimeZone adjustment on insert
Also found this pull request, but setting SqlServerOrmLiteDialectProvider.EnsureUtc(true) doesn't seem to do it either.
SqlServerOrmLiteDialectProvider.EnsureUtc(true) does work, there was something else going on with my test case that led me to believe that it didn't. Hopefully this will help someone else.
Here's some sample code:
model.cs
public class DateTimeTest {
[AutoIncrement]
public int Id { get; set; }
public DateTime CreatedOn { get; set; }
}
test.cs
var connectionString = "server=dblcl;database=flak;trusted_connection=true;";
var provider = new SqlServerOrmLiteDialectProvider();
provider.EnsureUtc(true);
var factory = new OrmLiteConnectionFactory(connectionString, provider);
var connection = factory.Open();
connection.CreateTable(true, typeof(DateTimeTest));
connection.ExecuteSql("insert DateTimeTest (CreatedOn) values (getutcdate())");
var results = connection.SqlList<DateTimeTest>("select * from DateTimeTest");
foreach(var result in results) {
Console.WriteLine("{0},{1},{2},{3},{4}", result.Id, result.CreatedOn, result.CreatedOn.Kind, result.CreatedOn.ToLocalTime(), result.CreatedOn.ToUniversalTime());
}
output
1,9/13/2013 5:19:12 PM,Utc,9/13/2013 10:19:12 AM,9/13/2013 5:19:12 PM