LINQ to SQL Data Context logs don't show the WHERE clause - sql-server

The following is the C# code and generated SQL in a LINQ to SQL query for two cases.
Case 1
using (JulianDataContext dc = new JulianDataContext(this.CurrentConnectionString))
{
#if DEBUG
dc.Log = new DebugTextWriter();
#endif
IEnumerable<UserNewsfeedDeliveryTime> temp = dc.UserNewsfeedDeliveryTimes.Where(u => u.NewsfeedEmailPeriodicity > 0 && DateTime.Today >= u.NextNewsfeedDelivery.Value.Date);
ids = temp.Select(p => p.Id).ToList();
}
SELECT [t0].[Id], [t0].[NewsfeedEmailPeriodicity], [t0].[LastSentNewsfeedEmail], [t0].[NextNewsfeedDelivery]
FROM [dbo].[UserNewsfeedDeliveryTimes] AS [t0]
WHERE ([t0].[NewsfeedEmailPeriodicity] > #p0) AND (#p1 >= CONVERT(DATE, [t0].[NextNewsfeedDelivery]))
-- #p0: Input Int (Size = -1; Prec = 0; Scale = 0) [0]
-- #p1: Input DateTime (Size = -1; Prec = 0; Scale = 0) [15-11-2012 00:00:00]
Case 2
using (JulianDataContext dc = new JulianDataContext(this.CurrentConnectionString))
{
#if DEBUG
dc.Log = new DebugTextWriter();
#endif
IEnumerable<UserNewsfeedDeliveryTime> temp = dc.GetTable<UserNewsfeedDeliveryTime>();
temp = temp.Where(u => u.NewsfeedEmailPeriodicity > 0 && DateTime.Today >= u.NextNewsfeedDelivery.Value.Date);
ids = temp.Select(p => p.Id).ToList();
}
SELECT [t0].[Id], [t0].[NewsfeedEmailPeriodicity], [t0].[LastSentNewsfeedEmail], [t0].[NextNewsfeedDelivery]
FROM [dbo].[UserNewsfeedDeliveryTimes] AS [t0]
The difference
The difference between these two linq queries:
dc.UserNewsfeedDeliveryTimes
and
dc.GetTable<UserNewsfeedDeliveryTime>()
Why? Could it be that, in case 2, LINQ to SQL is retrieving all data from database and finish the query by filtering all objects in memory?
If so, how can we make keep this generic and still force all the T-SQL to be generated?
Solution
Both answers, are correct but I had to pick one, sorry! I think also it is interesting to add that in this case, since I changed to work with an IQueryable (inherits from IEnumerable), in this line:
temp = temp.Where(u => u.NewsfeedEmailPeriodicity > 0 && DateTime.Today >= u.NextNewsfeedDelivery.Value.Date);
I had two overload methods, one from the IQueryable interface and another to the IEnumerable interface.
public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate);
public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate);
So I had to convert my predicate explicitly to Expression> predicate, otherwise the IEnumerable interface method would have been picked up at compile time and, if I am not mistaken, I would get some dynamic sql exception saying the T-SQL could not have been generated.

From my understanding, IEnumerable does not transform the original query information that IQueryable holds. It's almost as if the cast freezes any changes to the IQueryable query at the point of casting. If you look at MSDN, it turns out that IQueryable inherits IEnumerable:
http://msdn.microsoft.com/en-us/library/system.linq.iqueryable.aspx
Hence, you see this behaviour. It is important with LINQ-SQL to work with IQueryable unless you want the query frozen at the point it is turned to an IEnumerable.
In your first example, the where is inclusive of the original query. The select is not hence the query generated.
In your second example, you capture the table itself into an IEnumerable. Any changes on top of this are done in memory on top of the original query.
When you think, the IEnumerable version of where will not be able to transform the original data of the IQueryable due to the cast and how inheritance works.
When you also consider deferred loading, and how LINQ works, this seems to make sense.
To me it is a big annoyance, as it can lead you into generating some terrible performing code.

Try using IQueryable instead of IEnumerable.
Weird, because on my examples I get the opposite results from you, ie with IEnumerable, case 1 works fast and case 2 retrieves all the data. But using IQueryable fixes the issue.

Related

Empty TVP in Dapper

I have created a procedure that receives some primitives parameters and one TVP.
I have created a library that receives a List<T> and returns a List<SqlDataRecord> for the TVP (https://github.com/Kunze/EasyTVP). And it works, except when my TVP is empty/null.
When the tvp is null I get (becausa TVP cannot be null in sql server):
"Table Valued Parameter cannot be null"
But TVP works if you dont pass the parameter if it's null, so I tried to use a expando like this (the same error for DynamicParameters):
dynamic expando = new ExpandoObject();
...
if(model.Products != null && model.Products.Count> 0){
expando.Products = list_of_sql_data_record;
}
but the Query method of dapper throws this error (it works if it is a class, but not for dynamic):
{"The member of type Microsoft.SqlServer.Server.SqlDataRecord cannot be used as a parameter value"}
I can get this work if I create 2 classes, one for Products.Count > 0 and one different for Products = null or 0, but this is ridiculous, right?.
So, How can I make this works?
conexao.Query(#"Add", new
{
model.Name,
Products = list_of_sql_data_record //this may be empty
}, commandType: System.Data.CommandType.StoredProcedure);
I could do it works calling AsTableValuedParameter:
if(model.Products != null && model.Products.Count > 0){
dynamicParameters.Add("Products", list_of_sql_data_record.AsTableValueParameter());
}

Using contains to match two fields

I have a database table which has a bunch of fields including one called Type and another called Code. I also have a list of Type and Code pairs that are encapsulated in a class. Something like this:
public class TypeAndCode
{
public byte Type { get; set; }
public int Code { get; set; }
// overrides for Equals and GetHashCode to compare if Type and Code are equal
}
Now what I need to do is select from the table only those entries who type AND code match an entry in my collection. So, for example, I tried something like this:
var query = myTable.Where(a => myTCList.Contains(new TypeAndCode() { Type = a.Type, Code = a.Code }).ToList();
But it'll give me a NotSupportedException:
Unable to create a constant value of type 'TypeAndCode'. Only primitive types
or enumeration types are supported in this context.
Is there a way to make this work so that I can retrieve from the database only those entries that have a Code and Type that match my list of Code and Type? I'm trying to avoid having to retrieve all the entries (it's a big table) and match them in memory.
I'm aware that I could try something like
var query = myTable.Where(a => listOfTypes.Contains(a.Type) && listOfCodes.Contains(a.Codes))
But that will make some spurious matches where the type and code are from different pairs in my original list.
You can use Any instead:
var query = myTable
.Where(a => myTCList.Any(t => t.Type == a.Type && t.Code == a.Code ))
.ToList();
You should be able to just do this manually without the overloaded methods from your class:
myTCList.Any(x => x.Type == a.Type && x.Code == a.Code)
My ulitmate solution here, in case anybody else encounters a similar problem, was to set up a temporary table that I could write the pairs I wanted to match to which I could them join with the database table. After doing the join and materializing the results, you can delete the temporary table.
Something like:
ctx.myTempTable = (from pair in mypairs
select new myTempTable() { Type = pair.Type, Code = pair.Code }).ToList();
ctx.SaveChanges();
var query = from q in myTable
join t in ctx.myTempTable
on new { q.Type, q.Code } equals new { t.Code, t.Type }
select q;
The whole thing is in a try/catch/finally block with the finally block used to clear up the temporary table(s)

Caching of Function Results

I essentially want to write a bunch of commonly used queries in a web application of this format:
SELECT *
FROM secure_table
WHERE security_function(value 1, value 2) = true;
Value 1 and value 2 will have a limited enough range of values for the idea of caching the result of the security function to be potentially very useful in improving application performance. We would also need to be able to trigger a reset of the cache at will since some conditions would render the cached values out of date.
Is there an out of the box way of doing this with SQL Server (I believe we will be using the 2012 version)? I've had a google around and seen nothing concrete, some references to ASP.NET state but nothing concrete about what that actually involves, and some references to memcached, but that wouldn't seem to go down to function level, so doesn't seem suitable either.
EDIT:
So I would like the function to work something like this:
function security_function(val1, val2) {
result = getFromCache(val1, val2)
if result is empty then
result = //do big complicated query
addToCache(val1, val2, result)
end
return result
}
If you are using ASP.Net you can use the cache object to store the results of the query:
in c#:
Results GetResults(string value1, string value2)
{
string cacheItemName = "cacheItem-" + value1 + "-" + value2;
if (Cache[cacheItemName] != null)
{
return Cache[cacheItemName];
}
else
{
var result = // do big complicated query;
Cache.Insert(cacheItemName, result,
null, DateTime.Now.AddMinutes(15d), // Expire after 15 minutes
System.Web.Caching.Cache.NoSlidingExpiration);
return result;
}
}

Querying a timestamp column from LINQ to SQL

My table has a timestamp column named "RowVer" which LINQ maps to type System.Data.Linq.Binary. This data type seems useless to me because (unless I'm missing something) I can't do things like this:
// Select all records that changed since the last time we inserted/updated.
IEnumerable<UserSession> rows = db.UserSessions.Where
( usr => usr.RowVer > ???? );
So, one of the solutions I'm looking at is to add a new "calculated column" called RowTrack which is defined in SQL like this:
CREATE TABLE UserSession
(
RowVer timestamp NOT NULL,
RowTrack AS (convert(bigint,[RowVer])),
-- ... other columns ...
)
This allows me to query the database like I want to:
// Select all records that changed since the last time we inserted/updated.
IEnumerable<UserSession> rows = db.UserSessions.Where
( usr => usr.RowTrack > 123456 );
Is this a bad way to do things? How performant is querying on a calculated column? Is there a better work-around?
Also, I'm developing against Sql Server 2000 for ultimate backwards compatibility, but I can talk the boss into making 2005 the lowest common denominator.
AS Diego Frata outlines in this post there is a hack that enables timestamps to be queryable from LINQ.
The trick is to define a Compare method that takes two System.Data.Linq.Binary parameters
public static class BinaryComparer
{
public static int Compare(this Binary b1, Binary b2)
{
throw new NotImplementedException();
}
}
Notice that the function doesn't need to be implemented, only it's name (Compare) is important.
And the query will look something like:
Binary lastTimestamp = GetTimeStamp();
var result = from job in c.GetTable<tblJobs>
where BinaryComparer.Compare(job.TimeStamp, lastTimestamp)>0
select job;
(This in case of job.TimeStamp>lastTimestamp)
EDIT:
See Rory MacLeod's answer for an implementation of the method, if you need it to work outside of SQL.
SQL Server "timestamp" is only an indicator that the record has changed, its not actually a representation of Date/Time. (Although it is suppose to increment each time a record in the DB is modified,
Beware that it will wrap back to zero (not very often, admittedly), so the only safe test is if the value has changed, not if it is greater than some arbitrary previous value.
You could pass the TimeStamp column value to a web form, and then when it is submitted see if the TimeStamp from the form is different to the value in the current record - if its is different someone else has changed & saved the record in the interim.
// Select all records that changed since the last time we inserted/updated.
Is there a better work-around?
Why not have two columns, one for createddate another for lastmodifieddate. I would say that is more traditional way to handle this scenario.
Following on from jaraics' answer, you could also provide an implementation for the Compare method that would allow it to work outside of a query:
public static class BinaryExtensions
{
public static int Compare(this Binary b1, Binary b2)
{
if (b1 == null)
return b2 == null ? 0 : -1;
if (b2 == null)
return 1;
byte[] bytes1 = b1.ToArray();
byte[] bytes2 = b2.ToArray();
int len = Math.Min(bytes1.Length, bytes2.Length);
int result = memcmp(bytes1, bytes2, len);
if (result == 0 && bytes1.Length != bytes2.Length)
{
return bytes1.Length > bytes2.Length ? 1 : -1;
}
return result;
}
[DllImport("msvcrt.dll")]
private static extern int memcmp(byte[] arr1, byte[] arr2, int cnt);
}
The use of memcmp was taken from this answer to a question on comparing byte arrays. If the arrays aren't the same length, but the longer array starts with the same bytes as the shorter array, the longer array is considered to be greater than the shorter one, even if the extra bytes are all zeroes.

Custom query with Castle ActiveRecord

I'm trying to figure out how to execute a custom query with Castle ActiveRecord.
I was able to run simple query that returns my entity, but what I really need is the query like that below (with custom field set):
select count(1) as cnt, data from workstationevent where serverdatetime >= :minDate and serverdatetime < :maxDate and userId = 1 group by data having count(1) > :threshold
Thanks!
In this case what you want is HqlBasedQuery. Your query will be a projection, so what you'll get back will be an ArrayList of tuples containing the results (the content of each element of the ArrayList will depend on the query, but for more than one value will be object[]).
HqlBasedQuery query = new HqlBasedQuery(typeof(WorkStationEvent),
"select count(1) as cnt, data from workstationevent where
serverdatetime >= :minDate and serverdatetime < :maxDate
and userId = 1 group by data having count(1) > :threshold");
var results =
(ArrayList)ActiveRecordMediator.ExecuteQuery(query);
foreach(object[] tuple in results)
{
int count = (int)tuple[0]; // = cnt
string data = (string)tuple[1]; // = data (assuming this is a string)
// do something here with these results
}
You can create an anonymous type to hold the results in a more meaningful fashion. For example:
var results = from summary in
(ArrayList)ActiveRecordMediator.ExecuteQuery(query)
select new {
Count = (int)summary[0], Data = (string)summary[1]
};
Now results will contain a collection of anonymous types with properties Count and Data. Or indeed you could create your own summary type and populate it out this way too.
ActiveRecord also has the ProjectionQuery which does much the same thing but can only return actual mapped properties rather than aggregates or functions as you can with HQL.
Be aware though, if you're using ActiveRecord 1.0.3 (RC3) as I was, this will result in a runtime InvalidCastException. ActiveRecordMediator.ExecuteQuery returns an ArrayList and not a generic ICollection. So in order to make it work, just change this line:
var results = (ICollection<object[]>) ActiveRecordMediator.ExecuteQuery(query);
to
var results = (ArrayList) ActiveRecordMediator.ExecuteQuery(query);
and it should work.
Also note that using count(1) in your hql statement will make the query return an ArrayList of String instead of an ArrayList of object[] (which is what you get when using count(*).)
Just thought I'd point this out for the sake of having it all documented in one place.

Resources