I have the following Linq queries that differs only on the where clause:
Query 1:
var initialList = await
context.AlertDetailsDbSet
// Pouring the TrackedVehicule related graph
.Include(ad => ad.TrackedVehicule.Registration)
.Include(ad => ad.TrackedVehicule.Device)
.Include(ad => ad.TrackedVehicule.GpsBox.Icon)
.Include(ad => ad.TrackedVehicule.GpsBoxTypeNavigation.Icon)
.Include(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo.VehiculeConfigurationNavigation)
// Loading the associated alert
.Include(ad => ad.Alert)
#if NETSTANDARD2_1
// This filter is not supported with EF Core 2.X
// It should be done client side
.Where(ad => ad.Alert.CompanyId == companyId && ad.Alert.AlertRule.Contains("SerializableContextAlert") && ad.AlertDateTime >= startDate && ad.AlertDateTime <= endDate && ad.TrackedVehicule != null && ad.TrackedVehicule.CompanyId == companyId && ad.TrackedVehicule.RowEnabled == true && (ad.TrackedVehicule.GpsBoxType == 29 || ad.TrackedVehicule.GpsBoxType == 12))
#endif
.Skip(0)
// A limit is applied on returned elements
.Take(1000)
.OrderByDescending(ad => ad.AlertDateTime)
// .Skip(skip)
// .Take(take)
// We disable the tracking mechanism because the context will be thrown away as soon as we have the data
.AsNoTracking()
// We want an asynchrounous execution
.ToListAsync(cancellationToken)
Query 2:
var initialList = await
context.AlertDetailsDbSet
// Pouring the TrackedVehicule related graph
.Include(ad => ad.TrackedVehicule.Registration)
.Include(ad => ad.TrackedVehicule.Device)
.Include(ad => ad.TrackedVehicule.GpsBox.Icon)
.Include(ad => ad.TrackedVehicule.GpsBoxTypeNavigation.Icon)
.Include(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo.VehiculeConfigurationNavigation)
// Loading the associated alert
.Include(ad => ad.Alert)
#if NETSTANDARD2_1
// This filter is not supported with EF Core 2.X
// It should be done client side
.Where(ad => ad.Alert.CompanyId == companyId && ad.Alert.AlertRule.Contains("SerializableContextAlert") && ad.TrackedVehicule != null && ad.TrackedVehicule.RowEnabled == true && ad.TrackedVehicule.CompanyId == companyId && (ad.TrackedVehicule.GpsBoxType == 29 || ad.TrackedVehicule.GpsBoxType == 12) && ad.AckTime.HasValue == acknwoledged)
#endif
.OrderByDescending(ad => ad.AlertDateTime)
.Skip(skip)
.Take(take)
// We disable the tracking mechanism because the context will be thrown away as soon as we have the data
.AsNoTracking()
// We want an asynchrounous execution
.ToListAsync(cancellationToken)
Basically, the first query is (among aother things) filtering on a boolean property, while the second one is filtering on a range of dates.
My problem is that those queries are not translated to the same T-TSQL leading to different results.
TSQL 1 :
exec sp_executesql N'SELECT [t].[AlertDetailID], [t].[AckMachineName], [t].[AckPhoneNumber], [t].[AckTime], [t].[AckUserName], [t].[AlertDateTime], [t].[AlertID], [t].[AlertedCorridorId], [t].[AlertedItemId],
[t].[AssociatedVehiculeID], [t].[AssociatedVehiculeUserID], [t].[Context], [t].[CustomID], [t].[CustomInfo], [t].[DbInsertTime], [t].[IsFleetAlert], [t].[MessageStatus], [t].[ReceivedTime], [t].[RowVersion],
[t].[SafeProtectCustomInfo], [t].[TrackedVehiculeID], [t].[TrackedVehiculeUserID], [v0].[VehiculeID], [v0].[Address], [v0].[AddressProtocol], [v0].[BoardID], [v0].[Category], [v0].[CompanyID], [v0].[CustomId], [v0].[DbInsertTime], [v0].[Description], [v0].[GpsBoxSubType], [v0].[GpsBoxTrackingDelay], [v0].[GpsBoxType], [v0].[HardwareID], [v0].[HasGeoWorker], [v0].[IconID], [v0].[Name], [v0].[PhoneNumber], [v0].[RowEnabled], [v0].[RowVersion], [v0].[VehiculeUserID], [r].[VehiculeID], [r].[CompanyId], [r].[LatestRegistrationStatusChangeDate], [r].[RegistrationMailSent], [r].[RegistrationStatusId], [d].[VehiculeID], [d].[AppVersion], [d].[Brand], [d].[Details], [d].[Imei], [d].[Model], [d].[Name], [d].[OsVersion], [d].[RowVersion], [g].[GpsBoxTypeID], [g].[GpsBoxSubTypeID], [g].[IconId], [g].[Name], [i].[IconId], [i].[Category], [i].[FileName], [i].[Icon], [g0].[GpsBoxTypeId], [g0].[Category], [g0].[IconId], [g0].[Name], [i0].[IconId], [i0].[Category], [i0].[FileName], [i0].[Icon], [v1].[VehiculeID], [v1].[BoardConfiguration], [v1].[BoardConnected], [v1].[BoardCurrentAddress], [v1].[BoardLastCommunicationTime], [v1].[Connected], [v1].[CurrentAddress], [v1].[CurrentDelay], [v1].[FuelConsumptionEstimationTime], [v1].[HeartBeatPeriod], [v1].[InsureCoherence], [v1].[KillHedgehog], [v1].[LastCommunicationTime], [v1].[LastConnexionTime], [v1].[LastParkMileage], [v1].[LastParkMileageTime], [v1].[NormalTrackingMode], [v1].[SmartModeDelay], [v1].[SmartModeDistance], [v1].[TimeModeDelay], [v1].[UpdateStartTime], [v1].[UpdateStatus], [v1].[VehiculeConfiguration], [g1].[ConfigurationID], [g1].[ConfigName], [g1].[Firmware], [g1].[GpsBoxType], [g1].[Master], [g1].[MasterName], [g1].[SimContract], [g1].[System], [a1].[AlertID], [a1].[AlertEnabled], [a1].[AlertRule], [a1].[Always], [a1].[CalendarID], [a1].[Category], [a1].[CompanyID], [a1].[Description], [a1].[DisplayName], [a1].[RowEnabled], [a1].[RowVersion], [a1].[Shared], [a1].[UserID]
FROM (
SELECT [a].[AlertDetailID], [a].[AckMachineName], [a].[AckPhoneNumber], [a].[AckTime], [a].[AckUserName], [a].[AlertDateTime], [a].[AlertID], [a].[AlertedCorridorId], [a].[AlertedItemId], [a].[AssociatedVehiculeID], [a].[AssociatedVehiculeUserID], [a].[Context], [a].[CustomID], [a].[CustomInfo], [a].[DbInsertTime], [a].[IsFleetAlert], [a].[MessageStatus], [a].[ReceivedTime], [a].[RowVersion], [a].[SafeProtectCustomInfo], [a].[TrackedVehiculeID], [a].[TrackedVehiculeUserID], [a0].[AlertID] AS [AlertID0]
FROM [AlertDetail] AS [a]
INNER JOIN [Alert] AS [a0] ON [a].[AlertID] = [a0].[AlertID]
LEFT JOIN [Vehicule] AS [v] ON [a].[TrackedVehiculeID] = [v].[VehiculeID]
WHERE (((((([a0].[CompanyID] = #__companyId_0) AND (CHARINDEX(N''SerializableContextAlert'', [a0].[AlertRule]) > 0)) AND [v].[VehiculeID] IS NOT NULL) AND ([v].[RowEnabled] = CAST(1 AS bit))) AND ([v].[CompanyID] = #__companyId_0)) AND (([v].[GpsBoxType] = 29) OR ([v].[GpsBoxType] = 12))) AND (CASE
WHEN [a].[AckTime] IS NOT NULL THEN CAST(1 AS bit)
ELSE CAST(0 AS bit)
END = #__acknwoledged_1)
ORDER BY [a].[AlertDateTime] DESC
OFFSET #__p_2 ROWS FETCH NEXT #__p_3 ROWS ONLY
) AS [t]
LEFT JOIN [Vehicule] AS [v0] ON [t].[TrackedVehiculeID] = [v0].[VehiculeID]
LEFT JOIN [SafeProtect].[Registration] AS [r] ON [v0].[VehiculeID] = [r].[VehiculeID]
LEFT JOIN [SafeProtect].[Device] AS [d] ON [v0].[VehiculeID] = [d].[VehiculeID]
LEFT JOIN [GpsBoxSubType] AS [g] ON ([v0].[GpsBoxType] = [g].[GpsBoxTypeID]) AND ([v0].[GpsBoxSubType] = [g].[GpsBoxSubTypeID])
LEFT JOIN [cst].[Icon] AS [i] ON [g].[IconId] = [i].[IconId]
LEFT JOIN [GpsBoxType] AS [g0] ON [v0].[GpsBoxType] = [g0].[GpsBoxTypeId]
LEFT JOIN [cst].[Icon] AS [i0] ON [g0].[IconId] = [i0].[IconId]
LEFT JOIN [VehiculeGpsBoxInfo] AS [v1] ON [v0].[VehiculeID] = [v1].[VehiculeID]
LEFT JOIN [GpsBoxConfiguration] AS [g1] ON [v1].[VehiculeConfiguration] = [g1].[ConfigurationID]
INNER JOIN [Alert] AS [a1] ON [t].[AlertID] = [a1].[AlertID]
ORDER BY [t].[AlertDateTime] DESC',N'#__companyId_0 int,#__acknwoledged_1 bit,#__p_2 int,#__p_3 int',#__companyId_0=1013,#__acknwoledged_1=0,#__p_2=0,#__p_3=999
TSQL 2:
exec sp_executesql N'SELECT [t].[AlertDetailID], [t].[AckMachineName], [t].[AckPhoneNumber], [t].[AckTime], [t].[AckUserName], [t].[AlertDateTime], [t].[AlertID], [t].[AlertedCorridorId], [t].[AlertedItemId],
[t].[AssociatedVehiculeID], [t].[AssociatedVehiculeUserID], [t].[Context], [t].[CustomID], [t].[CustomInfo], [t].[DbInsertTime], [t].[IsFleetAlert], [t].[MessageStatus], [t].[ReceivedTime], [t].[RowVersion],
[t].[SafeProtectCustomInfo], [t].[TrackedVehiculeID], [t].[TrackedVehiculeUserID], [v0].[VehiculeID], [v0].[Address], [v0].[AddressProtocol], [v0].[BoardID], [v0].[Category], [v0].[CompanyID], [v0].[CustomId], [v0].[DbInsertTime], [v0].[Description], [v0].[GpsBoxSubType], [v0].[GpsBoxTrackingDelay], [v0].[GpsBoxType], [v0].[HardwareID], [v0].[HasGeoWorker], [v0].[IconID], [v0].[Name], [v0].[PhoneNumber], [v0].[RowEnabled], [v0].[RowVersion], [v0].[VehiculeUserID], [r].[VehiculeID], [r].[CompanyId], [r].[LatestRegistrationStatusChangeDate], [r].[RegistrationMailSent], [r].[RegistrationStatusId], [d].[VehiculeID], [d].[AppVersion], [d].[Brand], [d].[Details], [d].[Imei], [d].[Model], [d].[Name], [d].[OsVersion], [d].[RowVersion], [g].[GpsBoxTypeID], [g].[GpsBoxSubTypeID], [g].[IconId], [g].[Name], [i].[IconId], [i].[Category], [i].[FileName], [i].[Icon], [g0].[GpsBoxTypeId], [g0].[Category], [g0].[IconId], [g0].[Name], [i0].[IconId], [i0].[Category], [i0].[FileName], [i0].[Icon], [v1].[VehiculeID], [v1].[BoardConfiguration], [v1].[BoardConnected], [v1].[BoardCurrentAddress], [v1].[BoardLastCommunicationTime], [v1].[Connected], [v1].[CurrentAddress], [v1].[CurrentDelay], [v1].[FuelConsumptionEstimationTime], [v1].[HeartBeatPeriod], [v1].[InsureCoherence], [v1].[KillHedgehog], [v1].[LastCommunicationTime], [v1].[LastConnexionTime], [v1].[LastParkMileage], [v1].[LastParkMileageTime], [v1].[NormalTrackingMode], [v1].[SmartModeDelay], [v1].[SmartModeDistance], [v1].[TimeModeDelay], [v1].[UpdateStartTime], [v1].[UpdateStatus], [v1].[VehiculeConfiguration], [g1].[ConfigurationID], [g1].[ConfigName], [g1].[Firmware], [g1].[GpsBoxType], [g1].[Master], [g1].[MasterName], [g1].[SimContract], [g1].[System], [a1].[AlertID], [a1].[AlertEnabled], [a1].[AlertRule], [a1].[Always], [a1].[CalendarID], [a1].[Category], [a1].[CompanyID], [a1].[Description], [a1].[DisplayName], [a1].[RowEnabled], [a1].[RowVersion], [a1].[Shared], [a1].[UserID]
FROM (
SELECT [a].[AlertDetailID], [a].[AckMachineName], [a].[AckPhoneNumber], [a].[AckTime], [a].[AckUserName], [a].[AlertDateTime], [a].[AlertID], [a].[AlertedCorridorId], [a].[AlertedItemId], [a].[AssociatedVehiculeID], [a].[AssociatedVehiculeUserID], [a].[Context], [a].[CustomID], [a].[CustomInfo], [a].[DbInsertTime], [a].[IsFleetAlert], [a].[MessageStatus], [a].[ReceivedTime], [a].[RowVersion], [a].[SafeProtectCustomInfo], [a].[TrackedVehiculeID], [a].[TrackedVehiculeUserID], [a0].[AlertID] AS [AlertID0]
FROM [AlertDetail] AS [a]
INNER JOIN [Alert] AS [a0] ON [a].[AlertID] = [a0].[AlertID]
LEFT JOIN [Vehicule] AS [v] ON [a].[TrackedVehiculeID] = [v].[VehiculeID]
WHERE ((((((([a0].[CompanyID] = #__companyId_0) AND (CHARINDEX(N''SerializableContextAlert'', [a0].[AlertRule]) > 0)) AND ([a].[AlertDateTime] >= #__startDate_1)) AND ([a].[AlertDateTime] <= #__endDate_2)) AND [v].[VehiculeID] IS NOT NULL) AND ([v].[CompanyID] = #__companyId_0)) AND ([v].[RowEnabled] = CAST(1 AS bit))) AND (([v].[GpsBoxType] = 29) OR ([v].[GpsBoxType] = 12))
ORDER BY (SELECT 1)
OFFSET #__p_3 ROWS FETCH NEXT #__p_4 ROWS ONLY
) AS [t]
LEFT JOIN [Vehicule] AS [v0] ON [t].[TrackedVehiculeID] = [v0].[VehiculeID]
LEFT JOIN [SafeProtect].[Registration] AS [r] ON [v0].[VehiculeID] = [r].[VehiculeID]
LEFT JOIN [SafeProtect].[Device] AS [d] ON [v0].[VehiculeID] = [d].[VehiculeID]
LEFT JOIN [GpsBoxSubType] AS [g] ON ([v0].[GpsBoxType] = [g].[GpsBoxTypeID]) AND ([v0].[GpsBoxSubType] = [g].[GpsBoxSubTypeID])
LEFT JOIN [cst].[Icon] AS [i] ON [g].[IconId] = [i].[IconId]
LEFT JOIN [GpsBoxType] AS [g0] ON [v0].[GpsBoxType] = [g0].[GpsBoxTypeId]
LEFT JOIN [cst].[Icon] AS [i0] ON [g0].[IconId] = [i0].[IconId]
LEFT JOIN [VehiculeGpsBoxInfo] AS [v1] ON [v0].[VehiculeID] = [v1].[VehiculeID]
LEFT JOIN [GpsBoxConfiguration] AS [g1] ON [v1].[VehiculeConfiguration] = [g1].[ConfigurationID]
INNER JOIN [Alert] AS [a1] ON [t].[AlertID] = [a1].[AlertID]
ORDER BY [t].[AlertDateTime] DESC',N'#__companyId_0 int,#__startDate_1 datetime,#__endDate_2 datetime,#__p_3 int,#__p_4 int',#__companyId_0=1013,#__startDate_1='2020-09-07 21:59:59',#__endDate_2='2020-09-08 21:59:59',#__p_3=0,#__p_4=999
It took me some time to spot a tiny by impacting difference between the two queries.
If you look carefully, the first subquery contains an ORDER BY which is what I want:
ORDER BY [a].[AlertDateTime] DESC
The second linq query, despite using the same OderBy is translated to a TSQL subquery that contains
ORDER BY (SELECT 1)
That sounds crasy to me.
I hope that someone could explain what is going on.
It has nothing to do with filtering (Where), but the order of ordering (OrderBy{Descending}) and row limiting (Skip / Take) operators in LINQ query.
The fake ORDER BY (SELECT 1) definitely is generated for the LINQ query having the following fragment
.Skip(0)
.Take(1000)
.OrderByDescending(ad => ad.AlertDateTime)
// .Skip(skip)
// .Take(take)
As you can see, here the row limiting operators are applied before the ordering. Since SqlServer OFFSET FETCH SQL construct requires ORDER BY, EF Core translator inserts fake one (it would have been better to throw exception or let the database do that, but that's another story. Actually before EF Core 3.0 there was CoreEventId.RowLimitingOperationWithoutOrderByWarning which could be configured to throw, but for some unknown reason it has been obsoleted and has no effect in EF Core 3.x).
The correct place of row limiting of course is after ordering, as it is in the commented out code. So simply remove the (temporary I guess) Skip(0).Take(1000) and uncomment the .Skip(skip).Take(take). Problem solved.
I am developing a Catalyst app which uses Template::Toolkit as template engine. One page needs a list of equal input elements. They can be taken from an array but I need both sort order and a descriptive label for the element.
For having a sort order I would use an array. For storing an additional value per key a hash is perfect. How to combine both in TT? I could use both things but that seems ugly and can cause mistakes when changing the fields.
However, I prefer doing this in TT because both the descriptions and the order of form elements is a front-end thing.
This is how I would do it in pure Perl:
#!/usr/bin/perl -w
use 5.10.0;
# definition of description and order in 1 step
my #fields = (
property_foo => "Some property",
property_bar => "Important field",
property_baz => "Something else",
);
# extract information
my %descriptions = #fields;
my #order = #fields[grep {($_ + 1) % 2} 0..(scalar #fields - 1)];
say "=== natural perl sort order ===";
foreach (keys %descriptions) {say $_};
say "=== wanted output ===";
foreach (#order) {
say $descriptions{$_} . ": [label for $_]";
}
Outputs:
=== natural perl sort order ===
property_baz
property_foo
property_bar
=== wanted output ===
Some property: [label for property_foo]
Important field: [label for property_bar]
Something else: [label for property_baz]
This is what I write in my template:
[%
order = (
property_foo,
property_bar,
property_baz,
);
descriptions = {
property_foo => "Some property",
property_bar => "Important field",
property_baz => "Something else",
}
FOREACH property IN order %]
[% descriptions.$property %]: <input name="[% property %]" />
[% END %]
However, it is really ugly to have the same information (list of fields) twice. I want to avoid editing the list twice and with a longer list of fields it gets really annoying (about 20 items, not long enough to do some database stuff).
It's amazing how people complicate easy stuff!
You don't need the #fields array. Please read the perldoc about keys, values and sort.
# untested sketch
my %description = ( prop23 => "foo", prop24 => "bar" );
foreach my $key(sort (keys %description)) {
print $key, " is: ", $description{$key}, "\n"; # or whatever
}
Addendum: Regarding the order of keys, just do the following:
my #arbitraryOrder = qw(prop42 prop35 prop1 ...); # allows to map number to key
my %keytoNumber = (); # will map keys to numbers
foreach my $i(0..$#arbitraryOrder) $keyToNumber{$arbitraryOrder[$i]} = $i;
Writing the comparison function for sort is left as an exercise :)
You might be interested in Tie::IxHash
It is a "hash" that keeps the order in which you add keys (value updates do not affect sorting).
EDIT: A brief example:
use warnings;
use strict;
use Tie::IxHash;
tie my %H, "Tie::IxHash";
$H{foo} = 1;
$H{bar} = 2;
# order of keys is now always 'foo', 'bar'
print keys %H;
EDIT2: I've tried it out, and it actually works:
#!/usr/bin/perl -w
use strict;
use Template;
use Tie::IxHash;
# my %h; # this breaks ordering
tie my %H, "Tie::IxHash"; # this keeps ordering
#H{qw/f oo b a r/} = 1..100;
# don't define $H{'keys'} or you'll get disappointed
my $tpl = Template->new();
$tpl->process(\*DATA, {hash=>\%H});
__DATA__
[% FOREACH k IN hash.keys %]
[% k %] => [% hash.$k %]
[% END %]
If you need ordering and multiple pieces of information then you should consider an array of hash references.
my #fields = (
{ id => 'property_foo',
label => 'Some property' },
{ id => 'property_bar',
label => 'Important field' },
{ id => 'property_baz',
label => 'Something else' },
);
foreach (#fields) {
print "ID: $_->{id}, Label: $_->{label}\n";
}
If the complexity increases much beyond this, you might consider replacing the hashrefs with real objects.
And, in TT, it looks like this:
[%-
properties = [
{id => 'property_foo',
label => 'Some property'},
{id => 'property_bar',
label => 'Important field'},
{id => 'property_baz',
label => 'Something else'},
];
-%]
[%- FOREACH property IN properties %]
[% property.label %]: <input name="[% property.id %]" />
[% END %]
Actually if you want your hash alphabetically sorted by key, Template::Toolkit does that for you.
test.pl
use strict;
use warnings;
use Template;
my %hash = qw' a 1 b 2 c 3 ';
my $config = {
INCLUDE_PATH => '/search/path',
};
my $input = 'test.tt2';
my $template = Template->new( $config );
$template->process( $input, {
hash => \%hash,
})
test.tt2
[% FOREACH hash -%]
[% key %] => [% value %]
[% END %]
output
a => 1
b => 2
c => 3