EF optimize multiple order by select top 1 - sql-server

I have the following Entity Framework query:
var queryResponse = await db.DataPoints.GroupBy(x => x.Device).Select(x => new
{
lon = x.Key.DataPoints.OrderByDescending(y => y.DateTime).Select(y => y.Longitude).FirstOrDefault(),
lat = x.Key.DataPoints.OrderByDescending(y => y.DateTime).Select(y => y.Longitude).FirstOrDefault(),
date = x.Key.DataPoints.OrderByDescending(y => y.DateTime).Select(y => y.DateTime).FirstOrDefault(),
label = x.Key.Name,
})
.OrderByDescending(x => x.label)
.ToListAsync();
Now you can see in my select, I have to get lon,'lat', and 'date'. However the way im doing it, I have to orderby and select the first one 3 times. The 'DataPoints' is a very large table.
in C# i would normally do the orderBy once and just select the entire object, and then later on break it up into the 3 properties. However in this case I want SQL to return the exact fields.
is there a more efective way to write this query?

Try this:
var queryResponse =
(from g in db.DataPoints.GroupBy(x => x.Device)
let latestDataPoint = g.Key.DataPoints.OrderByDescending(p => p.DateTime)
.FirstOrDefault()
select new
{
lon = latestDataPoint.Longitude,
lat = latestDataPoint.Latitude,
date = latestDataPoint.DateTime,
label = g.Key.Name
})
.OrderByDescending(x => x.label)
.ToList();

Related

Converting SQL Server query containing GROUP BY into NHibernate LINQ

I'm basically trying to retrieve a paged list of unique GUIDs, sorted by (row) creation date.
I've been able to draft a SQL Server query that seems to work for me based on this answer, but now I have to translate that into LINQ.
SELECT TOP 15 payment.ClientRef,
MAX(payment.CreatedDateUtc)
FROM PaymentTransactionState payment
INNER JOIN OrderState orderstate ON payment.ClientRef = orderstate.ClientRef
WHERE orderstate.UserId = 2 AND
payment.PaymentState IN (
'Rejected',
'Authorized')
GROUP BY payment.ClientRef
ORDER BY MAX(payment.CreatedDateUtc) DESC,
payment.ClientRef
Problem is, I can't apply GroupBy on an IQueryOver, I'm probably missing the appropiate syntax:
session
.QueryOver<Payment>()
.JoinAlias(orderState => orderState.OrderStateEntity, () => orderStateRow)
.Where(() => orderStateRow.UserId == customer.UserId)
.WhereRestrictionOn(payment => payment.PaymentState).IsIn(paymentStates)
.GroupBy(pts => pts.ClientRef)
.OrderBy(payment => payment.CreatedDateUtc).Desc
.Skip(pageIndex*pageSize)
.Take(pageSize)
.List();
I could probably do the group by in query syntax, but I'm not so sure about the Skip & Take bit.
I would try like this:
var query = db.PaymentTransactionState
.Where( pts => pts.OrderState.UserId == 2 &&
new string[] {"Rejected", "Authorized"}.Contains(pts.PaymentState) )
.GroupBy( pts => pts.ClientRef )
.OrderByDescending( pts => pts.Max( p => p.CreatedDateUtc))
.ThenBy( p => p.Key )
.Take(15);
So here's what worked for me: basically I had to use SelectList instead of GroupBy; SelectGroup, SelectMax & TransformUsing were easy to tackle once I found that;
PaymentRow paymentAlias = null;
OrderStateRow orderStateRow = null;
var transactionStateRows = session
.QueryOver<PaymentRow >()
.JoinAlias(orderState => orderState.OrderStateEntity, () => orderStateRow)
.Where(() => orderStateRow.UserId == customer.UserId)
.WhereRestrictionOn(payment => payment.PaymentState).IsIn(paymentStates)
.SelectList(list => list
.SelectGroup(payment => payment.ClientRef).WithAlias(() => paymentAlias.ClientRef)
.SelectMax(payment => payment.CreatedDateUtc).WithAlias(() => paymentAlias.CreatedDateUtc))
.TransformUsing(Transformers.AliasToBean<PaymentRow >())
.OrderBy(payment => payment.CreatedDateUtc).Desc
.Skip(pageIndex*pageSize)
.Take(pageSize)
.List();
I'll leave this here in case someone might find my travails useful in the future. Thank you for your replies.

Zend_Db join query

I am trying to apply join query on 3 tables in Zend_Db.My query is as follow:
$id_array = array("1","2");
$query = $this->select();
$query->from(array('b' => 'brands'), array('b.brand_id','b.brand_name'))->where('b.brand_id NOT in (?)', $id_array)->order('RAND()')->limit(5);
$query->join(array('p' => 'product'), 'b.brand_id = p.brand_id', array('p.product_id', 'p.product_price'));
$query->join(array('pimg' => 'product_img_map'), 'p.product_id = pimg.product_id', array('pimg.img_location'));
$query->setIntegrityCheck(false);
$resultRows = $this->fetchAll($query);
return $resultRows;
In that one brand may have more than one product,but in query I am applying limit to brand table i.e 5. By default it is applied to product also because if one brand having 5 products it is only giving information of one brand.Is there any suggestion on this.
Thanks.
You could use a sub-query to obtain the desired results:
$id_array = array("1","2");
$subQuery = $this->select()
->from(array('b' => 'brands'), array('b.brand_id','b.brand_name'))
->where('b.brand_id NOT in (?)', $id_array)
->order('RAND()')
->limit(5);
$query = $this->select()
->from(array('b' => $subQuery), array('*'))
->join(array('p' => 'product'), 'b.brand_id = p.brand_id', array('p.product_id', 'p.product_price'))
->join(array('pimg' => 'product_img_map'), 'p.product_id = pimg.product_id', array('pimg.img_location'));

Sum of multiple queries to optimise number of queries performed

I have a query which must sum the values from several tables and add the result. The system is simply an inventory system and I'm trying to get the stock level by calculating incomings (deliveries), outgoings (issues) and adjustments to items.
As the stock level is a calculated value (sum(deliveries) - sum(issues)) + sum(adjustments) I am trying to create a function that will get this value with a minimal number of queries.
At current I have linq that performs three separate queries to get each summed value and then perform the addition/subtraction in my function, however I am convinced there must be a better way to calculate the value without having to do three separate queries.
The current function is as follows:
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
var issueItemStock = db.IssueItems.Where(x => x.ItemId == itemId).Sum(x => x.QuantityFulfilled);
var deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId).Sum(x => x.Quantity);
var adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId).Sum(x => x.Quantity);
return (deliveryItemStock - issueItemStock) + adjustmentsStock;
}
}
In my mind the SQL query is quite simple, so I have considered a stored procedure, however I think there must be a way to do this with linq.
Many thanks
Edit: Answer
Taking the code from Ocelot20's answer, with a slight change. Each of the lets can return a null, and if it does then linq throws an exception. Using the DefaultIfEmpty command will negate this, and return a 0 for the final calculation. The actual code I have used is as follows:
from ii in db.Items
let issueItems = db.IssueItems.Where(x => x.ItemId == itemId).Select(t => t.QuantityFulfilled).DefaultIfEmpty(0).Sum()
let deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId).Select(t => t.Quantity).DefaultIfEmpty(0).Sum()
let adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId).Select(t => t.Quantity).DefaultIfEmpty(0).Sum()
select (deliveryItemStock - issueItems) + adjustmentsStock);
Without knowing what your entities look like, you could do something like this:
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
// Note: Won't work if there are no IssueItems found.
return (from ii in db.IssueItems
let issueItems = db.IssueItems.Where(x => x.ItemId == itemId)
.Sum(x => x.QuantityFulfilled)
let deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId)
.Sum(x => x.Quantity)
let adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId)
.Sum(x => x.Quantity)
select issueItems + deliveryItemStock + adjustmentsStock).FirstOrDefault() ?? 0;
}
}
I tested a similar query on my own db and it worked in a single query. I suspect that since they all have a common ItemId, that using entity relations could make this look something like:
// Ideal solution:
(from i in db.Items
where i.Id == itemId
let issueItems = i.IssueItems.Sum(x => x.QuantityFulfilled)
let deliveryItemStock = i.DeliveryItems.Sum(x => x.Quantity)
let adjustmentsStock = i.Adjustments.Sum(x => x.Quantity)
select issueItems + deliveryItemStock + adjustmentsStock).SingleOrDefault() ?? 0;
Have you considered adding a view to the database that performs the calculations that you can then just use a simple select query (or SP) to return the values that you need?
I reckon this should work and the SQL generated is not particularly complex. If you think there is something wrong with it let me know and I will update my answer.
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
return db.IssueItems.Where(x => x.ItemId == itemId).GroupBy(x => x.ItemId)
.GroupJoin(db.DeliveryItems, x => x.First().ItemId, y => y.ItemId, (x, y) => new
{ Issues = x, Deliveries = y})
.GroupJoin(db.Adjustments, x=> x.Issues.First().ItemId, y=> y.ItemId, (x, y) => new
{
IssuesSum = x.Issues.Sum(i => i.QuantityFullfilled),
DeliveriesSum = x.Deliveries.Sum(d => d.Quantity),
AdjustmentsSum = y.Sum(a => a.Quantity)})
.Select(x => x.IssuesSum - x.DeliverysSum + x.AdjustmentsSum);
}
}

How to group by and order by, with LINQ, NHibernate and uNHAddins

We're building a WPF application, using Oracle database, also using NHibernate and uNHAddins extensions. In a DataGrid, we're trying to get values from a table, with this query LINQ:
return (from f in rFConsumption.GetAll()
let d = f.CounterDtm
where f.CounterDtm >= dataHoraInicio && f.CounterDtm <= dataHoraFim
group f by (d.Year - 2000) * 384 + d.Month * 32 + d.Day into g
select new RFConsumption
{
COGCounter1 = (g.Sum(f => f.COGCounter1)),
BFCounter1 = (g.Sum(f => f.BFCounter1)),
NatGasCounter1 = (g.Sum(f => f.NatGasCounter1)),
MixGasCounter1 = (g.Sum(f => f.MixGasCounter1)),
COGCounter2 = (g.Sum(f => f.COGCounter2)),
BFCounter2 = (g.Sum(f => f.BFCounter2)),
NatGasCounter2 = (g.Sum(f => f.NatGasCounter2)),
MixGasCounter2 = (g.Sum(f => f.MixGasCounter2)),
COGCounter3 = (g.Sum(f => f.COGCounter3)),
BFCounter3 = (g.Sum(f => f.BFCounter3)),
NatGasCounter3 = (g.Sum(f => f.NatGasCounter3)),
MixGasCounter3 = (g.Sum(f => f.MixGasCounter3)),
}
).ToList<RFConsumption>();
So, my question is:
How I do to use group by, with order by, using NHibernate;
How can i make group by return date data field to that specified group.
You can write the same query with NHibernate with several ways, the most interesting one for me really is the NHibernate QueryOver<>.
So, if your query works fine then, this query should work:
return Session.QueryOver<rFConsumption>()
.Where( fc => (fc.CounterDtm >= dataHoraInicio && fc.CounterDtm <= dataHoraFim))
.SelectList(list => list
.SelectGroup(f => ((f.CounterDtm.Year - 2000) * 384 + f.CounterDtm.Month * 32 + f.CounterDtm.Day)) //use group by
.Select(exp =>
new RFConsumption() //here you define the return data type based on your specified group
{
// exp[0] represents the data returned and grouped by the above statements, so here you can reform it to fit into the form of your new entity
// exp[0] here will be equivilant to g in your query
})
.OrderBy( ee => ee.COGCounter1 ) //order by any of the properties of RFConsumption
.ToList<RFConsumption>();
you should first add the entity RFConsumption:
public calss RFConsumption
{
public int COGCounter1 { get; set; }
public int BFCounter { get; set; }
....
}

LINQ to Entities combine two IQueryable<AnonymousType>

I have 2 queries which work fine:
var q = (from c in _context.Wxlogs
where (SqlFunctions.DatePart("Month", c.LogDate2) == m3) && (SqlFunctions.DatePart("Year", c.LogDate2) == y1)
group c by c.LogDate2
into g
orderby g.Key
let maxTemp = g.Max(c => c.Temp)
let minTemp = g.Min(c => c.Temp)
let maxHum = g.Max(c => c.Humidity)
let minHum = g.Min(c => c.Humidity)
select new
{
LogDate = g.Key,
MaxTemp = maxTemp,
MaxTempTime = g.FirstOrDefault(c => c.Temp == maxTemp).LogTime,
MinTemp = minTemp,
MinTempTime = g.FirstOrDefault(c => c.Temp == minTemp).LogTime,
MaxHum = maxHum,
MaxHumTime = g.FirstOrDefault(c => c.Humidity == maxHum).LogTime,
MinHum = minHum,
MinHumTime = g.FirstOrDefault(c => c.Humidity == minHum).LogTime,
});
var r = (from c in _context.Wxlogs
where
(SqlFunctions.DatePart("Month", c.LogDate2) == m3) &&
(SqlFunctions.DatePart("Year", c.LogDate2) == y1)
group c by c.LogDate2
into g
orderby g.Key
let maxDew = g.Max(c => c.Dew_Point)
let minDew = g.Min(c => c.Dew_Point)
//let maxWind = g.Max(c=> c.Wind_Gust)
let maxRainRate = g.Max(c => c.Rain_rate_now)
let maxPres = g.Max(c => c.Barometer)
let minPres = g.Min(c => c.Barometer)
select new
{
LogDate = g.Key,
MaxRainRateTime = g.FirstOrDefault(c => c.Rain_rate_now == maxRainRate).LogTime,
MaxPres = maxPres,
MaxPresTime = g.FirstOrDefault(c => c.Barometer == maxPres).LogTime,
MinPres = minPres,
MinPresTime = g.FirstOrDefault(c => c.Barometer == minPres).LogTime,
MinDew = minDew,
MinDewTime = g.FirstOrDefault(c => c.Dew_Point == minDew).LogTime,
MaxDew = maxDew,
MaxDewTime = g.FirstOrDefault(c => c.Dew_Point == maxDew).LogTime,
MaxRainRate = maxRainRate,
});
however when I try to combine them using union, in order to output the results to a WPF datgrid:
var result = r.Union(q);
the following error is thrown on the union:
Error 1 Instance argument: cannot convert from 'System.Linq.IQueryable<AnonymousType#1>' to 'System.Linq.ParallelQuery<AnonymousType#2>'
I can't seem to find a way to make this work and any help would be appreciated.
A "union" operation combines two sequences of the same type into a single set (i.e. eliminating all the duplicates). Since you clearly have sequences of two different types, you don't want a union operation. It looks like you want a "concat" operation, which just chains two sequences together. You need something like:
var result = r.Concat<object>(q);
However, since you're using L2E, your query will try to get executed on the server. Since your server won't allow you to combine your two queries (due to mismatched types), you need to execute them separately and then concat the sequences on the client:
var result = r.AsEnumerable().Concat<object>(q.AsEnumerable());
The use of AsEnumerable() runs the queries on the server and brings the results to the client.
Since it turns out that you want to combine the sequences next to each other (i.e. using the same rows in your grid but another group of columns), you actually want a join operation:
var result = from rrow in r.AsEnumerable()
join qrow in q.AsEnumerable() on rrow.LogDate equals qrow.LogDate
select new { rrow.LogDate,
rrow.MaxTemp, rrow.MaxTempTime,
rrow.MinTemp, rrow.MinTempTime,
rrow.MaxHum, rrow.MaxHumTime,
rrow.MinHum, rrow.MinHumTime,
qrow.MaxRainRate, qrow.MaxRainRateTime,
qrow.MaxPres, qrow.MaxPresTime,
qrow.MinPres, qrow.MinPresTime,
qrow.MaxDew, qrow.MaxDewTime,
qrow.MinDew, qrow.MinDewTime };

Resources