Covert Collection of Objects to Collection of Dictionary<string, object> - c

I'm fetching a large Amount of data from RIA Service. the return type have group of objects like RouteA, HistroyRouteA. HistroyLogRouteA all have records for different years with same Unique Key.
I have to Bind this data dynamically to a RadGridView. Always I have unknown columns in result.
For this I followed
http://blogs.telerik.com/vladimirenchev/posts/11-09-28/dynamic-binding-for-your-silverlight-applications.aspx
http://www.telerik.com/forums/rowdetailstemplate-binding-with-dynamic-data
And build My data Collection with Code :
private void OnShowPreviousYear(object parameter)
{
GridViewHeaderCell cell = parameter as GridViewHeaderCell;
var head = cell.Column.Header;
this.context.Load<Route>(this.context.GetRoutesQuery(), LoadBehavior.MergeIntoCurrent, OnRouteHistoryLoadComplete, null);
}
private void OnRouteHistoryLoadComplete(LoadOperation<Route> lo)
{
object ro = null;
if (lo.Entities != null)
{
this.context.Load<Routeshistory>(this.context.GetRouteshistoriesQuery(), LoadBehavior.MergeIntoCurrent, (lp) =>
{
Route recent = lo.Entities.FirstOrDefault();
int year =(int)recent.Hpmsyear-1;
var rows = this.context.Routes.Join(this.context.Routeshistories,
r => r.Routeid.ToString(),
h => h.Routeid.ToString(),
(r, h) => new { r, h });//.Where(t => t.r.Routeid == t.h.Routeid );
RoutesGridData = new ObservableCollection<DataRow>();
int count = 0;
foreach (var tmpR in rows)
{
//Debug.WriteLine(tmpR.r.Routeid + " -- " + tmpR.h.Routeid);
if (count < 50)
{
DataRow row = new DataRow();
if (tmpR.r is Route)
{
Type type = tmpR.r.GetType();
foreach (PropertyInfo info in type.GetProperties())
{
// Debug.WriteLine(info.Name + "--- NAME OF PRR");
var val = info.GetValue(tmpR.r, null);
if (!info.Name.Equals("EntityConflict")
&& !info.Name.Equals("ValidationErrors")
&& !info.Name.Equals("HasValidationErrors")
&& !info.Name.Equals("EntityState")
&& !info.Name.Equals("HasChanges")
&& !info.Name.Equals("IsReadOnly")
&& !info.Name.Equals("EntityActions"))
{
row[info.Name] = val;
}
}
}
// other tables...
RoutesGridData.Add(row);
}
count++;
}
}, null);
}
// var b = ro;
}
this code works fine for small record like 50 rows. but I when It try to convert all data it become slow. and screen crashes. I think this is because of Reflection. Is there any other way to convert my fetch data into Dictionary? Means I can map my table to dictionary in Entity Framework or Linq can do this for me without getting my code slow etc.
My Entities are mapped with EF 6 & I m using Deart oracle connector.

Due to reflection it was getting extremely slow so I did in during Linq query it's working for a while what data I have with me.
var rowss = this.context.Routes.Join(this.context.Routeshistories,
r => r.Routeid,
h => h.Routeid,
(r, h) => new DataRow(
(from x in r.GetType().GetProperties() select x).Where(x => x.Name != "EntityConflict"
&& x.Name != "ValidationErrors"
&& x.Name != "HasValidationErrors"
&& x.Name != "HasChanges"
&& x.Name != "EntityState"
&& x.Name != "IsReadOnly"
&& x.Name != "EntityActions")
.ToDictionary(x => x.Name, x => (x.GetGetMethod().Invoke(r, null) == null ? "" : x.GetGetMethod().Invoke(r, null))),
(from x in h.GetType().GetProperties() select x).Where(x => x.Name == head)
.ToDictionary(x => x.Name + "-" + year.ToString(), x => (x.GetGetMethod().Invoke(h, null) == null ? "" : x.GetGetMethod().Invoke(h, null))))
);// , new EqualityComparerString()
RoutesGridData = new ObservableCollection<DataRow>(rowss);

Related

Create a Schema from an existing one?

I have two databases:
database A with missing indexes,constraints,foreign keys etc ...
database B with all the missing features
I have loaded the schemas of the two databases in my C# code and after comparing them I added the missing features to the first schema.
Now I want to update that schema back to SQL Server. How to do that? Could I create a schema from an existing schema?
This is what I have tried:
public static void Compare(ref DbMetaData dbSchema , ref DbMetaData dbSchema2)
{
foreach (var schemaGroup in dbSchema.SchemasDic.Values )
{
DbUserSchemaInfo schemaGroup2=dbSchema2.SchemasDic.Values.First(sc => sc.SchemaName == schemaGroup.SchemaName);
foreach (var x in schemaGroup.TablesDic)
{
var y = schemaGroup2.TablesDic.First(sc => sc.Value.TableName == x.Value.TableName);
// GET the difference in columns
var colDiff = x.Value.Columns.Except(y.Value.Columns);
// GET the difference in indexes
var indexDiff = x.Value.Indexes.Except(y.Value.Indexes);
// GET the difference in constraints
var constraintDiff = x.Value.DfConstraints.Except(y.Value.DfConstraints);
// GET the difference in Fk
var foreignKeysDiff = x.Value.ForeignKeys.Except(y.Value.ForeignKeys);
//check if a primary key exists
var primaryKeysDiff = y.Value.PrimaryKey == null ? x.Value.PrimaryKey : null;
// GET the difference in triggers
var triggersDiff = x.Value.Triggers.Except(y.Value.Triggers);
//merge the difference
if (indexDiff.Count() != 0)
{
foreach (var index in indexDiff)
{
y.Value.AddTableIndex(index.Value);
}
}
if (constraintDiff.Count() != 0)
{
foreach (var constraint in constraintDiff)
{
y.Value.AddDefaultConstraint(constraint.Value);
}
}
if (foreignKeysDiff.Count() != 0)
{
foreach (var foreignKey in foreignKeysDiff)
{
y.Value.AddFk(foreignKey.Value);
}
}
if (triggersDiff.Count() != 0)
{
foreach (var trigger in triggersDiff)
{
y.Value.AddTableTrigger(trigger.Value);
}
}
}
}
//Update DbSchema in sqlserver
}

Sum of multiple queries to optimise number of queries performed

I have a query which must sum the values from several tables and add the result. The system is simply an inventory system and I'm trying to get the stock level by calculating incomings (deliveries), outgoings (issues) and adjustments to items.
As the stock level is a calculated value (sum(deliveries) - sum(issues)) + sum(adjustments) I am trying to create a function that will get this value with a minimal number of queries.
At current I have linq that performs three separate queries to get each summed value and then perform the addition/subtraction in my function, however I am convinced there must be a better way to calculate the value without having to do three separate queries.
The current function is as follows:
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
var issueItemStock = db.IssueItems.Where(x => x.ItemId == itemId).Sum(x => x.QuantityFulfilled);
var deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId).Sum(x => x.Quantity);
var adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId).Sum(x => x.Quantity);
return (deliveryItemStock - issueItemStock) + adjustmentsStock;
}
}
In my mind the SQL query is quite simple, so I have considered a stored procedure, however I think there must be a way to do this with linq.
Many thanks
Edit: Answer
Taking the code from Ocelot20's answer, with a slight change. Each of the lets can return a null, and if it does then linq throws an exception. Using the DefaultIfEmpty command will negate this, and return a 0 for the final calculation. The actual code I have used is as follows:
from ii in db.Items
let issueItems = db.IssueItems.Where(x => x.ItemId == itemId).Select(t => t.QuantityFulfilled).DefaultIfEmpty(0).Sum()
let deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId).Select(t => t.Quantity).DefaultIfEmpty(0).Sum()
let adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId).Select(t => t.Quantity).DefaultIfEmpty(0).Sum()
select (deliveryItemStock - issueItems) + adjustmentsStock);
Without knowing what your entities look like, you could do something like this:
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
// Note: Won't work if there are no IssueItems found.
return (from ii in db.IssueItems
let issueItems = db.IssueItems.Where(x => x.ItemId == itemId)
.Sum(x => x.QuantityFulfilled)
let deliveryItemStock = db.DeliveryItems.Where(x => x.ItemId == itemId)
.Sum(x => x.Quantity)
let adjustmentsStock = db.Adjustments.Where(x => x.ItemId == itemId)
.Sum(x => x.Quantity)
select issueItems + deliveryItemStock + adjustmentsStock).FirstOrDefault() ?? 0;
}
}
I tested a similar query on my own db and it worked in a single query. I suspect that since they all have a common ItemId, that using entity relations could make this look something like:
// Ideal solution:
(from i in db.Items
where i.Id == itemId
let issueItems = i.IssueItems.Sum(x => x.QuantityFulfilled)
let deliveryItemStock = i.DeliveryItems.Sum(x => x.Quantity)
let adjustmentsStock = i.Adjustments.Sum(x => x.Quantity)
select issueItems + deliveryItemStock + adjustmentsStock).SingleOrDefault() ?? 0;
Have you considered adding a view to the database that performs the calculations that you can then just use a simple select query (or SP) to return the values that you need?
I reckon this should work and the SQL generated is not particularly complex. If you think there is something wrong with it let me know and I will update my answer.
public static int GetStockLevel(int itemId)
{
using (var db = EntityModel.Create())
{
return db.IssueItems.Where(x => x.ItemId == itemId).GroupBy(x => x.ItemId)
.GroupJoin(db.DeliveryItems, x => x.First().ItemId, y => y.ItemId, (x, y) => new
{ Issues = x, Deliveries = y})
.GroupJoin(db.Adjustments, x=> x.Issues.First().ItemId, y=> y.ItemId, (x, y) => new
{
IssuesSum = x.Issues.Sum(i => i.QuantityFullfilled),
DeliveriesSum = x.Deliveries.Sum(d => d.Quantity),
AdjustmentsSum = y.Sum(a => a.Quantity)})
.Select(x => x.IssuesSum - x.DeliverysSum + x.AdjustmentsSum);
}
}

Query Slow in Linq, Fast in LinqPad, SQL Management Studio and SQL Profiler

I have this linq query i'm using and it's taking 50 seconds to run when i am running it my asp.net application, however the same query executes in 500ms in LinqPad and Sql Management Studio.
I even took the query from the SQL Profiler and ran it again in SQL Management Studio and it takes around 500ms. What overhead Linq could be doing, that an extra 49s??
Below is the code for reference, thanks for your help.
var rCampaign =
(from a in db.AdCreative
join h in db.AdHit on a.ID equals h.AdID into gh
join l in db.AdGroup_Location on a.AdGroupID equals l.AdGroupID into gj
from subloc in gj.DefaultIfEmpty()
from subhits in gh.DefaultIfEmpty()
where a.AdGroup.AdHost.Select(q => q.ID).Contains(rPlatform.ID) &&
a.AdGroup.AdPublisher.Select(q => q.ID).Contains(rPublisher.ID) &&
a.AdDimensionID == AdSize &&
a.AdGroup.Campaign.Starts <= rNow &&
a.AdGroup.Campaign.Ends >= rNow &&
subhits.HitType == 1 &&
(subloc == null || subloc.LocationID == rLocationID)
select new {
ID = a.ID,
Name = a.Name,
Spent = (subhits.AdDimension != null) ? ((double)subhits.AdDimension.Credit / 1000) : 0,
CampaignID = a.AdGroup.Campaign.ID,
CampaignName = a.AdGroup.Campaign.Name,
CampaignBudget = a.AdGroup.Campaign.DailyBudget
}).GroupBy(adgroup => adgroup.ID)
.Select(adgroup => new {
ID = adgroup.Key,
Name = adgroup.FirstOrDefault().Name,
Spent = adgroup.Sum(q => q.Spent),
CampaignID = adgroup.FirstOrDefault().CampaignID,
CampaignName = adgroup.FirstOrDefault().CampaignName,
CampaignBudget = adgroup.FirstOrDefault().CampaignBudget,
})
.GroupBy(q => q.CampaignID)
.Select(campaigngroup => new {
CampaignID = campaigngroup.Key,
DailyBudget = campaigngroup.FirstOrDefault().CampaignBudget,
Consumed = campaigngroup.Sum(q => q.Spent),
RemainningCredit = campaigngroup.FirstOrDefault().CampaignBudget - campaigngroup.Sum(q => q.Spent),
Ads = campaigngroup.Select(ag => new {
ID = ag.ID,
Name = ag.Name,
Spent = ag.Spent
}).OrderBy(q => q.Spent)
})
.Where(q => q.Consumed <= q.DailyBudget).OrderByDescending(q => q.RemainningCredit).First();
There are a few ways you can simplify that query:
select into lets you keep it all in query syntax.
The join ... into/from/DefaultIfMany constructs implementing left joins can be replaced with join ... into construcs representing group joins.
Some of the groups near the end cannot be empty, so FirstOrDefault is unnecessary.
Some of the where conditions can be moved up to the top before the query gets complicated.
Here's the stab I took at it. The revisions were significant, so it might need a little debugging:
var rCampaign = (
from a in db.AdCreative
where a.AdDimensionID == AdSize &&
a.AdGroup.Campaign.Starts <= rNow &&
a.AdGroup.Campaign.Ends >= rNow &&
a.AdGroup.AdHost.Select(q => q.ID).Contains(rPlatform.ID) &&
a.AdGroup.AdPublisher.Select(q => q.ID).Contains(rPublisher.ID)
join hit in db.AdHit.Where(h => h.HitType == 1 && h.LocationID == rLocationID)
on a.ID equals hit.AdID
into hits
join loc in db.AdGroup_Location
on a.AdGroupID equals loc.AdGroupID
into locs
where !locs.Any() || locs.Any(l => l.LocationID == rLocationID)
select new {
a.ID,
a.Name,
Spent = hits.Sum(h => h.AdDimension.Credit / 1000) ?? 0,
CampaignID = a.AdGroup.Campaign.ID,
CampaignName = a.AdGroup.Campaign.Name,
CampaignBudget = a.AdGroup.Campaign.DailyBudget,
} into adgroup
group adgroup by adgroup.CampaignID into campaigngroup
select new
{
CampaignID = campaigngroup.Key,
DailyBudget = campaigngroup.First().CampaignBudget,
Consumed = campaigngroup.Sum(q => q.Spent),
RemainingCredit = campaigngroup.First().CampaignBudget - campaigngroup.Sum(q => q.Spent),
Ads = campaigngroup.Select(ag => new {
ag.ID,
ag.Name,
ag.Spent,
}).OrderBy(q => q.Spent)
} into q
where q.Consumed <= q.DailyBudget
orderby q.RemainingCredit desc)
.First()
I refactored using Query syntax (Not sure if it improved readability). Removed one group by. Made some minor adjustments (replaced FirstOrDefault with Key property, changed Contains to Any). Hopefully it has some effect of speed.
var rCampaign = (from cg in
(from a in db.AdCreative
from subhits in db.AdHit.Where(h => a.ID == h.AdID)
.DefaultIfEmpty()
from subloc in db.AdGroup_Location.Where(l => a.AdGroupID == l.AdGroupID)
.DefaultIfEmpty()
where a.AdGroup.AdHost.Any(q => q.ID == rPlatform.ID) &&
a.AdGroup.AdPublisher.Any(q => q.ID == rPublisher.ID) &&
a.AdDimensionID == AdSize &&
a.AdGroup.Campaign.Starts <= rNow &&
a.AdGroup.Campaign.Ends >= rNow &&
subhits.HitType == 1 &&
(subloc == null || subloc.LocationID == rLocationID)
group new { a, subhits } by new { ID = a.ID, a.Name, CampaignID = a.AdGroup.Campaign.ID, CampaignName = a.AdGroup.Campaign.Name, CampaignBudget = a.AdGroup.Campaign.DailyBudget } into g
select new
{
ID = g.Key.ID,
Name = g.Key.Name,
Spent = g.Sum(x => (x.subhits.AdDimension != null) ? ((double)subhits.AdDimension.Credit / 1000) : 0),
CampaignID = g.Key.CampaignID,
CampaignName = g.Key.CampaignName,
CampaignBudget = g.Key.CampaignBudget
})
group cg by new { cg.CampaignID, cg.CampaignBudget } into cg
let tempConsumed = cg.Sum(q => q.Spent)
let tempRemainningCredit = cg.Key.CampaignBudget - tempConsumed
where tempConsumed <= cg.Key.CampaignBudget
orderby tempRemainningCredit desc
select new
{
CampaignID = cg.Key.CampaignID,
DailyBudget = cg.Key.CampaignBudget,
Consumed = tempConsumed,
RemainningCredit = tempRemainningCredit,
Ads = from ag in cg
orderby ag.Spent
select new
{
ID = ag.ID,
Name = ag.Name,
Spent = ag.Spent
}
}).First();

How to group by and order by, with LINQ, NHibernate and uNHAddins

We're building a WPF application, using Oracle database, also using NHibernate and uNHAddins extensions. In a DataGrid, we're trying to get values from a table, with this query LINQ:
return (from f in rFConsumption.GetAll()
let d = f.CounterDtm
where f.CounterDtm >= dataHoraInicio && f.CounterDtm <= dataHoraFim
group f by (d.Year - 2000) * 384 + d.Month * 32 + d.Day into g
select new RFConsumption
{
COGCounter1 = (g.Sum(f => f.COGCounter1)),
BFCounter1 = (g.Sum(f => f.BFCounter1)),
NatGasCounter1 = (g.Sum(f => f.NatGasCounter1)),
MixGasCounter1 = (g.Sum(f => f.MixGasCounter1)),
COGCounter2 = (g.Sum(f => f.COGCounter2)),
BFCounter2 = (g.Sum(f => f.BFCounter2)),
NatGasCounter2 = (g.Sum(f => f.NatGasCounter2)),
MixGasCounter2 = (g.Sum(f => f.MixGasCounter2)),
COGCounter3 = (g.Sum(f => f.COGCounter3)),
BFCounter3 = (g.Sum(f => f.BFCounter3)),
NatGasCounter3 = (g.Sum(f => f.NatGasCounter3)),
MixGasCounter3 = (g.Sum(f => f.MixGasCounter3)),
}
).ToList<RFConsumption>();
So, my question is:
How I do to use group by, with order by, using NHibernate;
How can i make group by return date data field to that specified group.
You can write the same query with NHibernate with several ways, the most interesting one for me really is the NHibernate QueryOver<>.
So, if your query works fine then, this query should work:
return Session.QueryOver<rFConsumption>()
.Where( fc => (fc.CounterDtm >= dataHoraInicio && fc.CounterDtm <= dataHoraFim))
.SelectList(list => list
.SelectGroup(f => ((f.CounterDtm.Year - 2000) * 384 + f.CounterDtm.Month * 32 + f.CounterDtm.Day)) //use group by
.Select(exp =>
new RFConsumption() //here you define the return data type based on your specified group
{
// exp[0] represents the data returned and grouped by the above statements, so here you can reform it to fit into the form of your new entity
// exp[0] here will be equivilant to g in your query
})
.OrderBy( ee => ee.COGCounter1 ) //order by any of the properties of RFConsumption
.ToList<RFConsumption>();
you should first add the entity RFConsumption:
public calss RFConsumption
{
public int COGCounter1 { get; set; }
public int BFCounter { get; set; }
....
}

LINQ to Entities combine two IQueryable<AnonymousType>

I have 2 queries which work fine:
var q = (from c in _context.Wxlogs
where (SqlFunctions.DatePart("Month", c.LogDate2) == m3) && (SqlFunctions.DatePart("Year", c.LogDate2) == y1)
group c by c.LogDate2
into g
orderby g.Key
let maxTemp = g.Max(c => c.Temp)
let minTemp = g.Min(c => c.Temp)
let maxHum = g.Max(c => c.Humidity)
let minHum = g.Min(c => c.Humidity)
select new
{
LogDate = g.Key,
MaxTemp = maxTemp,
MaxTempTime = g.FirstOrDefault(c => c.Temp == maxTemp).LogTime,
MinTemp = minTemp,
MinTempTime = g.FirstOrDefault(c => c.Temp == minTemp).LogTime,
MaxHum = maxHum,
MaxHumTime = g.FirstOrDefault(c => c.Humidity == maxHum).LogTime,
MinHum = minHum,
MinHumTime = g.FirstOrDefault(c => c.Humidity == minHum).LogTime,
});
var r = (from c in _context.Wxlogs
where
(SqlFunctions.DatePart("Month", c.LogDate2) == m3) &&
(SqlFunctions.DatePart("Year", c.LogDate2) == y1)
group c by c.LogDate2
into g
orderby g.Key
let maxDew = g.Max(c => c.Dew_Point)
let minDew = g.Min(c => c.Dew_Point)
//let maxWind = g.Max(c=> c.Wind_Gust)
let maxRainRate = g.Max(c => c.Rain_rate_now)
let maxPres = g.Max(c => c.Barometer)
let minPres = g.Min(c => c.Barometer)
select new
{
LogDate = g.Key,
MaxRainRateTime = g.FirstOrDefault(c => c.Rain_rate_now == maxRainRate).LogTime,
MaxPres = maxPres,
MaxPresTime = g.FirstOrDefault(c => c.Barometer == maxPres).LogTime,
MinPres = minPres,
MinPresTime = g.FirstOrDefault(c => c.Barometer == minPres).LogTime,
MinDew = minDew,
MinDewTime = g.FirstOrDefault(c => c.Dew_Point == minDew).LogTime,
MaxDew = maxDew,
MaxDewTime = g.FirstOrDefault(c => c.Dew_Point == maxDew).LogTime,
MaxRainRate = maxRainRate,
});
however when I try to combine them using union, in order to output the results to a WPF datgrid:
var result = r.Union(q);
the following error is thrown on the union:
Error 1 Instance argument: cannot convert from 'System.Linq.IQueryable<AnonymousType#1>' to 'System.Linq.ParallelQuery<AnonymousType#2>'
I can't seem to find a way to make this work and any help would be appreciated.
A "union" operation combines two sequences of the same type into a single set (i.e. eliminating all the duplicates). Since you clearly have sequences of two different types, you don't want a union operation. It looks like you want a "concat" operation, which just chains two sequences together. You need something like:
var result = r.Concat<object>(q);
However, since you're using L2E, your query will try to get executed on the server. Since your server won't allow you to combine your two queries (due to mismatched types), you need to execute them separately and then concat the sequences on the client:
var result = r.AsEnumerable().Concat<object>(q.AsEnumerable());
The use of AsEnumerable() runs the queries on the server and brings the results to the client.
Since it turns out that you want to combine the sequences next to each other (i.e. using the same rows in your grid but another group of columns), you actually want a join operation:
var result = from rrow in r.AsEnumerable()
join qrow in q.AsEnumerable() on rrow.LogDate equals qrow.LogDate
select new { rrow.LogDate,
rrow.MaxTemp, rrow.MaxTempTime,
rrow.MinTemp, rrow.MinTempTime,
rrow.MaxHum, rrow.MaxHumTime,
rrow.MinHum, rrow.MinHumTime,
qrow.MaxRainRate, qrow.MaxRainRateTime,
qrow.MaxPres, qrow.MaxPresTime,
qrow.MinPres, qrow.MinPresTime,
qrow.MaxDew, qrow.MaxDewTime,
qrow.MinDew, qrow.MinDewTime };

Resources