I've been attempting to write a peewee select query which results in a table with 2 counts (one for the number of prizes associated with the lottery, and the for the number of packages associated with the lottery), as well as the fields in the Lottery model.
I've managed to write select queries with 1 count working (seen below), and then I've had to convert the ModelSelects to lists and join them manually (which I think is very hacky).
I did manage to write a select query where the results were joined, but it would multiply the packages count with the prizes count (I've since lost that query).
I also tried using a .switch(Lottery) but I didn't have any luck with this.
query1 = (Lottery.select(Lottery,fn.count(Package.id).alias('packages'))
.join(LotteryPackage)
.join(Package)
.order_by(Lottery.id)
.group_by(Lottery)
.dicts())
query2 = (Lottery.select(Lottery.id.alias('lotteryID'), fn.count(Prize.id).alias('prizes'))
.join(LotteryPrize)
.join(Prize)
.group_by(Lottery)
.order_by(Lottery.id)
.dicts())
lottery = list(query1)
query3 = list(query2)
for x in range(len(lottery)):
lottery[x]['prizes'] = query3[x]['prizes']
While the above code works, is there a cleaner way to write this query?
Your best bet is to do this with subqueries.
# Create query which gets lottery id and count of packages.
L1 = Lottery.alias()
subq1 = (L1
.select(L1.id, fn.COUNT(LotteryPackage.package).alias('packages'))
.join(LotteryPackage, JOIN.LEFT_OUTER)
.group_by(L1.id))
# Create query which gets lottery id and count of prizes.
L2 = Lottery.alias()
subq2 = (L2
.select(L2.id, fn.COUNT(LotteryPrize.prize).alias('prizes'))
.join(LotteryPrize, JOIN.LEFT_OUTER)
.group_by(L2.id))
# Select from lottery, joining on each subquery and returning
# the counts.
query = (Lottery
.select(Lottery, subq1.c.packages, subq2.c.prizes)
.join(subq1, on=(Lottery.id == subq1.c.id))
.join(subq2, on=(Lottery.id == subq2.c.id))
.order_by(Lottery.name))
for row in query.objects():
print(row.name, row.packages, row.prizes)
Related
This is on Windows SQL Server Cluster.
Query is coming from 3rd party application so I can not modify the query permanently.
Query is:
DECLARE #FromBrCode INT = 1001
DECLARE #ToBrCode INT = 1637
DECLARE #Cdate DATE = '31-mar-2017'
SELECT
a.PrdCd, a.Name, SUM(b.Balance4) as Balance
FROM
D009021 a, D010014 b
WHERE
a.PrdCd = LTRIM(RTRIM(SUBSTRING(b.PrdAcctId, 1, 8)))
AND substring(b.PrdAcctId, 9, 24) = '000000000000000000000000'
AND a.LBrCode = b.LBrCode
AND a.LBrCode BETWEEN #FromBrCode AND #ToBrCode
AND b.CblDate = (SELECT MAX(c.CblDate)
FROM D010014 c
WHERE c.PrdAcctId = b.PrdAcctId
AND c.LBrCode = b.LBrCode
AND c.CblDate <= #Cdate)
GROUP BY
a.PrdCd, a.Name
HAVING
SUM(b.Balance4) <> 0
ORDER BY
a.PrdCd
This particular query is taking too much time to complete execution. The same problem happens on a different SQL Server.
No table lock was found, processor and memory usage is normal while the query is running.
Normal "select top 1000" working and showing output instantly in both tables (D009021, D010014)
Reindex and rebuild / update stats done in both tables but problem did not resolve (D009021, D010014)
The same query is working if we reduce number of branch but slowly
(
DECLARE #FromBrCode INT =1001
DECLARE #ToBrCode INT =1001
)
The same query is working faster giving output within 2 mins if we replace any one variable and use the value directly
AND a.LBrCode BETWEEN #FromBrCode AND #ToBrCode
changed to
AND a.LBrCode BETWEEN 1001 AND #ToBrCode
The same query is working faster and giving output within 2 mins if we add "OPTION (RECOMPILE)" at end
I tried to clean cache query execution plan and optimized new one but problem still exists
Found that the query estimate plan and actual execution plan are different (see screenshots)
Table D010014 is aliased twice once as b and once as c
the they are joined to the same table.
Try toto remove the sub query below and create a temp table to store
the values you need. I added * to the fields you self join
SELECT MAX(c.CblDate)
FROM D010014 c
WHERE c.PrdAcctId = b.PrdAcctId
AND c.LBrCode = b.LBrCode
AND c.CblDate <= #Cdate
if you cant do that then try
SELECT TOP 1 c.CblDate
FROM D010014 c
WHERE c.PrdAcctId = b.PrdAcctId
AND c.LBrCode = b.LBrCode
AND c.CblDate <= #Cdate
ORDER BY c.CblDate DESC
I'm currently using JPQL queries to retrieve information from a database. The purpose of the project is testing a sample environments through randomness with different elements so I need the queries to retrieve random single results all through the project.
I am facing that JPQL does not implement a proper function for random retrieval and postcalculation of random takes too long (14 seconds for the attached function to return a random result)
public Player getRandomActivePlayerWithTransactions(){
List<Player> randomPlayers = entityManager.createQuery("SELECT pw.playerId FROM PlayerWallet pw JOIN pw.playerId p"
+ " JOIN p.gameAccountCollection ga JOIN ga.iDAccountStatus acs"
+ " WHERE (SELECT count(col.playerWalletTransactionId) FROM pw.playerWalletTransactionCollection col) > 0 AND acs.code = :status")
.setParameter("status", "ACTIVATED")
.getResultList();
return randomPlayers.get(random.nextInt(randomPlayers.size()));
}
As ORDER BY NEWID() is not allowed because of JPQL restrictions I have tested the following inline conditions, all of them returned with syntax error on compilation.
WHERE (ABS(CAST((BINARY_CHECKSUM(*) * RAND()) as int)) % 100) < 10
WHERE Rnd % 100 < 10
FROM TABLESAMPLE(10 PERCENT)
Have you consider to generate a random number and skip to that result?
I mean something like this:
String q = "SELECT COUNT(*) FROM Player p";
Query query=entityManager.createQuery(q);
Number countResult=(Number) query.getSingleResult();
int random = Math.random()*countResult.intValue();
List<Player> randomPlayers = entityManager.createQuery("SELECT pw.playerId FROM PlayerWallet pw JOIN pw.playerId p"
+ " JOIN p.gameAccountCollection ga JOIN ga.iDAccountStatus acs"
+ " WHERE (SELECT count(col.playerWalletTransactionId) FROM pw.playerWalletTransactionCollection col) > 0 AND acs.code = :status")
.setParameter("status", "ACTIVATED")
.setFirstResult(random)
.setMaxResults(1)
.getSingleResult();
I have figured it out. When retrieving the player I was also retrieving other unused related entity and all the entities related with that one and so one.
After adding fetch=FetchType.LAZY (don't fetch entity until required) to the problematic relation the performance of the query has increased dramatically.
I have the below tables in my DB.
Orders:
OrderNo.........ItemNo........OrderQty
1000________10_________10
2000________20_________10
VendorPO:
OrderNo.........ItemNo........VendorPO........POQty
1000________10_________100__________5
2000________20_________100__________7
2000________20_________200__________3
And I used this Query:
SELECT Order.OrderNo, Order.ItemNo, Order.OrderQty, VendorPO.VendorPO, VendorPO.POQty
FROM [Order]
INNER JOIN VendorPO ON (Order.ItemNo = VendorPO.ItemNo)
AND (Order.OrderNo = VendorPO.OrderNo);
With these results:
Query Result
OrderNo.........ItemNo........OrderQty........VendorPO........POQty
1000________10__________10_________100__________5
2000________20__________10_________100__________7
2000________20__________10_________200__________3
I want to avoid repetition of the quantity in the query result, where Item quantity 10 is repeated twice against two PO reference with POQty 3 and 7.
I appreciate your support on finding a way to avoid order quantity repetition.
This query takes too much time, so I try to optimize it. Do you have any idea or suggestion ?
I tried with fulltext on a procedure and a while loop ... it gets worst ( dbo.url has more than 100 000 lines ; dbo.url where status = 'tocheck' only 1000)
select tocheck.*
from dbo.url tocheck inner join dbo.url done
on tocheck.id != done.id
and tocheck.url like done.url+'%'
and done.status in ('tocheck','todo','done')
where tocheck.status = 'tocheck'
Edit :
I call a webservice multiple times with different urls :
urls look like http://ws.com/query?p1=a&p2=b (url1).
If I already called url http://ws.com/query?p1=a (url2), i don't want to call url1 cause :
url1 like url2+'%'
Thanks for your help.
Edit2 :
I add a column suburl that contains 'query?p1=a' for each url and modify the query :
select tocheck.*
from dbo.url tocheck inner join dbo.url done
on tocheck.id != done.id
and tocheck.suburl = done.suburl --NEW
and tocheck.url like done.url+'%'
and done.status in ('tocheck','todo','done')
where tocheck.status = 'tocheck'
More than 10 times shorter ... Phew !!
I think because of joining the table to itself through ids not equal there is much overhead as this is a cartesian product only excluding self joins for same id.
I suggest trying with a subquery. Then the outer query returns only 1000 (as you mentioned) tochecks whereas the subquery additionally excludes urls starting with the same characters:
select
tocheck.*
from
dbo.url tocheck
where
tocheck.status = 'tocheck'
and
tocheck.id not in (
select
done.id
from
dbo.url done
where
tocheck.url like done.url+'%'
and
done.status in ('tocheck','todo','done')
)
I have a LINQ to SQL query:
from at in Context.Transaction
select new {
at.Amount,
at.PostingDate,
Details =
from tb in at.TransactionDetail
select new {
Amount = tb.Amount,
Description = tb.Desc
}
}
This results in one SQL statement being executed. All is good.
However, if I attempt to return known types from this query, even if they have the same structure as the anonymous types, I get one SQL statement executed for the top level and then an additional SQL statement for each "child" set.
Is there any way to get LINQ to SQL to issue one SQL statement and use known types?
EDIT: I must have another issue. When I plugged a very simplistic (but still hieararchical) version of my query into LINQPad and used freshly created known types with just 2 or 3 members, I did get one SQL statement. I will post and update when I know more.
EDIT 2: This appears to be due to a bug in Take. See my answer below for details.
First - some reasoning for the Take bug.
If you just Take, the query translator just uses top. Top10 will not give the right answer if cardinality is broken by joining in a child collection. So the query translator doesn't join in the child collection (instead it requeries for the children).
If you Skip and Take, then the query translator kicks in with some RowNumber logic over the parent rows... these rownumbers let it take 10 parents, even if that's really 50 records due to each parent having 5 children.
If you Skip(0) and Take, Skip is removed as a non-operation by the translator - it's just like you never said Skip.
This is going to be a hard conceptual leap to from where you are (calling Skip and Take) to a "simple workaround". What we need to do - is force the translation to occur at a point where the translator can't remove Skip(0) as a non-operation. We need to call Skip, and supply the skipped number at a later point.
DataClasses1DataContext myDC = new DataClasses1DataContext();
//setting up log so we can see what's going on
myDC.Log = Console.Out;
//hierarchical query - not important
var query = myDC.Options.Select(option => new{
ID = option.ParentID,
Others = myDC.Options.Select(option2 => new{
ID = option2.ParentID
})
});
//request translation of the query! Important!
var compQuery = System.Data.Linq.CompiledQuery
.Compile<DataClasses1DataContext, int, int, System.Collections.IEnumerable>
( (dc, skip, take) => query.Skip(skip).Take(take) );
//now run the query and specify that 0 rows are to be skipped.
compQuery.Invoke(myDC, 0, 10);
This produces the following query:
SELECT [t1].[ParentID], [t2].[ParentID] AS [ParentID2], (
SELECT COUNT(*)
FROM [dbo].[Option] AS [t3]
) AS [value]
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY [t0].[ID]) AS [ROW_NUMBER], [t0].[ParentID]
FROM [dbo].[Option] AS [t0]
) AS [t1]
LEFT OUTER JOIN [dbo].[Option] AS [t2] ON 1=1
WHERE [t1].[ROW_NUMBER] BETWEEN #p0 + 1 AND #p1 + #p2
ORDER BY [t1].[ROW_NUMBER], [t2].[ID]
-- #p0: Input Int (Size = 0; Prec = 0; Scale = 0) [0]
-- #p1: Input Int (Size = 0; Prec = 0; Scale = 0) [0]
-- #p2: Input Int (Size = 0; Prec = 0; Scale = 0) [10]
-- Context: SqlProvider(Sql2005) Model: AttributedMetaModel Build: 3.5.30729.1
And here's where we win!
WHERE [t1].[ROW_NUMBER] BETWEEN #p0 + 1 AND #p1 + #p2
I've now determined this is the result of a horrible bug. The anonymous versus known type turned out not to be the cause. The real cause is Take.
The following result in 1 SQL statement:
query.Skip(1).Take(10).ToList();
query.ToList();
However, the following exhibit the one sql statement per parent row problem.
query.Skip(0).Take(10).ToList();
query.Take(10).ToList();
Can anyone think of any simple workarounds for this?
EDIT: The only workaround I've come up with is to check to see if I'm on the first page (IE Skip(0)) and then make two calls, one with Take(1) and the other with Skip(1).Take(pageSize - 1) and addRange the lists together.
I've not had a chance to try this but given that the anonymous type isn't part of LINQ rather a C# construct I wonder if you could use:
from at in Context.Transaction
select new KnownType(
at.Amount,
at.PostingDate,
Details =
from tb in at.TransactionDetail
select KnownSubType(
Amount = tb.Amount,
Description = tb.Desc
)
}
Obviously Details would need to be an IEnumerable collection.
I could be miles wide on this but it might at least give you a new line of thought to pursue which can't hurt so please excuse my rambling.