Related
I am using EF Core for my .NET Core 2.1 project to interact with Azure SQL Server.
I have query which fetches around 1.5L rows by joining multiple tables. It's taking a long time to return the result and it is getting timed out (more than 30 seconds). So I am going with pagination using skip and take.
So my final call looks like
query.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList();
This call works absolutely fine and give the expected results.
But when I add order by clause to this query:
query.OrderBy(sortField)
.Skip((pageNumber - 1) * pageSize).Take(pageSize).ToList()
Below is the generated SQL
SELECT [p].[Id], [p].[CreatedById], [p].[CreatedOn], [p].[CurrencyId], [p].[CustomerId], [p].[DataMigrationLogId], [p].[FollowUp], [p].[IsActive], [p].[ProjectName], [p].[PlantCode], [p].[ShipToDistanceFromPlant], [p].[StatusId],
[p].[UpdatedById], [p].[UpdatedOn], [p.DataMigrationLog].[Id], [p.DataMigrationLog].[CreatedOn], [p.DataMigrationLog].[GeneratedOn], [p.DataMigrationLog].[HasEdsBid], [p.DataMigrationLog].[HasMbiBid], [p.DataMigrationLog].[Log],
[p.DataMigrationLog].[RequestXml], [p.DataMigrationLog].[Status], [p.DataMigrationLog].[Xq1ProjectId], [p.UpdatedBy].[Id], [p.UpdatedBy].[CellPhone], [p.UpdatedBy].[CreatedOn], [p.UpdatedBy].[Discriminator], [p.UpdatedBy].[Email],
[p.UpdatedBy].[FirstName], [p.UpdatedBy].[HasAccessToDoors], [p.UpdatedBy].[HasAccessToWindows], [p.UpdatedBy].[IsActive], [p.UpdatedBy].[LastName], [p.UpdatedBy].[Prefix], [p.UpdatedBy].[RecentProjectId], [p.UpdatedBy].[UpdatedOn],
[p.UpdatedBy].[WorkPhone], [p.UpdatedBy].[XQ1LoginName], [p.UpdatedBy].[IsSuperAdmin], [p.UpdatedBy].[HasAccessToRestrictedReports], [p.UpdatedBy].[HasEdsProgramAndDealerPriceAccess], [p.UpdatedBy].[IsIss],
[p.UpdatedBy].[ProjectVisibilityId], [p.UpdatedBy].[CustomerId], [p.UpdatedBy].[IsApiUser], [p.CreatedBy].[Id], [p.CreatedBy].[CellPhone], [p.CreatedBy].[CreatedOn], [p.CreatedBy].[Discriminator], [p.CreatedBy].[Email],
[p.CreatedBy].[FirstName], [p.CreatedBy].[HasAccessToDoors], [p.CreatedBy].[HasAccessToWindows], [p.CreatedBy].[IsActive], [p.CreatedBy].[LastName], [p.CreatedBy].[Prefix], [p.CreatedBy].[RecentProjectId], [p.CreatedBy].[UpdatedOn],
[p.CreatedBy].[WorkPhone], [p.CreatedBy].[XQ1LoginName], [p.CreatedBy].[IsSuperAdmin], [p.CreatedBy].[HasAccessToRestrictedReports], [p.CreatedBy].[HasEdsProgramAndDealerPriceAccess], [p.CreatedBy].[IsIss],
[p.CreatedBy].[ProjectVisibilityId], [p.CreatedBy].[CustomerId], [p.CreatedBy].[IsApiUser], [p.Status].[Id], [p.Status].[Code], [p.Status].[Name], [p.Customer].[Id], [p.Customer].[AxCustomerNumber], [p.Customer].[CreatedById],
[p.Customer].[CreatedOn], [p.Customer].[CreditTermId], [p.Customer].[CurrencyId], [p.Customer].[CustomerTypeId], [p.Customer].[DefaultPricing], [p.Customer].[FabricSystemFreight], [p.Customer].[IsActive], [p.Customer].[IsSpecialFreight],
[p.Customer].[LocalityRepId], [p.Customer].[MinFreightCharge], [p.Customer].[CompanyName], [p.Customer].[Notes], [p.Customer].[PrimaryBusinessId], [p.Customer].[ProgramAccountId], [p.Customer].[Prospect],
[p.Customer].[ShippingDollarsThreshold], [p.Customer].[ShippingMilesThreshold], [p.Customer].[SpecialNote], [p.Customer].[SpecialPlantInstruction], [p.Customer].[UpdatedById], [p.Customer].[UpdatedOn],
[p.Customer].[Xq1ProgGroupName], [p.Customer].[Xq1SrNo], [p.Customer].[XqCustomerNumber], [p.Customer.ProgramAccount].[Id], [p.Customer.ProgramAccount].[AxCustomerNumber], [p.Customer.ProgramAccount].[CreatedById],
[p.Customer.ProgramAccount].[CreatedOn], [p.Customer.ProgramAccount].[CreditTermId], [p.Customer.ProgramAccount].[CurrencyId], [p.Customer.ProgramAccount].[CustomerTypeId], [p.Customer.ProgramAccount].[DefaultPricing],
[p.Customer.ProgramAccount].[FabricSystemFreight], [p.Customer.ProgramAccount].[IsActive], [p.Customer.ProgramAccount].[IsSpecialFreight], [p.Customer.ProgramAccount].[LocalityRepId], [p.Customer.ProgramAccount].[MinFreightCharge],
[p.Customer.ProgramAccount].[CompanyName], [p.Customer.ProgramAccount].[Notes], [p.Customer.ProgramAccount].[PrimaryBusinessId], [p.Customer.ProgramAccount].[ProgramAccountId], [p.Customer.ProgramAccount].[Prospect],
[p.Customer.ProgramAccount].[ShippingDollarsThreshold], [p.Customer.ProgramAccount].[ShippingMilesThreshold], [p.Customer.ProgramAccount].[SpecialNote], [p.Customer.ProgramAccount].[SpecialPlantInstruction],
[p.Customer.ProgramAccount].[UpdatedById], [p.Customer.ProgramAccount].[UpdatedOn], [p.Customer.ProgramAccount].[Xq1ProgGroupName], [p.Customer.ProgramAccount].[Xq1SrNo], [p.Customer.ProgramAccount].[XqCustomerNumber],
[p.Customer.CustomerType].[Id], [p.Customer.CustomerType].[Code], [p.Customer.CustomerType].[Name]
FROM [Projects] AS [p]
LEFT JOIN [DataMigrationLogs] AS [p.DataMigrationLog] ON [p].[DataMigrationLogId] = [p.DataMigrationLog].[Id]
INNER JOIN [Security].[XQUsers] AS [p.UpdatedBy] ON [p].[UpdatedById] = [p.UpdatedBy].[Id]
INNER JOIN [Security].[XQUsers] AS [p.CreatedBy] ON [p].[CreatedById] = [p.CreatedBy].[Id]
INNER JOIN [ProjectStatuses] AS [p.Status] ON [p].[StatusId] = [p.Status].[Id]
LEFT JOIN [Customers] AS [p.Customer] ON [p].[CustomerId] = [p.Customer].[Id]
LEFT JOIN (
SELECT [p.Customer.LocalityRep].*
FROM [Security].[XQUsers] AS [p.Customer.LocalityRep]
WHERE [p.Customer.LocalityRep].[Discriminator] = N'SALES_SP'
) AS [t] ON [p.Customer].[LocalityRepId] = [t].[Id]
LEFT JOIN [Customers] AS [p.Customer.ProgramAccount] ON [p.Customer].[ProgramAccountId] = [p.Customer.ProgramAccount].[Id]
LEFT JOIN (
SELECT [p.Customer.ProgramAccount.LocalityRep].*
FROM [Security].[XQUsers] AS [p.Customer.ProgramAccount.LocalityRep]
WHERE [p.Customer.ProgramAccount.LocalityRep].[Discriminator] = N'SALES_SP'
) AS [t0] ON [p.Customer.ProgramAccount].[LocalityRepId] = [t0].[Id]
LEFT JOIN [CustomerTypes] AS [p.Customer.CustomerType] ON [p.Customer].[CustomerTypeId] = [p.Customer.CustomerType].[Id]
WHERE ([p.UpdatedBy].[Discriminator] IN (N'INT_SYSADMMIN', N'XqInternalUser', N'EXT', N'SALES_SP', N'XqUser') AND [p.CreatedBy].[Discriminator] IN (N'INT_SYSADMMIN', N'XqInternalUser', N'EXT', N'SALES_SP', N'XqUser')) AND (EXISTS (
SELECT 1
FROM [Security].[BusinessUserRegions] AS [r]
WHERE [r].[RegionId] IN (CAST(6 AS bigint), CAST(8 AS bigint), CAST(9 AS bigint)) AND ([t].[Id] = [r].[BusinessUserId])) OR ([p.Customer].[ProgramAccountId] IS NOT NULL AND EXISTS (
SELECT 1
FROM [Security].[BusinessUserRegions] AS [r0]
WHERE [r0].[RegionId] IN (CAST(6 AS bigint), CAST(8 AS bigint), CAST(9 AS bigint)) AND ([t0].[Id] = [r0].[BusinessUserId]))))
ORDER BY [p].[CreatedOn]
It takes more than 30 seconds to execute and Azure SQL Server has query timeout of maximum 30 second, so it results in exception
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
From my layman's understanding about SQL I think SQL us fetching the entire dataset for sorting and then trying to do pagination and hence it's taking more than 30 seconds. Please correct me if I am wrong.
I need to sort the result in query in self otherwise it will not provide the desired result set.
Is their any way to deal with this situation.
Any help is highly appreciated.
I have written a SQL query that joins two tables and calculates the difference between two variables. I want a while loop iterate through the code and do the procedure for a list of organizations numbers orgnr.
Use Intra;
drop table #Tabell1
go
select Månad = A.Manad, Intrastat = A.varde, Moms = B.vardeutf
into #Tabell1
From
IntrastatFsum A
join
Momsuppg B
on A.Orgnr = B.Orgnr
where A.Orgnr = 165564845492
AND A.Ar = 2017
AND B.Ar = A.AR
--AND A.Manad = 11
AND A.Manad = B.Manad
AND A.InfUtf = 'U'
order by A.Manad
select *, Differens = (Intrastat - Moms)/ Moms from #Tabell1
Where do I put the while loop and how should I write it? I don't have a lot of experience with SQL so any help is appreciated.
wasnt clear at all when posting this question was in a big hurry, so sorry guys for that. But what im trying to do is:
This code just runs trough some data, calculates the difference in % for a special company. Each month we get a list of companies that show high standard deviation for some variables. So we have to go through them and compare the reported value with the taxreturn. What im trying to do is write a code that compares the values for "Intra" which is the reported value and " Moms" which is reported tax value. That part i have already done what i need to do now is to insert a loop into the code that goes through a list instead where i have stored the orgnr for companies that show high std for some variables.I want it to keep a list of the companies where the difference between the reported values and taxreturn is high, so i can take an extra look at them. Because sometimes the taxreturn and reported value to ous is almost the same, so i try to remove the most obvious cases with automation
I don't think you need a loop in your query.
Try changing this where-clause:
where A.Orgnr = 165564845492
To this:
where A.Orgnr in (165564845492, 123, 456, 678)
just remove A.Orgnr = 165564845492 in where clause, there is no need of loop
Use Intra;
drop table #Tabell1
go
select Månad = A.Manad, Intrastat = A.varde,Moms = B.vardeutf
into #Tabell1
From
IntrastatFsum A
join
Momsuppg B
on A.Orgnr = B.Orgnr
where A.Ar = 2017
AND B.Ar = A.AR
--AND A.Manad = 11
AND A.Manad = B.Manad
AND A.InfUtf = 'U'
order by A.Manad
select *, Differens = (Intrastat - Moms)/ Moms from #Tabell1
Question is not clear (to me)
select Månad = A.Manad, Intrastat = A.varde, Moms = B.vardeutf
, Differens = (A.varde - B.vardeutf) / B.vardeutf
From
IntrastatFsum A
join
Momsuppg B
on A.Orgnr = B.Orgnr
AND A.Orgnr in (165564845492, ...)
AND A.Ar = 2017
AND A.Manad = B.Manad
AND A.InfUtf = 'U'
order by A.Manad
I am trying to create a MySQL statement that I can put in a php script to update the weight on a few thousand products in magento 1.9.
This is the statement I currently have:
UPDATE dp_catalog_product_entity_decimal AS ped JOIN dp_eav_attribute AS ea ON ea.entity_type_id = 10 AND ea.attribute_code = 'weight' AND ped.attribute_id = ea.attribute_id SET ped.value = 8 WHERE ped.entity_id = P1000X3;
It was partially taken from another post so I am not sure if it will work at all but I currently have the error "#1054 - Unknown column 'P1000X3' in 'where clause'"
I am not that great with sql joins and I dont really know the magento databse at all so any help to get this statement to work is much apreciated.
Thanks.
Matt
The correct SQL to update the weight of a product with a specific SKU in Magento is this (replace YOUR-MAGENTO-SKU with the sku of the item you want to update, and the 8 with the weight value).
UPDATE catalog_product_entity_decimal AS cped
JOIN eav_attribute AS ea ON ea.entity_type_id = (SELECT entity_type_id FROM eav_entity_type WHERE entity_type_code = 'catalog_product')
LEFT JOIN catalog_product_entity AS cpe ON cpe.entity_id = cped.entity_id
AND ea.attribute_code = 'weight'
AND cped.attribute_id = ea.attribute_id
SET cped.value = 8
WHERE cpe.sku = 'YOUR-MAGENTO-SKU';
In your case, you have the table prefix dp_, and the product sku P1000X3, so the correct SQL for you would be:
UPDATE dp_catalog_product_entity_decimal AS cped
JOIN dp_eav_attribute AS ea ON ea.entity_type_id = (SELECT entity_type_id FROM dp_eav_entity_type WHERE entity_type_code = 'catalog_product')
LEFT JOIN dp_catalog_product_entity AS cpe ON cpe.entity_id = cped.entity_id
AND ea.attribute_code = 'weight'
AND cped.attribute_id = ea.attribute_id
SET cped.value = 8
WHERE cpe.sku = 'P1000X3';
After running this, be sure to reindex your Magento Indexes.
Do your Magento tables have the prefix dp_? Make sure of this.
Also on this part:
WHERE ped.entity_id = P1000X3;
ped.entity_id will be an integer value (number).
I'm not sure where you got P1000X3, but using that won't work (it's a string value). Even so, strings should be wrapped with a single quotes ', like this:
'P1000X3';
I did ask this question before but deleted the post.
Here is my question: I have a table in SQL Server with multiple events in the table, I am trying to filter on the events and the datetime between the events on a certain X min.
I need help with T-SQL queries.
What can use use to do the following (any links will be appreciated):
take event_id and datetime of device, compare to a list of other event_id's to see if there are any other events withing the last 24 min for that device.
What would work for me? Nested queries? Where clause? CTE?
I have tried 6 queries but not getting the result I need, do I use (max) datetime and then compare on event_ID's?
I know this is not much but any help or links would be appreciated....
On Request (Full Query no changes)
1.
SELECT A.[Unit_id]
,A.[TransDate]
,A.[event_id]
,A.[event_msg]
,B.[TransDate]
,B.[event_id]
,B.[event_msg]
FROM
(Select
A.[Unit_id]
,MAX(A.[TransDate]) AS Transdate
,A.[event_id]
,A.[event_msg]
FROM [JammingEvents].[dbo].[EventLogExtended] AS A
WHERE event_id = '345'
GROUP BY A.[Unit_id]
,A.[event_id]
,A.[event_msg]
) AS A
INNER JOIN
(SELECT
B.[Unit_id]
,MAX(B.[TransDate]) AS Transdate
,B.[event_id]
,B.[event_msg]
FROM [JammingEvents].[dbo].[EventLogExtended] AS B
WHERE B.event_id = '985'
GROUP BY B.[Unit_id]
,B.[event_id]
,B.[event_msg]
) AS B
ON
A.Unit_id = B.Unit_id
WHERE (B.TransDate BETWEEN DATEADD(MINUTE,-24,A.Transdate) AND DATEADD(MINUTE,24,A.Transdate))
this works but only if you compare 2 event Id's. the problem is i need to do this on a bulk scale.
2.
SET NOCOUNT ON
GO
SELECT MAX(ELE.Transdate) AS Transdate
,ELE.[Unit_id]
,ELE.[event_id]
,ELE.[event_msg]
,ELE.[Latitude]
,ELE.[Longitude]
,ELE.[date_ack]
,ELE.[user_id_ack]
,ELE.[date_closed]
,ELE.[user_id_closed]
,ELE.[action]
,ELE.[EventSeqNo]
,ELE.[MsgSeqNo]
,ELE.[Speed]
,ELE.[Heading]
,ELE.[Status1]
,ELE.[Status2]
,ELE.[GeoLocation]
,ELE.[Reg_No]
,ELE.[Company]
,ELE.[Fleet_Code]
,ELE.[IgnStatus]
,ELE.[comboActioned]
FROM
[JammingEvents].[dbo].[EventLogExtended] ELE
INNER JOIN
(
SELECT
F.Transdate ,F.[Unit_id] ,F.[event_id] ,F.[event_msg] ,F.[Latitude] ,F.[Longitude] ,F.[date_ack] ,F.[user_id_ack]
,F.[date_closed] ,F.[user_id_closed] ,F.[action] ,F.[EventSeqNo] ,F.[MsgSeqNo] ,F.[Speed] ,F.[Heading] ,F.[Status1]
,F.[Status2] ,F.[GeoLocation] ,F.[Reg_No] ,F.[Company] ,F.[Fleet_Code] ,F.[IgnStatus] ,F.[comboActioned]
FROM [JammingEvents].[dbo].[EventLogExtended] AS F
WHERE
F.event_id IN
('302'
,'303'
,'304'
,'305'
,'309'
,'340'
,'341'
,'345'
,'962'
,'963'
,'973'
,'974'
,'975'
,'976'
,'985'
,'987'
,'989'
,'C220'
,'C222'
,'C224'
,'C227'
,'C228')
) AS A
ON
ELE.Unit_id = A.Unit_id
INNER JOIN
(
SELECT
G.Transdate ,G.[Unit_id] ,G.[event_id] ,G.[event_msg] ,G.[Latitude] ,G.[Longitude] ,G.[date_ack]
,G.[user_id_ack] ,G.[date_closed] ,G.[user_id_closed] ,G.[action] ,G.[EventSeqNo] ,G.[MsgSeqNo] ,G.[Speed] ,G.[Heading]
,G.[Status1] ,G.[Status2] ,G.[GeoLocation] ,G.[Reg_No] ,G.[Company] ,G.[Fleet_Code] ,G.[IgnStatus] ,G.[comboActioned]
FROM [JammingEvents].[dbo].[EventLogExtended] AS G
WHERE
G.event_id = '345'
) AS B
ON
A.Unit_id = B.Unit_id
WHERE (ELE.TransDate BETWEEN DATEADD(MINUTE,-24,A.Transdate) AND DATEADD(MINUTE,24,A.Transdate))
AND
(ELE.event_id IN
(
'302'
,'303'
,'304'
,'305'
,'309'
,'340'
,'341'
,'345'
,'962'
,'963'
,'973'
,'974'
,'975'
,'976'
,'985'
,'987'
,'989'
,'C220'
,'C222'
,'C224'
,'C227'
,'C228'
))
AND
(ELE.action = '0')
GROUP BY ELE.TransDate
,A.Unit_id
,ELE.[Unit_id]
,ELE.[event_id]
,ELE.[event_msg]
,ELE.[Latitude]
,ELE.[Longitude]
,ELE.[date_ack]
,ELE.[user_id_ack]
,ELE.[date_closed]
,ELE.[user_id_closed]
,ELE.[action]
,ELE.[EventSeqNo]
,ELE.[MsgSeqNo]
,ELE.[Speed]
,ELE.[Heading]
,ELE.[Status1]
,ELE.[Status2]
,ELE.[GeoLocation]
,ELE.[Reg_No]
,ELE.[Company]
,ELE.[Fleet_Code]
,ELE.[IgnStatus]
,ELE.[comboActioned]
ORDER BY ELE.TransDate, ELE.Unit_id DESC
so far this got me the closest but i have no idea what to do, i am very poor with SQL syntax and writing queries.
You could use an exists subquery to demand that there was another event for the same device within the preceeding 24 minutes:
select *
from Events e1
where exists
(
select *
from Events e2
where e2.Device_ID = e1.Device_ID
and e2.event_dt between
dateadd(minute, e1.event_dt, -24)
and e1.event_dt
)
Self join should work as well:
SELECT *, DATEDIFF(minute,b.transdate,a.transdate) diff FROM EventLogWxtended a
JOIN eventLogExtended b
ON (a.unit_id=b.unit_id AND b.transdate<a.transdate
AND DATEDIFF(minute,b.transdate,a.transdate)<24)
Is there a better way to write the following simple SQL Server 2005 update statement? This just seems a bit messy inefficient.
UPDATE QuotationItem
SET Recurring_Cost =
(SELECT TOP (1) Recurring_Cost
FROM Products
WHERE (Remote_ID = QuotationItem.Product_ID))
WHERE (Quotation_ID = 115)
Thanks,
Nick
How About using a join
UPDATE QuotationItem
SET Recurring_Cost = p.recurring_cost
FROM QuotationItem q join Products p on q.Product_ID = p.Remote_ID
WHERE q.Quotation_ID = 115
Is your TOP 1 really needed? If it is, since you don't specify an ordering, you've got pretty random results from your query anyway! If it is not really necessary, this will do:
UPDATE q
SET Recurring_Cost = p.RecurringCost
FROM QuotationItem q
INNER JOIN
Products p
ON p.Remote_ID = q.Product_ID
WHERE q.Quotation_ID = 115