Currently I have a Joomla 3.6.2 news website. Here we display news etc etc.
Now daily we upload tons or articles, we have a login system for paid subscribers.
The problem is that I have some "long slow queries' running which is eating up my resources.
I have 4 CPU cores 3.0 GHz and 7GB of RAM. Every few seconds the CPU and RAM kick up to a 100% and then drop slowly again. Which results in a 503 on the website.
As you can see this is quite annoying. Now my host told me that the following queries are causing this issue;
1. Executed 3h 55m 25s ago for 47.63166 sec on Database --> *********
Date: 2016-10-11 05:25:56 Query_time: 47.631660 Rows_examined: 179766: Rows_sent 10757 Lock_time: 0.000539
SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out,
a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias,
CASE WHEN a.modified = '0000-00-00 00:00:00'
THEN a.created ELSE a.modified END as modified, a.modified_by,
uam.name as modified_by_name,
CASE WHEN a.publish_up = '0000-00-00 00:00:00'
THEN a.created ELSE a.publish_up END as publish_up,
a.publish_down, a.images, a.urls, a.attribs, a.metadata,
a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured,
a.language, LENGTH(a.fulltext) AS readmore,
CASE WHEN badcats.id is not null THEN 0 ELSE a.state END AS state,
c.title AS category_title, c.path AS category_route,
c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_published FROM sxm_content AS a LEFT JOIN sxm_categories AS c ON c.id = a.catid LEFT JOIN sxm_users AS ua ON ua.id = a.created_by LEFT JOIN sxm_users AS uam ON uam.id = a.modified_by LEFT JOIN sxm_categories as parent ON parent.id = c.parent_id LEFT JOIN sxm_content_rating AS v ON a.id = v.content_id LEFT OUTER JOIN (SELECT cat.id as id FROM sxm_categories AS cat JOIN sxm_categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.id WHERE a.access IN (1,1,5) AND c.access IN (1,1,5) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2016-10-11 10:25:09') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2016-10-11 10:25:09') ORDER BY c.lft, CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created;
I am quite new when it comes to these queries and correct me if I am wrong, I see that the query is trying to pull info from the articles table.
I am not sure why it would go through the whole table, I don't display all the articles on 1 page, nor do I have pagination where all articles from 1 category or more are being displayed.
This just happened out of the blue. I didn't make any changes to the website, no updating. Only thing I installed was a SSL Certificate.
I am also using CloudFlare as my CDN which takes quite a big load of the web-server. I don't use the Joomla cache but instead I use JOTCache simply because of the fact that I can exclude certain position on the website that I don't want cached.
We have around the 5k unique visitors per day.
Related
I need to join these two queries ....please help
SELECT P.EMAIL_ADDRESS, ECS.FST_NAME, ECS.LAST_NAME, ECS.COMPANY_NAME, UPPER(P.URL) AS URL,
ECS.COUNTRY, ECS.PER_REGION, ECS.EEA_EMAILABLE_FLG, ECS.PHONEABLE_FLG, ECS.MAILABLE_FLG--INCLUDING THESE FOR INFO ONLY, NOT REQUESTED
--CAST(P.ACTIVITYDATE AS DATE) PGVIST_DATE
FROM [OPSODSADM].[DBO].[ELQR_PAGEVISIT] P WITH (NOLOCK)
JOIN [OPSSOAADM].[DBO].[ELOQUA_CONTACT_STAGE] ECS WITH (NOLOCK) ON P.EMAIL_ADDRESS = ECS.EMAIL_ADDRESS --JOINING TO GET NAME AND COMPANY FIELDS
WHERE ACTIVITYDATE >= '2022-01-01' --STARTING DATE INCLUDED IN MARS REQUEST
AND CHARINDEX('GARTNER.COM', UPPER(P.EMAIL_ADDRESS),1) = 0 --DON'T WANT GARTNER CONTACTS
AND CHARINDEX('RAINFOCUS', UPPER(P.EMAIL_ADDRESS),1) = 0 --DON'T WANT RAINFOCUS CONTACTS
AND (ECS.EVT_EXCL_FLG IS NULL OR ECS.EVT_EXCL_FLG = 'N') --DON'T WANT TO INCLUDE CONTACTS WITH ACTIVE/CURRENT REGISTRATIONS
AND ECS.PER_REGION = 'NA'
AND (UPPER(URL) = 'HTTPS://WWW.GARTNER.COM/EN/CONFERENCES/NA/CFO-FINANCE-US/REGISTER')
--AND CHARINDEX('FES22/RFR', UPPER(URL),1) > 0 --RUN TO SEE ALL VARIATIONS OF RF PAGES BUT ONLY THE RFR/ORDER URL IS WHAT WE TARGET FOR CART ABANDONS
GROUP BY P.EMAIL_ADDRESS, ECS.FST_NAME, ECS.LAST_NAME, ECS.COMPANY_NAME, UPPER(P.URL),
ECS.COUNTRY, ECS.PER_REGION, ECS.EEA_EMAILABLE_FLG, ECS.PHONEABLE_FLG, ECS.MAILABLE_FLG
----ORDER BY P.EMAIL_ADDRESS
SELECT P.EMAIL_ADDRESS, ECS.FST_NAME, ECS.LAST_NAME, ECS.COMPANY_NAME, UPPER(P.URL) AS URL,
ECS.COUNTRY, ECS.PER_REGION, ECS.EEA_EMAILABLE_FLG, ECS.PHONEABLE_FLG, ECS.MAILABLE_FLG --INCLUDING THESE FOR INFO ONLY, NOT REQUESTED
--CAST(P.ACTIVITYDATE AS DATE) PGVIST_DATE
FROM [OPSODSADM].[DBO].[ELQR_PAGEVISIT] P WITH (NOLOCK)
JOIN [OPSSOAADM].[DBO].[ELOQUA_CONTACT_STAGE] ECS WITH (NOLOCK) ON P.EMAIL_ADDRESS = ECS.EMAIL_ADDRESS --JOINING TO GET NAME AND COMPANY FIELDS
WHERE ACTIVITYDATE >= '2022-01-01' --STARTING DATE INCLUDED IN MARS REQUEST
AND CHARINDEX('GARTNER.COM', UPPER(P.EMAIL_ADDRESS),1) = 0 --DON'T WANT GARTNER CONTACTS
AND CHARINDEX('RAINFOCUS', UPPER(P.EMAIL_ADDRESS),1) = 0 --DON'T WANT RAINFOCUS CONTACTS
AND (ECS.EVT_EXCL_FLG IS NULL OR ECS.EVT_EXCL_FLG = 'N') --DON'T WANT TO INCLUDE CONTACTS WITH ACTIVE/CURRENT REGISTRATIONS
AND ECS.PER_REGION = 'NA'
AND (UPPER(URL) = 'HTTPS://REG.GARTNER.COM/FLOW/GARTNER/FES22/RFR/ORDER')
--AND CHARINDEX('FES22/RFR', UPPER(URL),1) > 0 --RUN TO SEE ALL VARIATIONS OF RF PAGES BUT ONLY THE RFR/ORDER URL IS WHAT WE TARGET FOR CART ABANDONS
GROUP BY P.EMAIL_ADDRESS, ECS.FST_NAME, ECS.LAST_NAME, ECS.COMPANY_NAME, UPPER(P.URL),
ECS.COUNTRY, ECS.PER_REGION, ECS.EEA_EMAILABLE_FLG, ECS.PHONEABLE_FLG, ECS.MAILABLE_FLG
--ORDER BY P.EMAIL_ADDRESS
ORDER BY P.EMAIL_ADDRESS
If you want a quick and dirty solution, you could make both queries common table expressions and then join them as you would two normal tables:
with Tab1 AS ([YOUR FIRST BIG QUERY])
,Tab2 AS ([YOUR SECOUND BIG QUERY])
SELECT * FROM Tab1 INNER JOIN Tab2 ON Tab1.[FIELD] = Tab2.[FIELD]
If you want something that will not look so unpleasant, I would recommend taking the URL out and making it a variable, so you can call the script more succinctly and go from there, but that is very reliant on context.
I want to group the results in a separate column as Idle Total, Lock Total etc. I'm not a programmer and certainly not an db admin so i may not have all tools and access but I'll appreciate any help.
I've tried using other methods, here and other websites but it always comes down to the SUM or GROUP BY functions which I understand very little of.
SELECT TOP 10000
AGENT.AGENT_NAME,
ID.AGENT_ID,
TIME.DURATION_SECONDS,
SUM(CASE
WHEN ID.AGENT_ID IN ('IDLE','LOCK','LOGIN')
THEN 'WORKSTATE'
ELSE NULL
END)
FROM
WORKSTATE TIME
LEFT JOIN
DATABASE ID ON ID.INDEX = TIME.AGENT_INDEX
LEFT JOIN
DATABASE ORG ON TIME.AGENT_LOGIN_ID = ORG.AGENT_NAME
WHERE
(TIME.START_TIME = 2019-08-19) AND ORG.TEAM IN 'COMPANY'
GROUP BY
AGENT_AGENT_NAME,
ID.AGENT_ID,
TIME.DURATION_SECONDS
I would like the results to group TOTAL time an agent was Idle, Locked or Login on a given date.
Date Name Idle hrs Lock hrs Login
--- --- ------ ------ -----
08/19 John 2.3 1.7 4
You need to add CASE expression for each column, following query would help you to start with:
SELECT
TIME.START_TIME,
TIME.AGENT_LOGIN_ID,
SUM(CASE
WHEN ID.AGENT_ID = 'IDLE'
THEN TIME.DURATION_SECONDS END) as IdleTime, -- You may calculate You can formula to get HOURS from SECONDS or replace the column "DURATION_SECONDS" with the one that you looking to do aggregation
SUM(CASE
WHEN ID.AGENT_ID = 'LOCK'
THEN TIME.DURATION_SECONDS END) as LockTime,
SUM(CASE
WHEN ID.AGENT_ID = 'LOGIN'
THEN TIME.DURATION_SECONDS END) as LogTime
FROM WORKSTATE TIME
LEFT JOIN
DATABASE ID ON TIME.AGENT_INDEX = ID.INDEX
LEFT JOIN
DATABASE ORG ON TIME.AGENT_LOGIN_ID = ORG.AGENT_NAME
WHERE
(TIME.START_TIME = 2019-08-19) AND ORG.TEAM = 'COMPANY'
GROUP BY
TIME.START_TIME, TIME.AGENT_LOGIN_ID,
I have been doing a lot of research and everything I find is using PHP and MySQL. I need to be able to access my ODBC SQL Server (my MRP system) and display a custom query on a webpage for a visual for my plant. I have not been able to find a way to establish a connection to the database nor the correct way to format the query to display data. Any and all recommendations and formatting corrections are greatly appreciated. I am not even sure HTML is the route I need to go. I just need to build a table based on this data with further ability to format using specific string values from the table.
The query is below:
with cte as
(
SELECT DISTINCT
orout.PartNo,
sched.JobNo,
sched.StepNo,
orout.DeptNum AS MachCell,
orout.WorkCntr as Descrip,
agg.sumHours as HrsLeft,
agg.sumManHrs,
agg.minStartDate,
agg.maxEndDate,
sched.Priority,
od.QtyToMake,
isnull(tt2.QtyComplete,0) as QtyComplete,
CASE
WHEN isnull(tt2.QtyComplete,0) = 0 THEN od.QtyToMake
ELSE (od.QtyToMake - tt2.QtyComplete)
END as QtyOpen,
orout.TotActHrs,
orout.TotEstHrs,
emp.EmplCode,
Emp.NewEmplShortName,
emp.WorkCntr,
wc.Descrip as wcDescrip,
CASE
WHEN emp.EmplCode IS NOT null THEN 1
ELSE 0
END AS LoginPriority
FROM Scheduling as sched
LEFT JOIN OrderRouting as orout
ON orout.JobNo = sched.JobNo AND orout.StepNo = sched.StepNo
LEFT JOIN OrderDet as od
ON orout.JobNo = od.JobNo
LEFT JOIN
(
SELECT
JobNo,
online.EmplCode,
EmplCode.NewEmplShortName,
WorkCntr,
StepNo
FROM Online
INNER JOIN EmplCode
ON EmplCode.EmplCode = online.EmplCode
) emp
ON emp.JobNo = orout.JobNo and emp.StepNo = orout.StepNo
LEFT JOIN
(
SELECT
JobNo,
StepNo,
SUM(ManHrs) as sumManHrs,
SUM(Hours) as sumHours,
MIN(StartDate) as minStartDate,
MAX(EndDate) as maxEndDate
FROM
Scheduling
GROUP BY JobNo, StepNo
) agg
ON agg.JobNo = sched.JobNo AND agg.StepNo = sched.StepNo
LEFT JOIN TimeTicketDet as tt
ON tt.JobNo = orout.JobNo and tt.StepNo = orout.StepNo
LEFT JOIN
(
SELECT
JobNo,
StepNo,
SUM(PiecesFinished+PiecesScrapped) as QtyComplete
FROM
TimeTicketDet as tt2
GROUP BY JobNo, StepNo
) tt2
ON tt2.JobNo = orout.JobNo and tt2.StepNo = tt.StepNo
LEFT JOIN WorkCntr as wc
ON emp.WorkCntr = wc.WorkCntr
WHERE
sched.Priority > 0
)
SELECT *
FROM CTE
WHERE
Descrip = 'MACHINE2'
ORDER BY
LoginPriority DESC, Priority DESC, minStartDate
I would suggest looking for an off-the-shelf application that can use a SQL view as a data source. You can then write a view in SQL to return the data you need displaying, then leave the application to handle everything in the UI.
I've not used any in-depth, and I'm not affiliated with any, but the two I've played with in the past are https://www.tableau.com/ and https://powerbi.microsoft.com. Tableau more of a dashboard tool (and maybe a bit pricey), but I think PowerBI is free, but they're just examples of something you can use. Have a look around, but you should be able to find something that you can use.
I searched the internet far and wide looking for a way to query the WSUS database to view the number of updates that need to be installed on a server. I found a partial answer on the following blog, but this answer will assume if the update is pending approval it is still required by a server (WSUS assumes this in their UI as well).
http://theboywonder.co.uk/2010/11/04/sql-query-for-wsus-3-needed-updates/
I hope this solution is useful for others.
SELECT left(tbComputerTarget.FullDomainName,30) as [Machine Name]
,count(tbComputerTarget.FullDomainName) as [# of Missing patches]
,tbComputerTarget.LastSyncTime as [Last Sync Time]
FROM tbUpdateStatusPerComputer INNER JOIN tbComputerTarget ON tbUpdateStatusPerComputer.TargetID =
tbComputerTarget.TargetID
WHERE (NOT (tbUpdateStatusPerComputer.SummarizationState IN (’1′, ’4′))) AND
tbUpdateStatusPerComputer.LocalUpdateID IN (SELECT LocalUpdateID FROM dbo.tbUpdate WHERE UpdateID IN
(SELECT UpdateID FROM PUBLIC_VIEWS.vUpdateApproval WHERE Action=’Install’))
GROUP BY tbComputerTarget.FullDomainName, tbComputerTarget.LastSyncTime
ORDER BY COUNT(*) DESC
i can filter for the update name
SELECT C.FULLDOMAINNAME AS COMPUTADOR,VU.DEFAULTTITLE AS ATUALIZAÇÃO,
'STATUS' = CASE
WHEN UP.SummarizationState = 1 THEN 'NÃO INSTALADO'
WHEN UP.SummarizationState = 2 THEN 'NÃO INSTALADO'
WHEN UP.SummarizationState = 3 THEN 'INSTALANDO'
WHEN UP.SummarizationState = 4 THEN 'INSTALADO'
WHEN UP.SummarizationState = 5 THEN 'FALHOU'
END
FROM TBCOMPUTERTARGET C
INNER JOIN tbUpdateStatusPerComputer UP ON C.TargetID = UP.TargetID
INNER JOIN tbUpdate U ON UP.LocalUpdateID = U.LocalUpdateID
INNER JOIN [PUBLIC_VIEWS].[vUpdate] VU ON U.UpdateID = VU.UpdateId
WHERE VU.DefaultTitle LIKE 'WINDOWS Internet Explorer 9 for Windows 7'
OR VU.DefaultTitle LIKE'Update for Microsoft Office 2010%'
I suppose this is quite a common SP present in socialnetworks and community type websites.
I have this SP that returns all of a user's friends on their 'friends' page order by those currently online, then alphabetically. It's taking quite a while to load and I am looking to speed it up.
I remember reading somewhere on SO that breaking up multiple joins into smaller result sets might speed it up. I haven't tried this yet but I am curious to see what other recommendations SO might have on this procedure.
DECLARE #userID INT -- This variable is parsed in
DECLARE #lastActivityMinutes INT
SET #lastActivitytMinutes = '15'
SELECT
Active = CASE WHEN DATEDIFF("n", b.LastActivityDate ,GETDATE()) < #lastActivityMinutes THEN 1 ELSE 0 END,
a.DisplayName, a.ImageFile, a.UserId, b.LastActivityDate
FROM
Profile AS a
INNER JOIN aspnet_Users as b on b.userId = a.UserId
LEFT JOIN Friend AS x ON x.UserID = a.UserID
LEFT JOIN Friend AS z ON z.FriendID = a.UserID
WHERE ((x.FriendId = #userID AND x.status = 1) -- Status = 1 means friendship accepted
OR (z.UserID = #userID AND z.Status = 1))
GROUP BY a.userID, a.DisplayName, a.ImageFile, a.UserId, b.LastActivityDate
ORDER BY Active DESC, DisplayName ASC
I am not sure how to clip in my execution plan but the main bottle neck seems to be occurring on a MERGE JOIN (Right Outer Join) that's costing me 29%. At various stages, Parallelism is also costing 9%, 6%, 5% and 9% for a total of 29% as well.
My initial thoughts are to first return the JOINED results from the Profile and aspnet tables with a CTE and then do LEFT JOINS to the Friends table.
You are joining Friend twice, using a LEFT JOIN, then you are removing the NULL's returned by the LEFT JOIN by WHERE condition, then using GROUP BY to get rid on distincts.
This is not the best query possible.
Why don't you just use this:
SELECT Active = CASE WHEN DATEDIFF("n", b.LastActivityDate ,GETDATE()) < #lastActivityMinutes THEN 1 ELSE 0 END,
a.DisplayName, a.ImageFile, a.UserId, b.LastActivityDate
FROM (
SELECT FriendID
FROM Friends
WHERE UserID = #UserId
AND status = 1
UNION
SELECT UserID
FROM Friends
WHERE FriendID = #UserId
AND status = 1
) x
INNER JOIN
Profile AS a
ON a.UserID = x.FriendID
INNER JOIN
aspnet_Users as b
ON b.userId = a.UserId
ORDER BY
Active DESC, DisplayName ASC