Is there a way to get the amount of 'selects' from a mssql table?
I mean without rewriting application to log every request or without parsing sql profile logs...
Is there any build-in tool (sql request)?
SQL Server maintains index usage statistics since last restart. You can use user_reads column from query bellow to get wanted statistics:
SELECT OBJECT_NAME(ddius.[object_id], ddius.database_id) AS [object_name] ,
ddius.index_id ,
ddius.user_seeks ,
ddius.user_scans ,
ddius.user_lookups ,
ddius.user_seeks + ddius.user_scans + ddius.user_lookups
AS user_reads ,
ddius.user_updates AS user_writes ,
ddius.last_user_scan ,
ddius.last_user_update
FROM sys.dm_db_index_usage_stats ddius
WHERE ddius.database_id > 4 -- filter out system tables
AND OBJECTPROPERTY(ddius.OBJECT_ID, 'IsUserTable') = 1
AND ddius.index_id > 0 -- filter out heaps
AND database_id = DB_ID()
ORDER BY user_reads DESC
Related
I tested this query using clickhouse-benchmark
select pageUrl ,
pageTitle ,
sum(visits) as visits ,
count(DISTINCT visitorId) as visitors ,
any(featuredImage) as featuredImage ,
any(pageType) as pageType ,ssid
FROM PageViews FINAL where (createdAtDay >= toDate('2022-06-01') and createdAtDay <= toDate('2022-10-01')) GROUP BY pageUrl,pageTitle,ssid ORDER BY visits DESC
If executing the command from terminal on the server that's hosting clickhouse instance I get pretty fast response :
but when I try to execute same query remotely (which is the usual scenario, I'm not hosting my api in same server as clickhouse)
I get slow response :
I tried to get logs using this query :
SELECT
event_time, query_duration_ms / 1000 AS secs,
formatReadableSize(memory_usage) AS memory,
Settings['max_threads'] AS threads,
query
FROM system.query_log
WHERE (user = 'default') AND (type = 'QueryFinish') and (query like '%select pageUrl ...%')
ORDER BY event_time DESC
and it seems even in database logs the execution time is very big :
last 5 are the remote ones, first 5 are from the server, the time diff is very big.
is this normal? is there any type of config that I must setup ?
I have two BI reporting servers (one is Development, the other Production) each with their own own Azure SQL instance, I am using Pentaho to automate the ETL process and part of my process is to execute a query joining two tables of similar data in order to join a user's login row with their logout row.
with login_cte as
(
select
os.[k_actions]
,[k_hosts]
,[k_logger_names]
,[k_principals]
,[k_sites]
,[k_roles]
,[k_date] as k_date_logged_in
,[k_time] as k_time_logged_in
,[k_date_ct] as k_date_ct_logged_in
,[k_time_ct] as k_time_ct_logged_in
,[key] as key_logged_in
,[type]
,[id]
,[sort]
,[level]
,[timestamp_utc] as timestamp_utc_logged_in
,[timestamp_ct] as timestamp_ct_logged_in
,[service]
,[version]
,[message] as message_logged_in
,[thread_name]
,[db_name]
,[customer_id]
from other_stage os
left join dim_actions da on os.k_actions = da.k_actions
where [action_name] = 'SET_ROLE_DEVICE_USER'
order by [key] asc
offset 0 rows
),
logout_cte as
(
select
[k_actions]
,[k_hosts] as k_hosts_out
,[k_logger_names] as k_logger_names_out
,[k_principals] as k_principals_out
,[k_sites] as k_sites_out
,null as k_roles_out
,[k_date] as k_date_logged_out
,[k_time] as k_time_logged_out
,[k_date_ct] as k_date_ct_logged_out
,[k_time_ct] as k_time_ct_logged_out
,[key] as key_logged_out
,[type] as type_out
,[id] as id_out
,[sort] as sort_out
,[level] as level_out
,[timestamp_utc] as timestamp_utc_logged_out
,[timestamp_ct] as timestamp_ct_logged_out
,[service] as service_out
,[version] as version_out
,[message] as message_logged_out
,[thread_name] as thread_name_out
,[db_name]
,[customer_id]
from logout_stage
order by [key] asc
offset 0 rows
)
select
li.k_actions
,li.k_hosts
,li.k_logger_names
,li.k_principals
,li.k_roles
,li.k_sites
,li.k_date_logged_in
,li.k_time_logged_in
,lo.k_date_logged_out
,lo.k_time_logged_out
,li.k_date_ct_logged_in
,li.k_time_ct_logged_in
,lo.k_date_ct_logged_out
,lo.k_time_ct_logged_out
,li.key_logged_in
,lo.key_logged_out
,li.type
,li.id
,li.sort
,li.level
,li.timestamp_utc_logged_in
,li.timestamp_ct_logged_in
,lo.timestamp_utc_logged_out
,lo.timestamp_ct_logged_out
,li.service
,li.version
,li.message_logged_in
,lo.message_logged_out
,li.thread_name
,li.db_name
,li.customer_id
,datediff(second, li.timestamp_utc_logged_in, lo.timestamp_utc_logged_out) as login_duration
,row_number() over (partition by li.key_logged_in order by lo.key_logged_out asc) as rn
from login_cte li left join logout_cte lo
on li.k_principals = lo.k_principals_out
and li.db_name = lo.db_name
and lo.key_logged_out > li.key_logged_in
and lo.timestamp_utc_logged_out > li.timestamp_utc_logged_in
order by lo.key_logged_out asc
The issue I'm having is that this query runs just fine by itself in my production instance but when run on the development instance, which has fewer rows in each table (~70k vs ~124k), the query fails with this error.
SQLServerException: The database 'tempdb' has reached its size quota. Partition or delete data, drop indexes, or consult the documentation for possible resolutions.
Doing some digging into the tempdb size on each I ran this query:
SELECT SUM(unallocated_extent_page_count) AS [free pages],
(SUM(unallocated_extent_page_count)*1.0/128) AS [free space in MB]
FROM tempdb.sys.dm_db_file_space_usage;
With these results:
Production:
free pages
free space in MB
923,792
7,217.125
Development:
free pages
free space in MB
8,386,864
65,522.375
while the query is running I can run file space usage query again and watch the free space slow go done to zero then the query fails. Meanwhile the same query on Production shows the free space go down but never hits zero and will succeed.
I am fairly new to Azure SQL and have tried searching for an answer but can only find answers on how to find a query that is using up the tempdb (of which I already know...) but not any on what would cause the same query to fail on one and not the other. I do not have access to the Azure console but can connect to the DB with a power user through either DBeaver or Microsoft SQL Server Management Studio.
Edit 12-14-21:
Paste of the query execution plan
There is always a need to find out details, either intentionally Or mistakenly someone executed DROP/DELETE command on any of following SQL Server database objects.
DROPPED - Table from your database
DROPPED - Stored Procedure from your database
DELETED - Rows from your database table
Q. Is there TSQL available to find db user who performed DELETE/DROP?
Q. What kind of permissions are needed for user to find out these details?
Did you check this ?
Right click on database.
Go to as shown in image :
Solution 2 :
This query gives alot of useful information for a database(apply filter as required) :
DECLARE #filename VARCHAR(255)
SELECT #FileName = SUBSTRING(path, 0, LEN(path)-CHARINDEX('\', REVERSE(path))+1) + '\Log.trc'
FROM sys.traces
WHERE is_default = 1;
SELECT gt.HostName,
gt.ApplicationName,
gt.NTUserName,
gt.NTDomainName,
gt.LoginName,
--gt.SPID,
-- gt.EventClass,
te.Name AS EventName,
--gt.EventSubClass,
-- gt.TEXTData,
gt.StartTime,
gt.EndTime,
gt.ObjectName,
gt.DatabaseName,
gt.FileName,
gt.IsSystem
FROM [fn_trace_gettable](#filename, DEFAULT) gt
JOIN sys.trace_events te ON gt.EventClass = te.trace_event_id
WHERE EventClass in (164) --AND gt.EventSubClass = 2
ORDER BY StartTime DESC;
I have around 40 different sql server jobs in one instance. They all have different schedules. Some run once a day some every two mins some every five mins. If I have a need to stop sql server agent, how can I find the best time when no jobs are running so I won't interrupt any of my jobs?
how can I find the best time when no jobs are running so I won't interrupt any of my jobs?
You basically want to find a good window to perform some maintenance. #MaxVernon has blogged about it here with a handy script
/*
Shows gaps between agent jobs
-- http://www.sqlserver.science/tools/gaps-between-sql-server-agent-jobs/
-- requires SQL Server 2012+ since it uses the LAG aggregate.
Note: On SQL Server 2005, SQL Server 2008, and SQL Server 2008 R2, you could replace the LastEndDateTime column definition with:
LastEndDateTime = (SELECT TOP(1) s1a.EndDateTime FROM s1 s1a WHERE s1a.rn = s1.rn - 1)
*/
DECLARE #EarliestStartDate DATETIME;
DECLARE #LatestStopDate DATETIME;
SET #EarliestStartDate = DATEADD(DAY, -1, GETDATE());
SET #LatestStopDate = GETDATE();
;WITH s AS
(
SELECT StartDateTime = msdb.dbo.agent_datetime(sjh.run_date, sjh.run_time)
, MaxDuration = MAX(sjh.run_duration)
FROM msdb.dbo.sysjobs sj
INNER JOIN msdb.dbo.sysjobhistory sjh ON sj.job_id = sjh.job_id
WHERE sjh.step_id = 0
AND msdb.dbo.agent_datetime(sjh.run_date, sjh.run_time) >= #EarliestStartDate
AND msdb.dbo.agent_datetime(sjh.run_date, sjh.run_time) < = #LatestStopDate
GROUP BY msdb.dbo.agent_datetime(sjh.run_date, sjh.run_time)
UNION ALL
SELECT StartDate = DATEADD(SECOND, -1, #EarliestStartDate)
, MaxDuration = 1
UNION ALL
SELECT StartDate = #LatestStopDate
, MaxDuration = 1
)
, s1 AS
(
SELECT s.StartDateTime
, EndDateTime = DATEADD(SECOND, s.MaxDuration - ((s.MaxDuration / 100) * 100)
+ (((s.MaxDuration - ((s.MaxDuration / 10000) * 10000))
- (s.MaxDuration - ((s.MaxDuration / 100) * 100))) / 100) * 60
+ (((s.MaxDuration - ((s.MaxDuration / 1000000) * 1000000))
- (s.MaxDuration - ((s.MaxDuration / 10000) * 10000))) / 10000) * 3600, s.StartDateTime)
FROM s
)
, s2 AS
(
SELECT s1.StartDateTime
, s1.EndDateTime
, LastEndDateTime = LAG(s1.EndDateTime) OVER (ORDER BY s1.StartDateTime)
FROM s1
)
SELECT GapStart = CONVERT(DATETIME2(0), s2.LastEndDateTime)
, GapEnd = CONVERT(DATETIME2(0), s2.StartDateTime)
, GapLength = CONVERT(TIME(0), DATEADD(SECOND, DATEDIFF(SECOND, s2.LastEndDateTime, s2.StartDateTime), 0))
FROM s2
WHERE s2.StartDateTime > s2.LastEndDateTime
ORDER BY s2.StartDateTime;
The question title scared me a bit - I thought you wanted to programmatically shut the SQL Server agent down anytime there were no jobs running. My answer to that question would be "Why?" There is no need to.
But if you are just looking to do a planned restart or shut down and you don't have a third party tool like Sentry One's SQL Sentry Event Manager to have a visualization, I would just let the SQL Server Agent Job History and Job Activity Monitor help here. The Job Activity monitor can show you which jobs are running right now in the status column. You can also see the last execute and next execute dates and times.
In the object browser in SSMS, connect to your instance, then expand SQL Server Agent, then you'll see Jobs and under that you'll see "Job Activity Monitor" - this view should show you what you need.
Also - don't worry about shutting down before a job executes. If you do that, you will either have that job just missing its schedule and you can let it run when it is next due to (depending on the job and its purpose) or you can manually right click and execute the job.
For more on the activity monitor for jobs, see Monitor Job Activity in the product documentation.
I recommend creating a script that will disable your jobs. Disabled jobs still exist but will not be automatically launched by their schedules. Run this script (based on procedure sp_update_job in the msdb database) to disable jobs, wait for any currently running jobs to finish execution, then stop SQL agent. A similar script to re-enable disabled jobs would be useful. You might need to plan around jobs that are and should remain disabled.
A complete “SQL Agent shutdown” process could be fully scripted, but I question the wisdom of doing so. A bit of research implies that there is no 100% reliable way of programmatically telling if a given job is or is not running, and while there is an undocumented (where "undocumented" means "you really shouldn't be using this") system procedure for stopping and starting services, doing so from with SQL Server itself seems like a pretty bad idea.
You can query the system tables as shown by Dattatrey Sindol in the MSSQLTips.com article Querying SQL Server Agent Job Information:
SELECT
[sJOB].[job_id] AS [JobID]
, [sJOB].[name] AS [JobName]
, [sDBP].[name] AS [JobOwner]
, [sCAT].[name] AS [JobCategory]
, [sJOB].[description] AS [JobDescription]
, CASE [sJOB].[enabled]
WHEN 1 THEN 'Yes'
WHEN 0 THEN 'No'
END AS [IsEnabled]
, [sJOB].[date_created] AS [JobCreatedOn]
, [sJOB].[date_modified] AS [JobLastModifiedOn]
, [sSVR].[name] AS [OriginatingServerName]
, [sJSTP].[step_id] AS [JobStartStepNo]
, [sJSTP].[step_name] AS [JobStartStepName]
, CASE
WHEN [sSCH].[schedule_uid] IS NULL THEN 'No'
ELSE 'Yes'
END AS [IsScheduled]
, [sSCH].[schedule_uid] AS [JobScheduleID]
, [sSCH].[name] AS [JobScheduleName]
, CASE [sJOB].[delete_level]
WHEN 0 THEN 'Never'
WHEN 1 THEN 'On Success'
WHEN 2 THEN 'On Failure'
WHEN 3 THEN 'On Completion'
END AS [JobDeletionCriterion]
FROM
[msdb].[dbo].[sysjobs] AS [sJOB]
LEFT JOIN [msdb].[sys].[servers] AS [sSVR]
ON [sJOB].[originating_server_id] = [sSVR].[server_id]
LEFT JOIN [msdb].[dbo].[syscategories] AS [sCAT]
ON [sJOB].[category_id] = [sCAT].[category_id]
LEFT JOIN [msdb].[dbo].[sysjobsteps] AS [sJSTP]
ON [sJOB].[job_id] = [sJSTP].[job_id]
AND [sJOB].[start_step_id] = [sJSTP].[step_id]
LEFT JOIN [msdb].[sys].[database_principals] AS [sDBP]
ON [sJOB].[owner_sid] = [sDBP].[sid]
LEFT JOIN [msdb].[dbo].[sysjobschedules] AS [sJOBSCH]
ON [sJOB].[job_id] = [sJOBSCH].[job_id]
LEFT JOIN [msdb].[dbo].[sysschedules] AS [sSCH]
ON [sJOBSCH].[schedule_id] = [sSCH].[schedule_id]
ORDER BY [JobName]
I'm stuck on an SQL query so I thought maybe an SQL MVP/GOD could find this here with small luck.
I'm using SQL Server 2008 and here's a description of my tables:
Tables - columns
NodesCustomProperties - NodeID / NodeZone
Application - NodeID / ID
Component - ApplicationID / Name
CurrentComponentStatus: ApplicationID / Data
I'd like to fetch the SUM of CurrentComponentStatus.Data when Component.Name is like 'HTTP%: Bytes Transferred Between Proxy and Servers' for the same ApplicatioID and filter these results when NodesCustomProperties.NodeZone = 'one particular zone'
Research and testing have lead me here so far:
SELECT
SUM(
CASE
WHEN [dbo].[APM_CurrentComponentStatus].StatisticData IS NOT NULL
THEN [dbo].[APM_CurrentComponentStatus].StatisticData
ELSE 0
END) AS 'Data'
FROM
[dbo].[APM_CurrentComponentStatus]
LEFT JOIN [dbo].[APM_Application]
ON [dbo].[APM_Application].ID = [dbo].[APM_CurrentComponentStatus].ApplicationID
WHERE
[dbo].[APM_Application].NodeID IN (
SELECT [dbo].[NodesCustomProperties].NodeID
FROM [dbo].[NodesCustomProperties]
WHERE [dbo].[NodesCustomProperties].NodeZone = 'one particular zone')
GROUP BY [dbo].[APM_CurrentComponentStatus].ApplicationID
HAVING [dbo].[APM_CurrentComponentStatus].ApplicationID
IN (
SELECT [dbo].[APM_Component].ApplicationID
FROM [dbo].[APM_Component]
WHERE [dbo].[APM_Component].Name LIKE 'HTTP%: Bytes Transferred Between Proxy and Servers')
This query actually works (Hooray !) but there's too few results so that's still not it. (Awwww !)