Just started rewriting of oracle jobs monitoring.
Currently which I'm using is that Nagios is calling two different functions to check DBMS and Scheduler job statuses.
What i'm checking now:
DBMS:
if job is broker.
if job worked longer then expected(actually this is not working correctly
because i cant determine any middle or approximately time which it takes)
if it executed late, not on time.
ok, in case non of above was true.
All this data is collected from sys.dba_jobs and custom conf tables
Scheduler:
count of failures in given interval
too few runs in given interval
worked for too long then expected i'm pretty sure that all result expect count of failures are not accurate. this data is collected from SYS.DBA_SCHEDULER_JOB_RUN_DETAILS and custom conf tables.
What is my gain:
avoid useless conf tables
need to monitor jobs without custom confs for each jobs, because there always is risk to don't add data about job in table or add incorrect data.
i need to somehow get accurate data for each job how long it could be take for execution and how many times had to be executed for given time.
If anyone has produced task like this please help or give some advice or source code where I can take a look and modify for my DB.
Related
I have an ADF solution which is metadata driven. It passes a connection string, and source and sink as parameters. My concern is that I also have SQL logging steps within pipelines and child pipelines and now for a simple Azure DB table copy into ADSL Parquet it is bottlenecked by the logging steps and child pipelines. I noticed that each step (mainly logging steps) take around 3-6 seconds.
I have tried the following:
upgrading the config database from basic to S1
changing the ADF's integration runtime to 32 core count
changing the TTL to 20 mins
checked the quick cache
Nothing seems to reduce the time to run these audit steps.
The audit step is a stored procedure which you pass in a load of parameters. This proc run in split seconds in SSMS so the proc isn't the issue.
Is there any way of reducing the time to execute logging steps?
As per the Microsoft SLA for ADF, they guarantee that at least 99.9% of the time, all activity runs will initiate within 4 minutes of their scheduled execution times.
As stated from the Product Team, any stored procedure activity that performs within 4 minutes has met SLA within ADF--this SLA covers the overhead of ADF communicating with SQL server. With that said, your current performance within ADF is normal.
Check the supported documents:
https://azure.microsoft.com/en-gb/support/legal/sla/data-factory/v1_2/
https://learn.microsoft.com/en-us/answers/questions/36323/adf-performance-troubleshooting.html
I tend to think adding excessive logging into pipelines is a bit of an anti-pattern as it leads to more complex pipelines with more components to maintain. You can also encounter this type of issue, particularly if you're running the logging inside a For Each activity for example. I tend to use this sparingly and concentrate more on the built-logging and making use of the custom user properties and annotations. Most of the information you need should be in the built-in logging which you can harvest via API calls either at the end of your pipelines or elsewhere, as ably described here and here.
The other thing you could do is move any non-critical tasks to run in parallel. For example a 'log start of activity' task does not necessarily need to run first sequentially. Consider making some of these tasks non-parallel, as explained in the diagram below. Obviously this does not apply when you need to capture information from the activity to log:
Disclaimer: I wouldn't be at all surprised to find out this is duplicate somewhere, but I have literally been searching for hours. All I can find it DBA information that I don't believe pertains to what I am trying to accomplish.
I am currently developing a small database for a friends startup. He has jobs that he needs to track that are performed on regular intervals (e.g. semi-annually, annually, quarterly, weekly, etc.).
I am trying to wrap my mind around how I can implement a solution in sql server where a customer has 0 to many tasks, and a specific task can have 0 many customers. Seems simple enough, but I also want to keep a history for each time the task was completed for each customer where it is assigned. This would be a record with a scheduled task and a completed date. Lastly, I need to be able to execute a query to retrieve upcoming scheduled tasks within a given period of time (e.g. all scheduled jobs coming up next month).
I know that there are is a ton of software that does stuff like this, so it can't be that uncommon, but I cannot find any information that is getting me anywhere. If there are any resources anyone could recommend it would greatly appreciated.
You should just create a table to hold all the list of tasks that need to be done, with columns where you can later record when they have been done.
You can easily then just query it to find what needs to be done.
Then create a stored procedure that does the work. Have the procedure do the right thing for different periods i.e. monthly, etc. Have it just work out what's needed, and do it.
Create a single SQL Server Agent job that runs that procedure each month. (Or however often it's needed).
As you do each step, no-one's going to mind if you ask for help with each part i.e. get comments on your table when you first design it, etc. (Or at least I wouldn't mind - some might but ignore them)
I am working on a flink project which write stream to a relational database.
In the current solution, we wrote a custom sink function which open transaction, execute SQL insert statement and close transaction. It works well until the the data volume increases and we started getting connection timeout issues. We tried a few connection pool configuration adjustment, it does not help much.
We are thinking of trying "batch-insert", so to decrease the number of "writes" to the database. We come across a few classes which do almost what we want: JDBCOutputFormat, JDBCSinkFunction. With JDBCOutputFormat, we can configure the batch size.
We would also like to force a "batch-insert" every 1 minutes if the number of records does not exceed the "batch-size". How would you normally deal with these kinds of problems? My first thoughts is to extend JDBCOutputFormat to use scheduled tasks to force flush every 1 minute, but it was not obvious how it could be done.
Do we have to write our own sink all together?
Updated:
JDBCSinkFunction does a flush and batch execute each time Flink checkpoints. So long as you are doing checkpointing, the batches won't be any longer than the checkpointing interval.
However, having read this mailing list thread, I see that JDBCSinkFunction does not support exactly-once output.
My job is running since more than 24 hours.When I checked, what is actually going on using this (how to know status of currently running jobs); its showing its still running. So is there any way to check, by what approximate time, this would complete? Is there any logic that has already implemented by Microsoft for this, like we have file transfer approximate time show?
There is no way to see an estimate of when a job will finish. The closest you can get is to put logging in that tells you when each task is started/completed (using the built in SSIS logging provider).
Logging to table or file, you can then check progress there and that will give you an indication how far through the job you are (and based off of your own knowledge of the package, you may be able to estimate how long until it will be finished)
Sorry for the long introduction but before I can ask my question, I think giving the background would help understanding our problem much better.
We are using sql server 2008 for our web services as the backend and from time to time it takes too much time for responding back for the requests that supposed to run really fast, like taking more than 20 seconds for a select request that queries a table that has only 22 rows. We went through many potential areas that could cause the issue from indexes to stored procedures, triggers etc, and tried to optimize whatever we can like removing indexes that are not read but write frequently or adding NOLOCK for our select queries to reduce the locking of the tables (we are OK with dirty reads).
We also had our DBA's reviewed the server and benchmarked the components to see any bottlenecks in CPU, memory or disk subsystem, and found out that hardware-wise we are OK as well. And since the pikes are occurring occasionally, it is really hard to reproduce the error on production or development because most of the time when we rerun the same query it yields response times that we are expecting, which are short, not the one that has been experienced earlier.
Having said that, I almost have been suspicious about I/O although it is not seem to be a bottleneck. But I think I was just be able to reproduce the error after running an index fragmentation report for a specific table on the server, which immediately caused pikes in requests not only run against that table but also in other requests that query other tables. And since the DB, and the server, is shared with other applications we use and also from time to time queries can be run on the server and database that take long time is a common scenario for us, my suspicion regarding occasional I/O bottleneck is, I believe, becoming a fact.
Therefore I want to find out a way that would prioritize requests that are coming from web services which will be processed even if there are other resource sensitive queries being run. I have been looking for some kind of prioritization I described above since very beginning of the resolution process and found out that SQL Server 2008 has a feature called 'Resource Governor' that allows prioritization of the requests.
However, since I am not an expert on Resource Governor nor a DBA, I would like to ask other people's experience who may have used or is using Resource Governor, as well as whether I can prioritize I/O for a specific login or a specific stored procedure (For example, if one I/O intensive process is being run at the time we receive a web service request, can SQL server stops, or slows down, I/O activity for that process and give a priority to the request we just received?).
Thank you for anyone that spends time on reading or helping out in advance.
Some Hardware Details:
CPU: 2x Quad Core AMD Opteron 8354
Memory: 64GB
Disk Subsystem: Compaq EVA8100 series (I am not sure but it should be RAID 0+1 accross 8 HP HSV210 SCSI drives)
PS:And I can almost 100 percent sure that application servers are not causing the error and there is no bottleneck we can identify there.
Update 1:
I'll try to answer as much as I can for the following questions that gbn asked below. Please let me know if you are looking something else.
1) What kind of index and statistics maintenance do you have please?
We have a weekly running job that defrags indexes every Friday. In addition to that, Auto Create Statistics and Auto Update Statistics are enabled. And the spikes are occurring in other times than the fragmentation job as well.
2) What kind of write data volumes do you have?
Hard to answer.In addition to our web services, there is a front end application that accesses the same database and periodically resource intensive queries needs to be run to my knowledge, however, I don't know how to get, let's say weekly or daily, write amount to DB.
3) Have you profiled Recompilation and statistics update events?
Sorry for not be able to figure out this one. I didn't understand what you are asking about by this question. Can you provide more information for this question, if possible?
first thought is that statistics are being updated because of the data change threshold is reached causing execution plans to be rebuilt.
What kind of index and statistics maintenance do you have please? Note: index maintenance updates index stats, not column stats: you may need separate stats updates.
What kind of write data volumes do you have?
Have you profiled Recompilation and statistics update events?
In response to question 3) of your Update to the original question, take a look at the following reference on SQL Server Pedia. It provides an explanation of what query recompiles are and also goes on to explain how you can monitor for these events. What I believe gbn is asking (feel free to correct me sir :-) ) is are you seeing recompile events prior to the slow execution of the troublesome query. You can look for this occurring by using the SQL Server Profiler.
Reasons for Recompiling a Query Execution Plan