I need to create a trigger on one of the sysjobxxx tables in MS SQL Server which when a record of a job is inserted, needs to be triggered.
For eg. when jobA is executed, some of the sysjobxxx tables (sysjobhistory, sysjobs, sysjobsteps ) get records inserted. My trigger needs to be
built on the table that inserts the success/failure info of jobA. I'd thought that sysjobhistory would be the one but when I tested with a dummy job that failed, it inserted 2 records, instead of 1 - this would run my trigger (upon insert) twice, instead of just once.
What I'm trying to accomplish is to get the full description of job failure, everytime a job fails and is inserted into the sysjobxxx tables.
I know there's an email notification that can be sent out but the description of failure is truncated (to 1024 chars) and unless a full 'View history' is done on the job, it's not possible to see the complete job failure description.
Is there a way I can create a trigger on one of the sysjobxxx tables, that upon a job record insertion, checks a column (I don't know which one
indicates failure of the job), then sends out an email (directly from within the trigger via sp_sendmail or calls another stored proc that then
executes sp_sendmail) to a list of recipients, with the full job failure description?
Related
We have some SQL Server stored procedure.
It selects some rows from a table and puts them into the temp table to apply and calculate some data validations.
The next part of the procedure either updates the actual table based on the temp table data or sends back the error status.
Initially selected rows can only be updated once and no further updates are allowed to the same rows.
The problem, we are facing is like some time, 2 simultaneous threads execute the procedure at the same time and both pass the initial validation block as in-memory temp data is not processed yet. 2nd thread is able to overwrite the first transaction.
We applied the transaction mechanism to prevent duplicate inserts and updates by checking the affected rows count by update query and aborting the transaction.
I am not sure if it's correct and optimized or not.
Also, can we lock rows with a select statements as well ?
This has been solved using the UPDLOCK on select query inside the transaction.
It locks the specific rows and allow the transaction to proceed in isolation.
Thanks Everyone for your help.
I am working on a spring batch service that pulls data from a db on a schedule. (e.g. every day at 12pm)
I am using JdbcPagingItemReader to read the data and a scheduler (#Scheduled provided by spring batch) to launch the job. The problem that I have now is: every time the job runs, it will just pull all the data from the beginning and not from the "last read" row.
The data from the db is changing everyday(deleting old ones and adding new ones) and all I have is a timestamp column to track them.
Is there a way to "remember" the last row read from the last execution of the job and read data only later than that row?
Since you need to pull data on a daily basis, and your records have a timestamp, then you can design your job instances to be based on a given date (ie using the date as an identifying job parameter). With this approach, you do not need to "remember" the last processed record. All you need to do is process records for a given date by using the correct SQL query. For example:
Job instance ID
Date
Job parameter
SQL
1
2021-03-22
date=2021-03-22
Select c1, c2 from table where date = 2021-03-22
2
2021-03-23
date=2021-03-23
Select c1, c2 from table where date = 2021-03-23
...
...
...
...
With that in place, you can use any cursor-based or paging-based reader to process records of a given date. If a job instance fails, you can restart it without a risk to interfere with other job instances. The restart could be done even several days after the failure since the job instance will always process the same data set. Moreover, in case of failure and job restart, Spring Batch will reprocess records from the last check point in the previous (failed) run.
Just want to post an update to this question.
So in the end I created two more steps to achieve what I wanted to do initially.
Since I don't have the privilege to modify the table where I read the data from, I couldn't use the "process indicator pattern" which involves having a column to mark if a record is processed or not. I created another table to store the last-read record's timestamp, and use it to update the sql query.
step 0: a tasklet that reads the bookmark from a table, pass it in the job context
step 1: a chunk step, get the bookmark from the context, use jdbcPagingItemReader to read the data
step 2: a tasklet to update the bookmark
But doing this I have to be very cautious with the bookmark table. If I lose that I lose everything
I am running a daily job in MSSQL agent, that runs a package to export a certain table to an excel file. After completion it sends me an email.
It happened couple of times that the table was empty and it exported 0 rows.
I did not receive a job failed notification, because the job didn't fail.
I would like to receive a failed notification when it's exporting 0 rows.
How would you achieve this?
Thanks
There are a number of ways to force a package to return a failure (which will then cause the calling Agent job to report a failure).
One simple way would be to put a Row Count task between the source and destination in the data flow task that populates the spreadsheet. Assign the value to a variable, say #RowsExported.
Back on the Control Flow tab, if there's more to the package, put a condition on the precedent constraint leading to the rest of the work where #RowsExported > 0 so the rest of the package will only continue if there were rows sent. Whether or not there's more to the package, add a new precedent constraint coming off the data flow with a condition #RowsExported == 0. Send that constraint to an Execute SQL task that just contains the script SELECT 1/0;.
Now, if zero rows are exported, the package will throw a division by zero error, and the calling job will fail.
There are other ways, of course, but this one can be implemented very quickly.
I'm trying to create SSIS package which will periodically send data to other database. I want to send only new records(I need to keep sent records) so I created status column in my source table.
I want my package to update this column after successfuly sending data, but I can't update all rows wih "unsent" status because during package execution some rows may have been added, and I also can't use transactions(I mean on isolation levels that would solve my problem: I can't use Serializable beacause i musn't prevent users from adding new rows, and Sequence Container doesn't support Snapshot).
My next idea was to use recordset and after sending data to other db use it to get ids of sent rows, but I couldn't find a way to use it as datasource.
I don't think I should set status "to send" and then update it to "sent", I believe it would be to costly.
Now I'm thinking about using temporary table, but I'm not convinced that this is the right way to do it, am I missing something?
Record Set is a destination. You cannot use it in Data Flow task.
But since the data is saved to a variable, it is available in the Control flow.
After completing the DataFlow, come to the control flow and create a foreach component that can run on the ResultSet varialbe.
Read each Record Set value into a variable and use it to run an update query.
Also, see if "Lookup Transform" can be useful to you. You can generate rows that match or doesn't match.
I will improve the answer based on discussions
What you have here is a very typical data mirroring problem. To start with, I would not simply have a boolean that signifies that a record was "sent" to the destination (mirror) database. At the very least, I would put a LastUpdated datetime column in the source table, and have triggers on that table, on insert and update, that put the system date into that column. Then, every day I would execute an SSIS package that reads the records updated in the last week, checks to see if those records exist in the destination, splitting the datastream into records already existing and records that do not exist in the destination. For those that do exist, if the LastUpdated in the destination is less than the LastUpdated in the source, then update them with the values from the source. For those that do not exist in the destination, insert the record from the source.
It gets a little more interesting if you also have to deal with record deletions.
I know it may seem wasteful to read and check a week's worth, every day, but your database should hardly feel it, it provides a lot of good double checking, and saves you a lot of headaches by providing a simple, error tolerant algorithm. Some record does not get transferred because of some hiccup on the network, no worries, it gets picked up the next day.
I would still set up the SSIS package as a server task that sends me an email with any errors, so that I can keep track. Most days, you get no errors, and when there are errors, you can wait a day or resolve the cause and let the next days run pick up the problems.
I am doing a similar thing, in my case, I have a status on the source record.
I read in all records with a status of new.
Then use a OLE DB Command to execute SQL on each row, changing
the status to "In progress"(in you where, enter a ? as the value in
the Component Property tab, and you can configure it as a parameter
from the table row like an ID or some pk in the Column Mappings
tab).
Once the records are processed, you can change all "In Progress"
records to "Success" or something similar using another OLE DB
Command.
Depending on what you are doing, you can use the status to mark records that errored at some point, and require further attention.
I got a task to do, but I don't know exactly how to do it.
I need a Stored Procedure to run every 5 min and validate if a new record has been inserted in one table. If new record is found then execute an insert into another table to make a copy of that record, but if not, then nothing happens and both tables remain the same.
In other words, I need something similar to "after insert" trigger, but I don't want to use a trigger.
Create the stored procedure that you want to run, and then install it as a scheduled job within sql server that runs every 5 minutes.
Do what jhilden suggests with the SQL job running every 5 minutes.
The SP needs to look at the latest record in the copy of the table (timestamp or MAX(ID) if you are conserving the IDs accross the two tables) then check if there is/are a record(s) in the original table with a higher timestamp (or ID), if so copy it/them accross.