Create SQL job that verifies daily entry of data into a table? - sql-server

Writing my first SQL query to run specifically as a SQL Job and I'm a little out of my depth. I have a table within a SQL Server 2005 Database which is populated each day with data from various buildings. To monitor the system better, I am attempting to write a SQL Job that will run a query (or stored procedure) to verify the following:
- At least one row of data appears each day per building
My question has two main parts;
How can I verify that data exists for each building? While there is a "building" column, I'm not sure how to verify each one. I need the query/sp to fail unless all locations have reported it. Do I need to create a control table for the query/sp to compare against? (as the number of building reporting in can change)
How do I make this query fail so that the SQL Job fails? Do I need to wrap it in some sort of error handling code?
Table:
Employee RawDate Building
Bob 2010-07-22 06:04:00.000 2
Sally 2010-07-22 01:00:00.000 9
Jane 2010-07-22 06:04:00.000 12
Alex 2010-07-22 05:54:00.000 EA
Vince 2010-07-22 07:59:00.000 30
Note that the building column has at least one non-numeric value. The range of buildings that report in changes over time, so I would prefer to avoid hard-coding of building values (a table that I can update would be fine).
Should I use a cursor or dynamic SQL to run a looping SELECT statement that checks for each building based on a control table listing each currently active building?
Any help would be appreciated.
Edit: spelling

You could create a stored procedure that checks for missing entries. The procedure could call raiserror to make the job fail. For example:
if OBJECT_ID('CheckBuildingEntries') is null
exec ('create procedure CheckBuildingEntries as select 1')
go
alter procedure CheckBuildingEntries(
#check_date datetime)
as
declare #missing_buildings int
select #missing_buildings = COUNT(*)
from Buildings as b
left join
YourTable as yt
on yt.Building = b.name
and dateadd(dd,0, datediff(dd,0,yt.RawDate)) =
dateadd(dd,0, datediff(dd,0,#check_date))
where yt.Building is null
if #missing_buildings > 0
begin
raiserror('OMG!', 16, 0)
end
go
An example scheduled job running at 4AM to check yesterday's entries:
declare #yesterday datetime
set #yesterday = dateadd(day, -1, GETDATE())
exec CheckBuildingEntries #yesterday
If an entry was missing, the job would fail. You could set it up to send you an email.
Test tables:
create table Buildings (id int identity, name varchar(50))
create table YourTable (Employee varchar(50), RawDate datetime,
Building varchar(50))
insert into Buildings (name)
select '2'
union all select '9'
union all select '12'
union all select 'EA'
union all select '30'
insert into YourTable (Employee, RawDate, Building)
select 'Bob', '2010-07-22 06:04:00.000', '2'
union all select 'Sally', '2010-07-22 01:00:00.000', '9'
union all select 'Jane', '2010-07-22 06:04:00.000', '12'
union all select 'Alex', '2010-07-22 05:54:00.000', 'EA'
union all select 'Vince', '2010-07-22 07:59:00.000', '30'

Recommendations:
Do use a control table for the buildings - you may find that one
already exists, if you use the Object
Explorer in SQL Server Management
Studio
Don't use a cursor or dynamic SQL to run a loop - use set based
commands instead, possibly something
like the following:
SELECT BCT.Building, COUNT(YDT.Building) Build
FROM dbo.BuildingControlTable BCT
LEFT JOIN dbo.YourDataTable YDT
ON BCT.Building = YDT.Building AND
CAST(FLOOR( CAST( GETDATE() AS FLOAT ) - 1 ) AS DATETIME ) =
CAST(FLOOR( CAST( YDT.RawDate AS FLOAT ) ) AS DATETIME )
GROUP BY BCT.Building

Related

SSRS Stepped reported based on number

Using SSRS with SQL Server 2008 R2 (Visual Studio environment).
I am trying to produce a stepped down report based on a level/value in a table on sql server. The level act as a indent position with sort_value been the recursive parent in the report.
Sample of table in SQL Server:
Sample of output required
OK, I've come up with a solution but please note the following before you proceed.
1. The process relies on the data being in the correct order, as per your sample data.
2. If this is your real data structure, I strongly recommend you review it.
OK, So the first things I did was recreate your table exactly as per example. I called the table Stepped as I couldn't think of anything else!
The following code can then be used as your dataset in SSRS but you can obviously just run the T-SQL directly to see the output.
-- Create a copy of the data with a row number. This means the input data MUST be in the correct order.
DECLARE #t TABLE(RowN int IDENTITY(1,1), Sort_Order int, [Level] int, Qty int, Currency varchar(20), Product varchar(20))
INSERT INTO #t (Sort_Order, [Level], Qty, Currency, Product)
SELECT * FROM Stepped
-- Update the table so each row where the sort_order is NULL will take the sort order from the row above
UPDATE a SET Sort_Order = b.Sort_Order
FROM #t a
JOIN #t b on a.RowN = b.rowN+1
WHERE a.Sort_Order is null and b.Sort_Order is not null
-- repeat this until we're done.
WHILE ##ROWCOUNT >0
BEGIN
UPDATE a SET Sort_Order = b.Sort_Order
FROM #t a
JOIN #t b on a.RowN = b.rowN+1
WHERE a.Sort_Order is null and b.Sort_Order is not null
END
-- Now we can select from our new table sorted by both sort oder and level.
-- We also separate out the products based on their level.
SELECT
CASE Level WHEN 1 THEN Product ELSE NULL END as ProdLvl_1
, CASE Level WHEN 2 THEN Product ELSE NULL END as ProdLvl_2
, CASE Level WHEN 3 THEN Product ELSE NULL END as ProdLvl_3
, QTY
, Currency
FROM #t s
ORDER BY Sort_Order, Level
The output looks like this...
You may also want to consider swapping out the final statement for this.
-- Alternatively use this style and use a single column in the report.
-- This is better if the number of levels can change.
SELECT
REPLICATE('--', Level-1) + Product as Product
, QTY
, Currency
FROM #t s
ORDER BY Sort_Order, Level
As this will give you a single column for 'product' indented like this.

Using data from SQL Server to generate report in Access?

I have a legacy VB6 application that's connected to SQL Server where all data is stored. I'm creating a report for the user where I'll need to pass 2 dates (FROM and TO). In SQL Server, I have created a few views that fulfill all my criteria and use the data from there to populate a dbo.tblTempTable (which is not actually #tempTable, but a normal table). For now the dates are hardcoded, and I'm struggling with finding a way of passing those dates. I'm pretty sure I am unable to pass parameters to a view. To populate the table I have a simple stored procedure that goes like this....
TRUNCATE TABLE dbo.tbl_TTableReport
INSERT INTO tbl_TTableReport (UserID, CompanyID, CompanyName, Sold, Voided, Returned, Subtotal)
SELECT
1234 AS USERID, CompanyID, CompanyName,
Sold, Voided, Returned, Subtotal
FROM dbo.vCompanyInfo
So in this select statement, every piece of data comes from my final view ..vCompanyInfo. Sold, Voided, and Returned need to be filtered by the FROM and TO dates. How would I do something like that?
EDITED. I'm trying to create a stored procedure but I'm getting some weird errors...I have never create a complex multi SELECT statement SP before so I'm still trying to work out the kinks.
This is me trynig to build a SP with select statement, but I keep seeing the same error : Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
TRUNCATE TABLE dbo.tbl_TTableReport
INSERT INTO tbl_TTableReport
(UserID
, CompanyID)
, CompanyName)
SELECT 1234 AS USERID
,(Select CompanyID from dbo.tblMainInv)
,(select CompanyName from dbo.tblCompanies)
For both the CompanyID and companyName there are about 30 values. I'm trying to have them get loaded into the table, I don't know where my error is.
I recommend you create an ADP Access project to connect to your SQL Server database. This allows you more control over the queries that you send to the Sql Server for reports and data entry forms. When creating the project file you designate the server and database with which you need to connect to. Then setting up the data source for your report is as simple as including the select statement to issue against the database into the data source of the report. You can also refer to text boxes on forms or have automatic prompts for parameters.
As for the error you are getting in your second query in your edit... you are using the sub-query as an expression. SQL Server is being nice in letting you use a sub-query as one of the values to output in your select statement, but it's expecting just that... one value. So you could do something like:
Select
(select count(*) from MyTableValues) as CountTotal,
(select sum(*) from MyTableValues) as SumTotal
However, what you are trying to do is return several rows when SQL Server is expecting you to return just one. If you want to join the data of two tables, you use a join... something like:
SELECT 1234 AS USERID, CompanyID, CompanyName
from dbo.tblMainInv
join dbo.tblCompanies
on tblMainInv.CompanyID = tblCompanies.CompanyID
Which is probably something that your view dbo.vCompanyInfo probably already looks like.
If your determined to use your temp table approach (although I highly recommend you reconsider as your just making more work for future maintenance)... this is something that your procedure might look like:
GO
CREATE PROCEDURE sp_LoadReportData
(
#UserID as int,
#StartDate as datetime,
#EndDate as datetime
) AS
delete from tbl_TTableReport
where USERID = #UserID
INSERT INTO tbl_TTableReport
(UserID, CompanyID, CompanyName, Sold, Voided, Returned, Subtotal)
SELECT
#UserID AS USERID,
CompanyID,
CompanyName,
Sold,
Voided,
Returned,
Subtotal
FROM vCompanyInfo
where
Sold between #StartDate and #EndDate
and Voided between #StartDate and #EndDate
and Returned between #StartDate and #EndDate
GO
You would then call the procedure with the filters you want and send the UserID as a parameter like this:
exec sp_LoadReportData 1234, '1/1/2016', '1/20/2016'
But this still doesn't solve your problem... because you'll still have to send the UserID as a parameter in your report's data source like:
select * from tbl_TTableReport where UserID=1234
Unless you want to create a table and procedure for every user, you should really rethink your strategy and learn how to do it the standard way.

How do I setup a daily archive job in SQL server to keep my DB small and quick?

I have a DB in SQL server and one of the tables recieves a large amount of data every day (+100 000). The data is reported on, but my client only needs it for 7 days. On the odd occasion he will require access to historic data, but this can be ignored.
I need to ensure that my primary lookup table stays as small as can be (so that my queries are as quick as possible), and any data older than 7 days goes into a secondary (archiving) table, within the same database. Data feeds in consistently to my primary table throughout the day from a variety of data sources.
How would I go about performing this? I managed to get to the code below through using other questions, butI am now recieving an error ("Msg 8101, Level 16, State 1, Line 12
An explicit value for the identity column in table 'dbo.Archived Data Import' can only be specified when a column list is used and IDENTITY_INSERT is ON. ").
Below is my current code:
DECLARE #NextIDs TABLE(IndexID int primary key)
DECLARE #7daysago datetime
SELECT #7daysago = DATEADD(d, -7, GetDate())
WHILE EXISTS(SELECT 1 FROM [dbo].[Data Import] WHERE [Data Import].[Receive Date] < #7daysago)
BEGIN
BEGIN TRAN
INSERT INTO #NextIDs(IndexID)
SELECT TOP 10000 IndexID FROM [dbo].[Data Import] WHERE [Data Import].[Receive Date] < #7daysago
INSERT INTO [dbo].[Archived Data Import]
SELECT *
FROM [dbo].[Data Import] AS a
INNER JOIN #NextIDs AS b ON a.IndexID = b.IndexID
DELETE [dbo].[Data Import]
FROM [dbo].[Data Import]
INNER JOIN #NextIDs AS b ON a.IndexID = b.IndexID
DELETE FROM #NextIDs
COMMIT TRAN
END
What am I doing wrong here? Im using SQL server 2012 Express, so cannot partition (which would be ideal).
Beyond this, how do I turn this into a daily recurring task? Any help would be much appreciated.
An explicit value for the identity column in table 'dbo.Archived Data Import' can only be specified when a column list is used and IDENTITY_INSERT is ON
So... set identity insert on. Also, use DELETE ... OUTPUT INTO ... rather than SELECT ->
INSERT -> DELETE.
DECLARE #7daysago datetime
SELECT #7daysago = DATEADD(d, -7, GetDate());
SET IDENTITY_INSERT [dbo].[Archived Data Import] ON;
WITH CTE as (
SELECT TOP 10000 *
FROM [dbo].[Data Import]
WHERE [Data Import].[Receive Date] < #7daysago)
DELETE CTE
OUTPUT DELETED.id, DELTED.col1, DELETED.col2, ...
INTO [dbo].[Archived Data Import] (id, col1, col2, ....);
Beyond this, how do I turn this into a daily recurring task?
Use conversation timers and activated procedures. See Scheduling Jobs in SQL Server Express.
Without seeing your Table definitions, I am going to assume that your archive table has the same definition as your current table. Am I right in assuming that You have an identity column as Archived Data Import.IndexID? If so, switch it to ba an int large enough to hold expected values.
In order to schedule, this you will need to create a bat file to run this procedure and schedule it with windows scheduler.

SQL running sum for an MVC application

I need a faster method to calculate and display a running sum.
It's an MVC telerik grid that queries a view that generates a running sum using a sub-query. The query takes 73 seconds to complete, which is unacceptable. (Every time the user hits "Refresh Forecast Sheet", it takes 73 seconds to re-populate the grid.)
The query looks like this:
SELECT outside.EffectiveDate
[omitted for clarity]
,(
SELECT SUM(b.Amount)
FROM vCI_UNIONALL inside
WHERE inside.EffectiveDate <= outside.EffectiveDate
) AS RunningBalance
[omitted for clarity]
FROM vCI_UNIONALL outside
"EffectiveDate" on certain items can change all the time... New items can get added, etc. I certainly need something that can calculate the running sum on the fly (when the Refresh button is hit). Stored proc or another View...? Please advise.
Solution: (one of many, this one is orders of magnitude faster than a sub-query)
Create a new table with all the columns in the view except for the RunningTotal col. Create a stored procedure that first truncates the table, then INSERT INTO the table using SELECT all columns, without the running sum column.
Use update local variable method:
DECLARE #Amount DECIMAL(18,4)
SET #Amount = 0
UPDATE TABLE_YOU_JUST_CREATED SET RunningTotal = #Amount, #Amount = #Amount + ISNULL(Amount,0)
Create a task agent that will run the stored procedure once a day. Use the TABLE_YOU_JUST_CREATED for all your reports.
Take a look at this post
Calculate a Running Total in SQL Server
If you have SQL Server Denali, you can use new windowed function.
In SQL Server 2008 R2 I suggest you to use recursive common table expression.
Small problem in CTE is that for fast query you have to have identity column without gaps (1, 2, 3,...) and if you don't have such a column you have to create a temporary or variable table with such a column and to move you your data there.
CTE approach will be something like this
declare #Temp_Numbers (RowNum int, Amount <your type>, EffectiveDate datetime)
insert into #Temp_Numbers (RowNum, Amount, EffectiveDate)
select row_number() over (order by EffectiveDate), Amount, EffectiveDate
from vCI_UNIONALL
-- you can also use identity
-- declare #Temp_Numbers (RowNum int identity(1, 1), Amount <your type>, EffectiveDate datetime)
-- insert into #Temp_Numbers (Amount, EffectiveDate)
-- select Amount, EffectiveDate
-- from vCI_UNIONALL
-- order by EffectiveDate
;with
CTE_RunningTotal
as
(
select T.RowNum, T.EffectiveDate, T.Amount as Total_Amount
from #Temp_Numbers as T
where T.RowNum = 1
union all
select T.RowNum, T.EffectiveDate, T.Amount + C.Total_Amount as Total_Amount
from CTE_RunningTotal as C
inner join #Temp_Numbers as T on T.RowNum = C.RowNum + 1
)
select C.RowNum, C.EffectiveDate, C.Total_Amount
from CTE_RunningTotal as C
option (maxrecursion 0)
There're may be some questions with duplicates EffectiveDate values, it depends on how you want to work with them - do you want to them to be ordered arbitrarily or do you want them to have equal Amount?

In SQL Server, how do I set the next value for an autoincrement field to an arbitrary value like you can in Postgres?

Is it possible to set the next value of an autoincrement field in SQL Server like you can do in Postgres?
For the curious, here's the whole backstory. My company used to use Postgres, which allows you to easily set the next value of an autoincrement field to an arbitrary value.
New company bought old company, and now we're importing Postgres data to SQL Server. Somehow the autoincremented AcctID field on Accounts got set to a 9-digit number even though there are thousands of 8 digit numbers to be had. Apparently someone did this a while back in Postgres for some now unknown reason.
So now in the new SQL Server database, new accounts are having 9-digit account ids, but the client's accounting software can't deal with 9-digit account numbers, so any new accounts they add can't be processed by their accounting department until this gets resolved.
Of course, there are up to 72 different tables which can have dependencies on the AcctID field of Accounts, and the client created about 360 new accounts before they realized the problems involved, so saving that data, truncating the table, and reinserting the data would be an onerous task.
Much better would be to set the autoincrement value of AcctID to the last 8-digit value + 1. Then at least they'd be able to add new accounts while a solution to the 9-digit accounts was being worked on. In fact they claim they only need 3 of the 360 accounts they've added.
So is it possible to reset the autoincrement value of a field in SQL Server like you can do in Postgres?
In SQL Server you can reset an autoincrement column like this:
dbcc checkident ( table_name, RESEED, new_value )
You can check MSDN's documentation about it here.
You can do this too:
CREATE TABLE #myTable
(
ID INT IDENTITY,
abc VARCHAR(20)
)
INSERT INTO #myTable
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'cba'
SELECT *
FROM #myTable
-- Jump Identities
SET IDENTITY_INSERT #myTable ON
INSERT INTO #myTable
( id, abc )
VALUES ( 50, 'cbd' )
SELECT *
FROM #myTable
SET IDENTITY_INSERT #myTable OFF
-- Back to contigious
INSERT INTO #myTable
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'abc'
UNION ALL
SELECT 'cba'
SELECT *
FROM #myTable
DROP TABLE #myTable

Resources