The SQL Agent Jobs sits above the user level and requires a login to be assigned to the owner. But it doesn't take a group login as an accepted parameter. I need to use the Windows AD group as owner because I have different SQL users and some of them should see only the specific jobs. As now Ive created separate jobs for every user using SQLAgentUserRole which is not good for sure and the database is full of 1:1 jobs, each of them with different owner to avoid seeing the other jobs.
The whole picture:
Lets say that I have 10 different jobs in the database. One of those jobs is named UserJob. I want specific users when connecting to the database and expand the jobs section to see ONLY the job named "UserJob" and be able to start it. I dont need it via Stored procedure, etc. I just need to start the job via the SSMS (right click, start job, enter parameters if needed). Thanks.
As per the docs SSMS checks user membership in the following Database Roles to show SQL Server Agent tree node:
SQLAgentUserRole
SQLAgentReaderRole
SQLAgentOperatorRole
I used SQL Server Profiler to find what queries are executed when you first connect to database in Object Browser and expand various nodes.
For SQL Server Agent it uses SELECT * FROM msdb.dbo.sysjobs_view view to list Jobs. This view can be modified.
Changes
Create a new Database Role in msdb database. I called it "CustomJobRole".
I then created a new Job (I assume you already have a Job) called "TestJob"
Create a low privilege user that should be able to see and run only "TestJob".
Add this user to "CustomJobRole" and "SQLAgentReaderRole" and/or "SQLAgentOperatorRole" (see linked above docs for details)
Modify sysjobs_view as follows:
(see comments in code)
ALTER VIEW sysjobs_view
AS
SELECT jobs.job_id,
svr.originating_server,
jobs.name,
jobs.enabled,
jobs.description,
jobs.start_step_id,
jobs.category_id,
jobs.owner_sid,
jobs.notify_level_eventlog,
jobs.notify_level_email,
jobs.notify_level_netsend,
jobs.notify_level_page,
jobs.notify_email_operator_id,
jobs.notify_netsend_operator_id,
jobs.notify_page_operator_id,
jobs.delete_level,
jobs.date_created,
jobs.date_modified,
jobs.version_number,
jobs.originating_server_id,
svr.master_server
FROM msdb.dbo.sysjobs as jobs
JOIN msdb.dbo.sysoriginatingservers_view as svr
ON jobs.originating_server_id = svr.originating_server_id
--LEFT JOIN msdb.dbo.sysjobservers js ON jobs.job_id = js.job_id
WHERE
-- Custom: Add Condition for your Custom Role and Job Name
( (ISNULL(IS_MEMBER(N'CustomJobRole'), 0) = 1) AND jobs.name = 'TestJob' )
OR (owner_sid = SUSER_SID())
OR (ISNULL(IS_SRVROLEMEMBER(N'sysadmin'), 0) = 1)
-- Custom: In order for users to be able to see and start Jobs they have to be members of SQLAgentReaderRole/SQLAgentOperatorRole
-- but these roles gives them ability to see all jobs so add an exclusion
OR ( ISNULL(IS_MEMBER(N'SQLAgentReaderRole'), 0) = 1 AND ISNULL( IS_MEMBER(N'CustomJobRole'), 0 ) = 0 )
OR ( (ISNULL(IS_MEMBER(N'TargetServersRole'), 0) = 1) AND
(EXISTS(SELECT * FROM msdb.dbo.sysjobservers js
WHERE js.server_id <> 0 AND js.job_id = jobs.job_id))) -- filter out local jobs
Note: commented out LEFT JOIN is original code and has nothing to do with the solution.
Summary
This method is "hacky" as it only modifies the job list for certain users and does not actually prevent them from running other jobs via code, in other words this does not offer any security, just convenience of clean UI. Implementation is simple but, obviously not scalable: Job name is hard-coded and negative membership presence is used (i.e. AND ISNULL( IS_MEMBER(N'CustomJobRole'), 0 ) = 0). IMO, it is the simplest and most reliable (least side effects) method though.
Tested on
SSMS v18.9.2 + SQL Server 2014 SP3
Editing Job Step Workaround
It is not possible to modify Job Step unless you are a Job Owner or a Sysadmin.
One, even more "hacky", way to work around this problem is to create a table that would hold all input parameters and give users insert/update access to this table. Your SP can then read parameters from this table. It should be easy for Users to Right-Click -> Edit on the table and modify data.
For the table structure I would recommend the following:
Assuming you have relatively few parameters I suggest that you create one column per parameter. This way you have correct data types for each
parameter.
Add an After Insert / Delete trigger to the table to ensure
that the table always has exactly one row of data.
Related
On the ETL server I have a DW user table.
On the prod OLTP server I have the sales database. I want to pull the sales only for users that are present in the user table on the ETL server.
Presently I am using an execute SQL task to fetch the DW users into a SSIS System.Object variable. Then using a for each loop to loop through each item (userid) in this variable and via a data flow task fetch the OLTP sales table for each user and dump it into the DW staging table. The for each is taking long time to run.
I want to be able to do an inner join so that the response is quicker, but I cant do this since they are on separate servers. Neither can I use a global temp table to make the inner join, for the same reason.
I tried to collect the DW users into a comma separated string variable and then using it (via string_split) to query into OLTP, but this is also taking more time at the pre-execute phase (not sure why exactly) even for small number of users.
I also am aware of lookup transform but that too will result in all oltp rows to be brought into the dw etl server to test the lookup condition.
Is there any alternate approach to be able to do an inner join by taking the list of users into the source?
Note: I do not have write permissions on the OLTP db.
Based on the comments, I think we can use a temporary table to solve this.
Can you help me understand this restriction? "Neither can I use a global temp table to make the inner join, for the same reason."
The restriction is since oltp server and dw server are separate so can't have global temp table common to both servers. Hope makes sense.
The general pattern we're going to do is
Execute SQL Task to create a temporary table on the OLTP server
A Data Flow task to populate the new temporary table. Source = DW. Destination = OLTP. Ensure Delay Validation = True
Modify existing Data Flow. Modify source to be a query that uses the temporary table i.e. SELECT S.* FROM oltp.sales AS S WHERE EXISTS (SELECT * FROM #SalesPerson AS SP WHERE SP.UserId = S.UserId); Ensure Delay Validation = True
A long form answer on using temporary tables (global to set the metadata, regular thereafter)
I don't use temp table in SSIS
Temporary tables, live in tempdb. Your OLTP and DW connection managers likely do not point to tempdb. To be able to reference a temporary table, local or global, in SSIS you need to either define an additional connection manager for the same server that points explicitly at tempdb so you can use the drop down in the source/destination components (technically accurate but dumb). Or, you use an SSIS Variable to hold the name of the table and use the ~From Variable~ named option in source/destination component (best option, maximum flexibility).
Soup to nuts example
I will use WideWorldImporters as my OLTP system and WideWorldImportersDW as my DW system.
One-time task
Open SQL Server Management Studio, SSMS, and connect to your OLTP system. Define a global temporary table with a unique name and the expected structure. Leave your connection open so the table structure remains intact during initial development.
I used the following statement.
DROP TABLE IF EXISTS #SO_70530036;
CREATE TABLE #SO_70530036(EmployeeId int NOT NULL);
Keep track of your query because we'll use it later on but as I advocate in my SSIS answers, perform the smallest task, test that it works and then go on to the next. It's the only way to debug.
Connection Managers
Define two OLE DB Connection Managers. WWI_DW uses points to the named instance DEV2019UTF8 and WWI_OLTP points to DEV2019EXPRESS. Right click on WWI_OLTP and select Properties. Find the property RetainSameConnection and flip that from the default of False to True. This ensures the same connection is used throughout the package. As temporary tables go out of scope when the connection goes away, closing and reopening a connection in a package will result in a fatal error.
These two databases on different instances so we can't cheat and directly comingle data.
Variables
Define 4 variables in SSIS, all of type String.
TempTableName - I used a value of ##SO_70530036 but use whatever value you specified in the One-time task section.
QuerySourceEmployees - This will be the query you run to generate the candidate set of data to go into the temporary table. I used SELECT TOP (3) E.[WWI Employee ID] AS EmployeeId FROM Dimension.Employee AS E WHERE E.[Is SalesPerson] = CAST(1 AS bit);
QueryDefineTables - Remember the drop/create statements from the on-time task? We're going to use the essence of them but use the expression builder to let us dynamically swap the table name. I clicked the ellipses, ..., on the Expression section and used the following "DROP TABLE IF EXISTS " + #[User::TempTableName] + "; CREATE TABLE " + #[User::TempTableName] + "( EmployeeId int NOT NULL);" You should be able to copy the Value from the row and paste it into SSMS to confirm it works.
QuerySales - This is the actual query you're going to use to pull your filtered set of sales data. Again, we'll use the Expression to allow us to dynamically reference the temporary table name. The prettified version of the expression would look something like
"SELECT
SI.InvoiceID
, SI.SalespersonPersonID
, SO.OrderID
, SOL.StockItemID
, SOL.Quantity
, SOL.OrderLineID
FROM
Sales.Invoices AS SI
INNER JOIN
Sales.Orders AS SO
ON SO.OrderID = SI.OrderID
INNER JOIN
Sales.OrderLines AS SOL
ON SO.OrderID = SOL.OrderID
WHERE
EXISTS (SELECT * FROM " + #[User::TempTableName] + " AS TT WHERE TT.EmployeeID = SI.SalespersonPersonID);"
Again, you should be able to pull the Value from the three queries and run them independently and verify they work.
Execute SQL Task
Add an Execute SQL task to the Control Flow. I named mine SQL Create temporary table My Connection Manager is WWI_OLTP and I changed the SQLSourceType to Variable and the SourceVariable is User::QueryDefineTables
Every time your package runs, the first thing it will do is establish create the temporary table. Which is good because SSIS is a metadata driven ETL engine and the next two steps would fail if the table didn't exist.
Data Flow Task - Prime the pump
This data flow is where we'll transfer DW data back to the OLTP system so can filter in the source system.
Drag a Data Flow Task onto the Control Flow. I named mine DFT Load Temp and before you click into it, right click on the Task and find the DelayValidation property and change this from the default of False to True. Normally, a package validates all metadata before actual execution begins as the idea is you want to know everything is good before any data starts moving. Since we're using temporary tables, we need to tell the execution engine "trust us, it'll be ready"
Double click inside the Data Flow Task.
Add an OLE DB Source. I named mine OLESRC SourceEmployees I use the connection manager WWI_DW. My data access mode changes to SQL command from variable and then I select my variable User::QuerySourceEmployees
Add an OLE DB Destination. I named mine OLEDST TempTableName and double clicked to configure it. The Connection Manager is WWI_OLTP and again, since the table lives in tempdb, we can't select it from the drop down. Change the Data access mode to Table name or view name variable - fast load and then select your variable name User::TempTableName. Click the Mapping tab and ensure source columns map to destination columns.
Data Flow Task - Transfer data
Finally, we will pull our source data, nicely filtered against the data from our target system.
Add an OLE DB Source. I named it OLESRC QuerySales. The Connection Manager is WWI_OLTP. Data access mode again changes to SQL command from variable and the variable name is User::QuerySales
From here, do whatever else you need to do to make the magic happen.
Instead of having 270k rows with an unfiltered query
I have 67k as there are only 3 employees in the temporary table.
Reference package
But wait, there's more!
Close out visual studio, open it back up and try to touch something in the data flows. Suddenly, there are red Xs everywhere! Any time you close a data flow component, it fires a revalidate metadata operation and guess what, it can't do that as the connection to the temporary table is gone.
The package will run fine, it will not throw VS_NEEDSNEWMETADATA but editing/maintenance becomes a pain.
If you switched from global temporary table to local, switch the table name variable's value back to a global and then run the define statement in SSMS. Once that's done, then you can continue editing the package.
I assure you, the local temporary table does work once you have the metadata set and you use queries via variables for source/destination.
No need for the global temporary table hack, or the SET FMTONLY OFF hack (which no longer works).
Just specify the result set metadata in the SQL query with WITH RESULT SETS. eg
EXEC ('
create table #t
(
ID INT,
Name VARCHAR(150),
Number VARCHAR(15)
)
insert into #t (Id, Name, Number)
select object_id, name, 12
from sys.objects
select * from #t
')
WITH RESULT SETS
(
(
ID INT,
Name VARCHAR(150),
Number VARCHAR(15)
)
)
If you need to parameterize the query, there's a bit of a catch because there are some limitations in how SSIS discovers parameters. SSIS runs sp_describe_undeclared_parameters, which doesn't really work with batches that call sp_executesql, because sp_executesql has a very unique way it handles parameters, one which you couldn't replicate with a user stored procedure.
So to parameterize the query you'll either need to pass the parameter values into the query using the "query from variable" and SSIS expressions, or push all this TSQL into a stored procedure.
In my SQL Server 2012 database, I have a linked server reference to a second SQL Server database that I need to pull records from and update accordingly.
I have the following update statement that I am trying to run:
UPDATE
Linked_Tbl
SET
Transferred = 1
FROM
MyLinkedServer.dbo.MyTable Linked_Tbl
JOIN
MyTable Local_Tbl ON Local_Tbl.LinkedId = Linked_Tbl.Id
JOIN
MyOtherTable Local_Tbl2 ON Local_Tbl.LocalId = Local_Tbl2.LocalId
Which I had to stop after an hour of running as it was still executing.
I've read online and found solutions stating that the best solution is to create a stored procedure on the Linked Server itself to execute the update statement rather than run it over the wire.
The problems I have are:
I don't have the ability to create any procedures on the other server.
Even if I could create that procedure, I would need to pass through all the Ids to the stored procedure for the update and I'm not sure how to do that efficiently with thousands of Ids (this, obviously, is the smaller of the issues, though since I can't create that procedure in the first place).
I'm hoping there are other solutions people may have managed to come up with given that it's often the case you don't have permissions to make changes to a different server.
Any ideas??
I am not sure, whether it can give more performance, you an try:
UPDATE
Linked_Tbl
SET
Transferred = 1
FROM OPENDATASOURCE([MyLinkedServer],'select Id, LocalId,Transferred from remotedb.dbo.MyTable') AS Linked_Tbl
JOIN MyTable Local_Tbl
ON Local_Tbl.LinkedId = Linked_Tbl.Id
JOIN MyOtherTable Local_Tbl2
ON Local_Tbl.LocalId = Local_Tbl2.LocalId
Now that SQL Server 2016 enables SSISDB to be fully High Available, I have a question regarding the job setup.
When I do create a SQL Agent Job that executes a SSIS Package that is deployed in SSISDB, should in the job step the Server be the Listener Name or the physical host name?
I am asking that because if I use the physical host name and create the job in both replicas, the secondary jobs will always fail because the DB is in read only mode. I didn't try placing the Listener name yet, because I wanted to get opinions first.
The server name should be listener name ,if you follow this approach,it is enough to deploy job in one instance
you also can use Physical host names and deploy jobs in all instances,provided you have below piece of code as first step
- fn_hadr_group_is_primary
USE master;
GO
IF OBJECT_ID('dbo.fn_hadr_group_is_primary', 'FN') IS NOT NULL
DROP FUNCTION dbo.fn_hadr_group_is_primary;
GO
CREATE FUNCTION dbo.fn_hadr_group_is_primary (#AGName sysname)
RETURNS bit
AS
BEGIN;
DECLARE #PrimaryReplica sysname;
SELECT
#PrimaryReplica = hags.primary_replica
FROM sys.dm_hadr_availability_group_states hags
INNER JOIN sys.availability_groups ag ON ag.group_id = hags.group_id
WHERE ag.name = #AGName;
IF UPPER(#PrimaryReplica) = UPPER(##SERVERNAME)
RETURN 1; -- primary
RETURN 0; -- not primary
END;
This post also deals with some of the common issues,that needs to be taken care off
https://blogs.msdn.microsoft.com/mattm/2012/09/19/ssis-with-alwayson/
When I want to select from table Y in database X I can use
select * from [X].[dbo].[Y]
or
USE X
select * from [Y]
Is there any reason to prefer one over the other?
dbo
Using dbo as the owner of all the database objects can simplify managing the objects. You will always have a dbo user in the database. Users in the database will be able to access any object owned by dbo without specifying the owner as long as the user has appropriate permission.
USE X
When a SQL Server login connects to SQL Server, the login is automatically connected to its default database and acquires the security context of a database user. If no database user has been created for the SQL Server login, the login connects as guest. If the database user does not have CONNECT permission on the database, the USE statement will fail. If no default database has been assigned to the login, its default database will be set to master.
Understanding the Difference between Owners and Schemas in SQL Server
USE (Transact-SQL)
I'd tend to use [server].[database].[schema].[table] in instances where a script may query mutliple tables from multiple databases.
The USE [database] would typically be used in scenarios where all statements were to apply to the same database and you needed to make sure they were applied to the correct database. Have you ever connected to a server and run a script only to find you ran it on the master database?
USE X will change the context to X and all the following statements will execute under the context X.
But X.dbo.Y will access the object Y without changing the current context.
Eg: Let us consider there is two databases DB1 and DB2. DB1 contains table T1 & T2 and DB2 contains tables U1 & U2.
Now,
USE DB1 -- here context set to DB1
select * from T1 -- works fine
select * from U1 -- gives error, because U1 is not in current context
select * from DB2.dbo.U1 -- works fine, because it access the context DB2 from current content context DB1
select * from T2 -- works fine
USE DB2 -- here context changed to DB2
select * from U2 -- works fine
select * from T1 -- gives error, because T1 is not in current context
select * from DB1.dbo.T1 -- works fine, because it access the context DB1 from current content context DB2
by using first Query you can perform that selection from other databases.In the same window you can have the selection for other data beses also.
But By using second selection, From the same window you can have selection for that(USE X) databse only.
Sometimes you want the schema and database to be dictated by the login, and in this case you should simply use the object name. That's one reason to not fully qualify them.
I have a database with multiple tables and I want to log the users activity via my MVC 3 web application.
User X updated category HELLO. Name changed from 'HELLO' to 'Hi There' on 24/04/2011
User Y deleted vehicle Test on 24/04/2011.
User Z updated vehicle Bla. Name changed from 'Blu' to 'Bla' on 24/04/2011.
User Z updated vehicle Bla. Wheels changed from 'WheelsX' to 'WheelsY' on 24/04/2011.
User Z updated vehicle Bla. BuildProgress changed from '20' to '50' on 24/04/2011
My initial idea is to have on all of my actions that have database crud, to add a couple lines of code that would enter those strings in a table.
Is there a better way of checking which table and column has been modified than to check every column one by one with if statements (first I select the current values, then check each of them with the value of the textbox) I did that for another ASPX web app and it was painful.
Now that I'm using MVC and ADO.NET Entity Data Model I'm wondering if a faster way to find the columns that were changed and build a log like the one above.
You can also accomplish this by putting your database into full recovery mode and then reading the transaction log.
When database is in a full recovery mode then sql server logs all Update, insert and delete (and others such as create, alter, drop..) statements into it's transaction log.
So, using this approach you dont need to make any additinal changes to your application or your database structure.
But you will need 3rd party sql transaction log reader. Red gate has a free solution for sql server 2000 only. If your server is 2005 or higher you would probably want to go with ApexSQL Log
Also, this approach will not be able to audit select statements but it's definately the easiest to implement if you dont really need to audit select queries.
The way I see, you have two options:
Create triggers in the database side, mapping changes in a table by table basis and getting result into a Log table
OR
Having the code handle the changes. You would have a base class with data and with reflection you could iterate all object properties and see what has changed. And then save that into your Log table. Of course, that coding would be on your Data Access Layer.
By the way, if you have a good code structure/architecture, I would go with the second option.
You could have a trigger (AFTER insert/update/deelte) on each table you want to monitor. The beauty is columns_updated() which returns a barbinary value, indicating which columns have been updated.
Here is some snippet of code that I put in each trigger:
IF (##ROWCOUNT = 0) return
declare #AuditType_ID int ,
#AuditDate datetime ,
#AuditUserName varchar(128),
#AuditBitMask varbinary(10)
select #AuditDate = getdate() ,
#AuditUserNAme = system_user,
#AuditBitMask = columns_updated()
-- Determine modification type
IF (exists (select 1 from inserted) and exists (select 1 from deleted))
select #AuditType_ID = 2 -- UPDATE
ELSE IF (exists (select * from inserted))
select #AuditType_ID = 1 -- INSERT
ELSE
select #AuditType_ID = 3 -- DELETE
(record this data to your table of choice)
I have a special function that can decode the bitmask values, but for some reason it is not pasting well here. Message me and I'll email it to you.