I'm wondering if the current process I'm using to update a table of user's (tblUsers) Windows ID's (NTID) is a good method. I'm wondering because LDAP will only return 1000 rows I believe, so that prevents me from just doing it all in one query.
tlbUsers has about 160,000 rows. I'm querying LDAP to update the NTID of each record in tblUsers. I'm using a linked server to ADSI to view LDAP data. My process uses two stored procedures, one for getting a WindowsID from LDAP (LdapPackage.GetUserNTID), another for updating the rows in tblUsers (LdapPackage.UpdateUserNTID).
The code below works for updating the table, however, it's pretty slow. It would seem to me this isn't the best way of doing it, that if I wanted to do a batch update like this from LDAP, there should be a simpler way than updating a record at a time.
This previous post gave an interesting example using UNION's to get around the 1000 record limit, but it only works if each query returns less than 1000 records, which at a large company would probably require lots of UNIONS... at least that's my initial take on it.
Querying Active Directory from SQL Server 2005
Thanks in advance guys!!!
<code>
CREATE PROCEDURE LdapPackage.GetUserNTID
(
#EmployeeID INT,
#OutNTID VARCHAR(20) OUTPUT
)
AS
BEGIN
DECLARE #SQLString NVARCHAR(MAX)
DECLARE #ParmDefinition NVARCHAR(MAX)
DECLARE #LdapFilter NVARCHAR(100)
--DECLARE #NTID VARCHAR(20)
SET #LdapFilter = 'employeeNumber = ' + CAST(#EmployeeID AS NVARCHAR(20))
SET #SQLString = 'SELECT DISTINCT #pNTID = samAccountName
FROM OPENQUERY(LDAP,
''select samAccountName, Mail
from ''''GC://domain.company.com''''
where objectClass=''''user'''' AND objectCategory=''''person'''' and ' + #LdapFilter + ''')
WHERE Mail IS NOT NULL'
SET #ParmDefinition = N'#pNTID varchar(20) OUTPUT'
EXECUTE sp_executesql
#SQLString,
#ParmDefinition,
#pNTID=#OutNTID OUTPUT
--SELECT NTID = #OutNTID
END
</code>
<code>
CREATE PROCEDURE LdapPackage.UpdateUserNTID
AS
BEGIN
DECLARE #EmployeeID AS INT
DECLARE #NTID AS VARCHAR(20)
DECLARE #RowCount AS INT
DECLARE #SQLString AS NVARCHAR(MAX)
DECLARE #ParmDefinition AS NVARCHAR(200)
SET #RowCount = 1
DECLARE Persons CURSOR
FOR SELECT DISTINCT EmployeeID FROM tblUsers
OPEN Persons
FETCH NEXT FROM Persons INTO #EmployeeID
WHILE ##FETCH_STATUS = 0
BEGIN
--GET NTID
SET #SQLString =N'EXEC LdapPackage.GetUserNTID #pEmployeeID, #pNTID OUTPUT'
SET #ParmDefinition =N'#pEmployeeID INT, #pNTID VARCHAR(20) OUTPUT'
EXECUTE sp_executesql
#SQLString,
#ParmDefinition,
#pEmployeeID=#EmployeeID,
#pNTID=#NTID OUTPUT
--UPDATE NTID
/*PRINT 'RowCount = ' + CAST(#RowCount AS VARCHAR(10))
PRINT 'EmployeeID = ' + CAST(#EmployeeID AS VARCHAR(20))
PRINT 'NTID = ' + #NTID
PRINT '-----------------------------'*/
UPDATE tblUsers
SET NTID = #NTID
WHERE EmployeeID = #EmployeeID
SET #RowCount = #RowCount + 1
FETCH NEXT FROM Persons INTO #EmployeeID
END
CLOSE Persons
DEALLOCATE Persons
END
</code>
my solution here was to have my that linked servers record limit to LDAP increased by the system admin. I would have preferred to have identified some sort of SQL Server interface like Oracle appears to have... so maybe I'll get to that in the future.
Related
I'm trying to build a stored procedure that will query multiple database depending on the databases required.
For example:
SP_Users takes a list of #DATABASES as parameters.
For each database it needs to run the same query and union the results together.
I believe a CTE could be my best bet so I have something like this at the moment.
SET #DATABASES = 'DB_1, DB_2' -- Two databases in a string listed
-- I have a split string function that will extract each database
SET #CURRENT_DB = 'DB_1'
WITH UsersCTE (Name, Email)
AS (SELECT Name, Email
FROM [#CURRENT_DB].[dbo].Users),
SELECT #DATABASE as DB, Name, Email
FROM UsersCTE
What I don't want to do is hard code the databases in the query. The steps I image are:
Split the parameter #DATABASES to extract and set the #CURRENT_DB Variable
Iterate through the query with a Recursive CTE until all the #DATABASES have been processed
Union all results together and return the data.
Not sure if this is the right approach to tackling this problem.
Using #databases:
As mentioned in the comments to your question, variables cant be used to dynamically select a database. Dynamic sql is indicated. You can start by building your template sql statement:
declare #sql nvarchar(max) =
'union all ' +
'select ''#db'' as db, name, email ' +
'from [#db].dbo.users ';
Since you have sql server 2016, you can split using the string_split function, with your #databases variable as input. This will result in a table with 'value' as the column name, which holds the database names.
Use the replace function to replace #db in the template with value. This will result in one sql statement for each database you passed into #databases. Then, concatenate the statements back together. Unfortunately, in version 2016, there's no built in function to do that. So we have to use the famous for xml trick to join the statements, then we use .value to convert it to a string, and finally we use stuff to get rid of the leading union all statement.
Take the results of the concatenated output, and overwrite the #sql variable. It is ready to go at this point, so execute it.
I do all that is described in this code:
declare #databases nvarchar(max) = 'db_1,db_2';
set #sql = stuff(
(
select replace(#sql, '#db', value)
from string_split(#databases, ',')
for xml path(''), type
).value('.[1]', 'nvarchar(max)')
, 1, 9, '');
exec(#sql);
Untested, of course, but if you print instead of execute, it seems to give the proper sql statement for your needs.
Using msForEachDB:
Now, if you didn't want to have to know which databases had 'users', such as if you're in an environment where you have a different database for every client, you can use sp_msForEachDb and check the structure first to make sure it has a 'users' table with 'name' and 'email' columns. If so, execute the appropriate statement. If not, execute a dummy statement. I won't describe this one, I'll just give the code:
declare #aggregator table (
db sysname,
name int,
email nvarchar(255)
);
insert #aggregator
exec sp_msforeachdb '
declare #sql nvarchar(max) = ''select db = '''''''', name = '''''''', email = '''''''' where 1 = 2'';
select #sql = ''select db = ''''?'''', name, email from ['' + table_catalog + ''].dbo.users''
from [?].information_schema.columns
where table_schema = ''dbo''
and table_name = ''users''
and column_name in (''name'', ''email'')
group by table_catalog
having count(*) = 2
exec (#sql);
';
select *
from #aggregator
I took the valid advice from others here and went with this which works great for what I need:
I decided to use a loop to build the query up. Hope this helps someone else looking to do something similar.
CREATE PROCEDURE [dbo].[SP_Users](
#DATABASES VARCHAR(MAX) = NULL,
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)
)
BEGIN
SET NOCOUNT ON;
--Local variables
DECLARE
#COUNTER INT = 0,
#SQL NVARCHAR(MAX) = '',
#CURRENTDB VARCHAR(50) = NULL,
#MAX INT = 0,
#ERRORMSG VARCHAR(MAX)
--Check we have databases entered
IF #DATABASES IS NULL
BEGIN
RAISERROR('ERROR: No Databases Provided,
Please Provide a list of databases to execute procedure. See stored procedure:
[SP_Users]', 16, 1)
RETURN
END
-- SET Number of iterations based on number of returned databases
SET #MAX = (SELECT COUNT(*) FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i)X)
-- Build SQL Statement
WHILE #COUNTER < #MAX
BEGIN
--Set the current database
SET #CURRENTDB = (SELECT X.Value FROM
(SELECT ROW_NUMBER() OVER (ORDER BY i.value) AS RowNumber, i.value
FROM dbo.udf_SplitVariable(#DATABASES, ',') AS i
ORDER BY RowNumber OFFSET #COUNTER
ROWS FETCH NEXT 1 ROWS ONLY) X);
SET #SQL = #SQL + N'
(
SELECT Name, Email
FROM [' + #CURRENTDB + '].[dbo].Users
WHERE
(Name = #PARAM1 OR #PARAM1 IS NULL)
(Email = #PARAM2 OR #PARAM2 IS NULL)
) '
+ N' UNION ALL '
END
PRINT #CURRENTDB
PRINT #SQL
SET #COUNTER = #COUNTER + 1
END
-- remove last N' UNION ALL '
IF LEN(#SQL) > 11
SET #SQL = LEFT(#SQL, LEN(#SQL) - 11)
EXEC sp_executesql #SQL, N'#CURRENTDB VARCHAR(50),
#PARAM1 VARCHAR(250),
#PARAM2 VARCHAR(250)',
#CURRENTDB,
#PARAM1 ,
#PARAM2
END
Split Variable Function
CREATE FUNCTION [dbo].[udf_SplitVariable]
(
#List varchar(8000),
#SplitOn varchar(5) = ','
)
RETURNS #RtnValue TABLE
(
Id INT IDENTITY(1,1),
Value VARCHAR(8000)
)
AS
BEGIN
--Account for ticks
SET #List = (REPLACE(#List, '''', ''))
--Account for 'emptynull'
IF LTRIM(RTRIM(#List)) = 'emptynull'
BEGIN
SET #List = ''
END
--Loop through all of the items in the string and add records for each item
WHILE (CHARINDEX(#SplitOn,#List)>0)
BEGIN
INSERT INTO #RtnValue (value)
SELECT Value = LTRIM(RTRIM(SUBSTRING(#List, 1, CHARINDEX(#SplitOn, #List)-1)))
SET #List = SUBSTRING(#List, CHARINDEX(#SplitOn,#List) + LEN(#SplitOn), LEN(#List))
END
INSERT INTO #RtnValue (Value)
SELECT Value = LTRIM(RTRIM(#List))
RETURN
END
Good day.
I have main task to do is next:
"Some" programm that has databases MonitorEDTest and DecNet in MSSQL 2012, i need to make an xml-file with detailed information from 4-5 tables in this database, which creates when StatusId of one entry becomes 150.
Now, i'm trying to solve this like this:
I made trigger on table with column StatusId, so it triggers when StatusId is 150. Made it with cursor for multiple entries with this status.
ALTER TRIGGER [dbo].[DT_Update_NEW]
ON [MonitorEDTest].[dbo].[LOG_DECL]
FOR UPDATE
AS
BEGIN
SET NOCOUNT ON;
if exists (select * from inserted where StatusId = 150)
begin
declare #nd nvarchar (100), #ProcessId nvarchar (100)
declare DTcursor cursor for
select [ND], [ProcessId] from inserted
open DTcursor;
FETCH NEXT FROM DTcursor into #nd, #ProcessId
WHILE ##FETCH_STATUS = 0
BEGIN
exec dbo.DT_info_Broker #nd, #ProcessId
FETCH NEXT FROM DTcursor into #nd, #ProcessId
END
CLOSE DTcursor
deallocate DTcursor
end
END
Then this trigger uses procedure dbo.DT_info_Broker that creates full xml-file with select...join..for xml statements, after that full xml code sends with service broker to himself (made ServiceBroker on base MonitorEDTest which contains table for trigger, and whole info takes from other DB DecNet).
ALTER PROCEDURE [dbo].[DT_Info_Broker]
#nd nvarchar (200), #ProcessId nvarchar (200)
AS
BEGIN
SET NOCOUNT ON;
declare #base nvarchar(100), #g07 nvarchar(1000), #sql nvarchar (MAX), #flag int, #code1 nvarchar (2000), #code2 nvarchar (1000), #code3 nvarchar (1000), #code4 nvarchar(1000), #code nvarchar(4000), #codeold nvarchar (1000),
#bat nvarchar(1000), #pwshell nvarchar(1000), #batdel nvarchar(1000), #msg xml
set #base = 'DecNet'
set #msg = (SELECT (select.....)for xml path(''), root('DT'))
declare #ch uniqueidentifier
begin dialog conversation #ch from service Monitor to service 'Monitor' on contract [Monitor_Contract] with encryption = off
select #ch, #nd, #g07, #msg
;send on conversation #ch message type [Monitor_Message] (#msg)
EXEC [MonitorEDTest].[dbo].[Que_Broker]
END
Then i had two ideas, first i tried to make activation procedure for Service Broker (SB) to make it like trigger for sending xml from 2., but it didn’t work, so after sending to SB I executed another procedure dbo.Que_Broker.
This procedure is similar to how people usually “receives” message from SB, code is above. Also on that procedure I take from ready xml-code one tag g07 for making name of xml output file like #g07.xml ~ 10000000-170718-0000000.xml
ALTER PROCEDURE [dbo].[Que_Broker]
AS
BEGIN
while 1 = 1
begin
declare #command nvarchar(2000), #xmlint int, #XmlText nvarchar(max), #g07 nvarchar (100)
declare #count int, #ch uniqueidentifier, #retvalue bit --, #msgtype sysname, #body varbinary(max)
select #count = count(*) from Monitor_Queue
if (#count = 0) break
set #xmlint = 0
select top (1) #XmlText = cast(message_body as nvarchar(max)), #ch = conversation_handle from Monitor_Queue where message_type_name = 'Monitor_Message' order by conversation_handle
exec sp_xml_preparedocument #xmlint OUTPUT, #XmlText
select #g07 = [text] from OPENXML (#xmlint, 'DT', 1) where [parentid] = 4
exec sp_XML_removedocument #xmlint
set #g07 = (select replace (#g07, '/', '-'))
set #command = 'bcp "receive top (1) cast(message_body as xml) from Monitor_Queue" queryout D:\' + #g07 + '.xml -T -x -c -C 1251 -d MonitorEDTest'
select #command, #g07
--waitfor delay '00:00:01'
EXEC master..xp_cmdshell #command
--break
end conversation #ch with cleanup
end
END
So the main problem is next, i wrote on last procedure code which goes to Queues of SB and while it has count>0 it makes bcp “receive top (1) cast(message_body as xml) from Monitor_Queue”...queryout “*.xml”
Then the conversation ends.
When i was checking how it works, everything worked fine, but bcp didn’t copy any entry of BS and when i check messages in table queue, the message still there. When i’m trying to execute that procedure manually bcp works fine.
output
------------
NULL
Starting copy...
NULL
0 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.) Total : 1
NULL
Also before idea with SB, i tried to do something like that with creating dynamic steps in job with sp_update_jobstep and sp_start_job, but it didn’t work, because of multiple entry within one time (like update top 5 set statusid = 150 where statusid = 150). Job couldn’t be opened by next entry because first didn’t finish.
Sorry for some mistakes, but first time writing here, hope to see some help and learn more of using sql, because I learned everything by myself and from some courses.
I'm facing deadlock
was deadlocked on lock resources with another process and has been
chosen as the deadlock victim.
problem In SQL-Server as i'm inserting data in database by picking max id against a specific column then add a increment got the value against which record will be inserted.
i'm calling a procedure as code mentioned below:
CREATE
PROCEDURE [dbo].[Web_GetMaxColumnID]
#Col_Name nvarchar(50)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
DECLARE #MaxID BIGINT;
SET NOCOUNT ON;
-- Insert statements for procedure here
BEGIN
BEGIN TRAN
SET #MaxID = (
SELECT Col_Counter
FROM Maintenance_Counter WITH (XLOCK, ROWLOCK)
WHERE COL_NAME = #Col_Name
)
UPDATE Maintenance_Counter
SET Col_Counter = #MaxID + 1
WHERE COL_NAME = #Col_Name
COMMIT
END
SELECT (
CONVERT(
VARCHAR,
(
SELECT office_id
FROM Maintenance
)
) + '' + CONVERT(VARCHAR, (#MaxID))
) AS MaxID
END
any one help me out .....
As Marc already answered, use SEQUENCE. It's available in all supported versions of SQL Server, ie 2012 and later. The only reason to avoid it is targeting an unsupported version like 2008.
In this case, you can set the counter variable in the same statement you update the counter value. This way, you don't need any transactions or locks, eg:
declare #counterValue bigint
UPDATE Maintenance_Counter
SET Col_Counter = Col_Counter + 1 , #counterValue=Col_Counter+1
WHERE COL_NAME = #Col_Name
select #counterValue
Yo can use sequences to generate incremental values avoiding any blocking.
I have adapted my own Counter Generator to be a direct replacement for yours. It creates dynamically the SQL statements to manage sequences, if a Sequence doesn't exist for the value we are looking for, it creates it.
ALTER PROCEDURE [dbo].[Web_GetMaxColumnID]
#Col_Name nvarchar(50)
AS
declare #Value bigint;
declare #SQL nvarchar(64);
BEGIN
if not exists(select * from sys.objects where object_id = object_id(N'dbo.MY_SEQUENCES_' + #Col_Name) and type = 'SO')
begin
set #SQL = N'create sequence dbo.MY_SEQUENCES_' + #Col_Name + ' as bigint start with 1';
exec (#SQL);
end
set #SQL = N'set #Value = next value for dbo.MY_SEQUENCES_' + #Col_Name;
exec sp_executesql #SQL, N'#Value bigint out', #Value = #Value out;
select #Value ;
END
The only inconvenience is that your values can get gaps within (because you could have retrieved a value but finally not used it). This is not a problem on my tables, but you have to consider it.
Hi I need to create a view or stored procedure that combines data and returns a result set from 3 different databases on the same server using a column that holds a schema (db) name.
For Example on the first DB I have this table:
CREATE TABLE [dbo].[CloudUsers](
ID int IDENTITY(1,1) NOT NULL,
Username nvarchar(50) NULL,
MainDB nvarchar(100) NULL
) ON [PRIMARY]
Each CloudUser has a separate DB so next now I need to fetch the data from the User database using the MainDB name. The data I need is always 1 row cause I'm using aggregate functions / query.
So in the User MainDB let's say I have this table.
CREATE TABLE [dbo].[CLIENT](
ID int NOT NULL,
Name nvarchar(50) NULL,
ProjectDBName [nvarchar](100) NULL
CreationDate datetime NULL
) ON [PRIMARY]
And I query like:
select min(CreationDate) from MainDB.Client
The same Idea for the Client I need to fetch even more data from a 3rd database that points to the Client ProjectDBName. Again it's aggregate data:
select Count(id) as TotalTransactions from ProjectDBName.Journal
My final result should have records from all databases. It's readonly data that I need for statistics.
Final result set example:
CloudUsers.Username, MainDB->CreationDate, ProjectDBName->TotalTransaction
How can I achieve that ?
This is not easy - and without a schema and sample data, I can't give you a precise answer.
You need to iterate through each client, and use dynamic SQL to execute a the query against the mainDB and projectDB join. You can either do that in one gigantic "union" query, or by creating a temporary table and inserting the data into that temporary table, and then selecting from the temp table at the end of the query.
For you who are curious of how to solve this issue I have found my own solution using some cursors + dynamic and a simple table variable, enjoy.
ALTER PROCEDURE CloudAnalysis as
DECLARE #objcursor cursor
DECLARE #innercursor cursor
DECLARE #userid int
,#maindb nvarchar(100)
,#clientid int
,#name nvarchar(50)
,#projdb nvarchar(100)
,#stat nvarchar(50)
,#sql nvarchar(max)
,#vsql nvarchar(max)
,#rowcount int
DECLARE #result table(userid int,clientid int,maindb nvarchar(100),name nvarchar(50),projdb nvarchar(100),stat nvarchar(50))
SET #objcursor = CURSOR FORWARD_ONLY STATIC FOR SELECT c.id,c.maindb,u.client_id FROM dbo.ClientUsers c join dbo.UserClients u on c.id = u.user_id open #objcursor
FETCH NEXT FROM #objcursor INTO #userid,#maindb,#clientid
WHILE (##FETCH_STATUS=0)
BEGIN
IF (EXISTS (SELECT name
FROM master.dbo.sysdatabases
WHERE ('[' + name + ']' = #maindb
OR name = #maindb)))
BEGIN
set #sql = N'SELECT #name = c.name,#projdb=c.ProjectDBName FROM ' + #maindb + '.dbo.CLIENT c WHERE c.id = ' + cast(#clientid as nvarchar)
EXECUTE sp_executesql #sql, N'#name NVARCHAR(50) OUTPUT,#projdb NVARCHAR(100) OUTPUT',
#name = #name OUTPUT
,#projdb = #projdb OUTPUT
SELECT #rowcount = ##ROWCOUNT
IF #rowcount > 0
BEGIN
--print ' client: ' + cast(#clientid as nvarchar)+
--':' + #name + ' projdb: ' + #projdb
IF (EXISTS (SELECT name
FROM master.dbo.sysdatabases
WHERE ('[' + name + ']' = #projdb
OR name = #projdb)))
BEGIN
SET #sql = N'SELECT #stat = j.stat FROM ' + #projdb + '.dbo.JournalTransaction j'
EXECUTE sp_executesql #sql
,N'#stat NVARCHAR(50) OUTPUT'
,#stat = #stat OUTPUT
END
INSERT INTO #result (userid,clientid,maindb,name,projdb,stat)
VALUES (#userid,#clientid,#maindb,#name,#projdb,#stat)
END
END
FETCH NEXT FROM #objcursor INTO #userid,#maindb,#clientid
END
CLOSE #objcursor
DEALLOCATE #objcursor
SELECT * FROM #result
I have some tables for storing different file information, like thumbs, images, datasheets, ...
I'm writing a stored procedure to retrieve filename of a specific ID. something like:
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
if #table='images'
select [filename] from images
where id = #id
if #table='icons'
select [filename] from icons
where id = #id
....
How can I rewrite this procedure using case when statement or should I just use table name as variable?
You can't use case .. when to switch between a table in the FROM clause (like you can in a conditional ORDER BY). i.e. so the following:
select * from
case when 1=1
then t1
else t2
end;
won't work.
So you'll need to use dynamic SQL. It's best to parameterize the query as far as possible, for example the #id value can be parameterized:
-- Validate #table is E ['images', 'icons', ... other valid names here]
DECLARE #sql NVARCHAR(MAX)
SET #sql = 'select [filename] from **TABLE** where id = #id';
SET #sql = REPLACE(#sql, '**TABLE**', #table);
sp_executesql #sql, N'#id INT', #id = #id;
As with all dynamic Sql, note that unparameterized values which are substituted into the query (like #table), make the query vulnerable to Sql Injection attacks. As a result, I would suggest that you ensure that #table comes from a trusted source, or better still, the value of #table is compared to a white list of permissable tables prior to execution of the query.
Just build SQL string in another variable and EXECUTE it
DECLARE #sql AS NCHAR(500)
SET #sql=
'SELECT [filename] '+
' FROM '+#table+
' WHERE id = #id'
EXECUTE(#sql)
CREATE PROCEDURE get_file_name(
#id int,
#table nvarchar(50)
)as
DECLARE #SQL nvarchar(max);
SET #SQL = 'select [filename] from ' + #table + ' where id = ' + #id
EXECUTE (#SQL)