I have a table in sql server 2005. Yesterday an update was performed on the table and then updated back to the origional record.
Now i want to figure out what the updated values was. At least i want to learn if it was really updated or not.
Is there someway to determine what i want from the transaction logs?
Thanks.
There is an undocumented command:
DBCC log ( dbname, 0|1|2|3|4 )
where
0: minimum information (Default)
1: Returns info available using 0 +
flags, tags and the log record length.
2: Returns info available using
1 + object, index, page ID and slot ID.
3: Maximum information about
each operation.
4: Maximum information about each operation +
hexadecimal dump of the current transaction log row
And read this: Looking for a SQL Transaction Log file viewer
Although is not a precise method (ex. you have data only from the last SQL Server restart), you can try to use sys.dm_exec_query_stats view:
CREATE TABLE dbo.LongTableName (ID INT IDENTITY(1,1) PRIMARY KEY, Column1 VARCHAR(10) NOT NULL);
INSERT LongTableName VALUES ('A');
INSERT LongTableName VALUES ('BB');
INSERT LongTableName VALUES ('CCC');
WAITFOR DELAY '00:00:05';
INSERT LongTableName VALUES ('DDDD');
GO
SELECT ca.[text], s.last_execution_time, s.last_logical_reads, s.last_logical_writes, s.execution_count
FROM sys.dm_exec_query_stats s
CROSS APPLY sys.dm_exec_sql_text(s.sql_handle) ca
WHERE ca.[text] LIKE '%INSERT%LongTableName%'
GO
DROP TABLE LongTableName;
GO
For example, one of the records will be:
text last_execution_time last_logical_reads last_logical_writes execution_count
(#1 varchar(8000))INSERT INTO [LongTableName] values(#1)
2011-10-04 10:51:17.070 2 0 4
Related
I have a stored procedure that performs a bulk insert of a large number of DNS log entries. I wish to summarise this raw data in a new table for analysis. The new table takes a given log entry for FQDN and Record Type and holds one record only with a hitcount.
Source table might include 100 rows of:
FQDN, Type
www.microsoft.com,A
Destination table would have:
FQDN, Type, HitCount
www.microsoft.com, A, 100
The SP establishes a unique ID made up of [FQDN] +'|'+ [Type], which is then used as the primary key in the destination table.
My plan was to have the SP fire a trigger that did an UPDATE...IF ##ROWCOUNT=0...INSERT. However, that of course failed because the trigger receives all the [inserted] rows as a single set so always throws a key violation error.
I'm having trouble getting my head around a solution and need some fresh eyes and better skills to take a look. The bulk insert SP works just fine and the raw data is exactly as desired. However trying to come up with a suitable method to create the summary data is beyond my present skills/mindset.
I have several 10s of Tb of data to process, so I don't see the summary as a something we could do dynamically with a SELECT COUNT - which is why I started down the trigger route.
The relevant code in the SP is driven by a cursor consisting of a list of compressed log files needing to be decompressed and bulk-inserted, and is as follows:
-- Bulk insert to a view because bulk insert cannot populate the UID field
SET #strDynamicSQL = 'BULK INSERT [DNS_Raw_Logs].[dbo].[vwtblRawQueryLogData] FROM ''' + #strTarFolder + '\' + #strLogFileName + ''' WITH (FIRSTROW = 1, FIELDTERMINATOR = '' '', ROWTERMINATOR = ''0x0a'', ERRORFILE = ''' + #strTarFolder + '\' + #strErrorFile + ''', TABLOCK)'
--PRINT #strDynamicSQL
EXEC (#strDynamicSQL)
-- Update [UID] field after the bulk insert
UPDATE [DNS_Raw_Logs].[dbo].[tblRawQueryLogData]
SET [UID] = [FQDN] + '|' + [Type]
FROM [tblRawQueryLogData]
WHERE [UID] IS NULL
I know that the UPDATE...IF ##ROWCOUNT=0...INSERT solution is wrong because it assumes that the input data is a single row. I'd appreciate help on a way to do this.
Thank you
First, at that scale make sure you understand columnstore tables. They are very highly compressed and fast to scan.
Then write a query that reads from the raw table and returns the summarized
create or alter view DnsSummary
as
select FQDN, Type, count(*) HitCount
from tblRawQueryLogData
group by FQDN, Type
Then if querying that view directly is too expensive, write a stored procedure that loads a table after each bulk insert. Or make the view into an indexed view.
Thanks for the answer David, obvious when someone else looks at it!
I ran the view-based solution with 14M records (about 4 hours worth) and it took 40secs to return, so I think i'll modify the SP to drop and re-create summary table each time it runs the bulk insert.
The source table also includes a timestamp for each entry. I would like to grab the earliest and latest times associated with each UID and add that to the summary.
My current summary query (courtesy of David) looks like this:
SELECT [UID], [FQDN], [Type], COUNT([UID]) AS [HitCount]
FROM [DNS_Raw_Logs].[dbo].tblRawQueryLogData
GROUP BY [UID], [FQDN], [Type]
ORDER BY COUNT([UID]) DESC
And returns:
UID, FQDN, Type, HitCount
www.microsoft.com|A, www.microsoft.com, A, 100
If I wanted to grab first earliest and latest times then I think I'm looking at nesting 3 queries to grab the earliest time (SELECT TOP N...ORDER BY... ASC), the latest time (SELECT TOP N...ORDER BY... DESC) and the hitcount. Is there a more efficient way of doing this, before I try and wrap my head around this route?
SQL Server 2017 (In Azure) - when I need to create a new client in our clients database, I have to run three separate queries, and in between each query, do a lookup to be able to populate a part of the next query. I'd like to see if there is a way to combine all this into one query, or, parameterized stored procedure:
All of this takes place in the same database called Clients:
Step 1 - Create the client record in dbo.clients:
INSERT INTO dbo.clients
(ClientGuid, Name, Permissions)
VALUES
(NEWID(), 'Contoso', 1)
Step 2 - Get the Primary Key which was auto-created in Step 1:
SELECT ClientKey from dbo.clients
WHERE Name = 'Contoso'
Now write down the primary key (ClientKey) from that record, we'll say 12345678
Step 3 - Create a new billing code in the dbo.billingcodes table:
INSERT INTO dbo.billingcodes
(BillingCodeGuid, ClientKey, Name, ScoreId)
VALUES
(NEWID(), 12345678, 'Contoso Production Billing Code', 1)
How can I combine all this into one query or parameterized stored procedure where all I have to enter in are the two names from step 1 and 3 (assume the Permissions and ScoreId integers are always going to be 1) and also get an output at the end of the process of the created values for dbo.clients.ClientKey and dbo.billingcodes.BillingCodeGuid?
You could create a procedure that consists of both inserts with a line in between to get the ID of the inserted client. Assign the ID to a variable and pass it in to the second part.
See this post about some different ways about getting the inserted record’s ID Best way to get identity of inserted row?
You could do it by using procedure. You may find this link for creating procedure in SQL Server Link.
In case of Procedure , need to insert your data into first table. Then using IDENT_CURRENT (Ident_Current) you'll get your last inserted id from table, which will further use to insert it into next table.
I am struggling to find a SQL Server replacement for select for update that works.
I have a master table that contains a column which is used for next order number. The application does a select from update on this row, reads the current value (while locked) adds one to this value and then updates the row, then uses the number it received. This process works perfectly on all databases I've tried but for SQL Server which does not seem to have any process for selecting data for exclusive use.
How do I do a locked read and update of something like a next order number from a sequence table is SQL Server?
BTW, I know I can use things like IDENTITY cols and stuff, to do this, but in this case I must read from this existing column. Get the value and inc it, and do it in a safe locked manner to avoid 2 users getting the same value.
UPDATE::
Thank you, that works for this case :)
DECLARE #Output char(30)
UPDATE scheme.sysdirm
SET #Output = key_value = cast(key_value as int)+1
WHERE system_key='OPLASTORD'
SELECT #Output
I have one other place I do something similar. I read and lock a stock record too.
SELECT STOCK
FROM PRODUCT
WHERE ID = ? FOR UPDATE.
I then do some validation and the do
UPDATE PRODUCT SET STOCK = ?
WHERE ID=?
I can't just use your above method here, as the value I update is based on things I do from the stock I read. But I need to ensure no one else can mess with the stock while I do this. Again, easy on other DB's with SELECT FOR UPDATE... is there a SQL Server workaround?? :)
You can simple do an UPDATE that also reads out the new value into a SQL Server variable:
DECLARE #Output INT
UPDATE dbo.YourTable
SET #Output = YourColumn = YourColumn + 1
WHERE ID = ????
SELECT #Output
Since it's an atomic UPDATE statement, it's safe against concurrency issues (since only one connection can get an update locks at any one given time). A potential second session that wants to get the incremented value at the same time will have to wait until the first one completes, thus getting the next value from the table.
As an alternative you can use the OUTPUT clause of the UPDATE statement, although this will insert into a table variable.
Create table YourTable
(
ID int,
YourColumn int
)
GO
INSERT INTO YourTable VALUES (1, 1)
GO
DECLARE #Output TABLE
(
YourColumn int
)
UPDATE YourTable
SET YourColumn = YourColumn + 1
OUTPUT inserted.YourColumn INTO #Output
WHERE ID = 1
SELECT TOP 1 YourColumn
FROM #Output
**** EDIT
If you want to ensure that no-one can change the data after you have read it, you can use a repeatable read. You should be aware that any reads of any tables you do will be locked for Update (pessimistic locking) and may cause Deadlocking. You can also sue the SELECT ... FROM TABLE (UPDLOCK) hint within a transaction.
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
BEGIN TRANSACTION
SELECT STOCK
FROM PRODUCT
WHERE ID = ?
.....
...
UPDATE Product
SET Stock = nnn
WHERE ID = ?
COMMIT TRANSACTION
I need to know if there is any way to have a SEQUENCE or something like that, as we have in Oracle. The idea is to get one number and then use it as a key to save some records in a table. Each time we need to save data in that table, first we get the next number from the sequence and then we use the same to save some records. Is not an IDENTITY column.
For example:
[ID] [SEQUENCE ID] [Code] [Value]
1 1 A 232
2 1 B 454
3 1 C 565
Next time someone needs to add records, the next SEQUENCE ID should be 2, is there any way to do it? the sequence could be a guid for me as well.
As Guillelon points out, the best way to do this in SQL Server is with an identity column.
You can simply define a column as being identity. When a new row is inserted, the identity is automatically incremented.
The difference is that the identity is updated on every row, not just some rows. To be honest, think this is a much better approach. Your example suggests that you are storing both an entity and detail in the same table.
The SequenceId should be the primary identity key in another table. This value can then be used for insertion into this table.
This can be done using multiple ways, Following is what I can think of
Creating a trigger and there by computing the possible value
Adding a computed column along with a function that retrieves the next value of the sequence
Here is an article that presents various solutions
One possible way is to do something like this:
-- Example 1
DECLARE #Var INT
SET #Var = Select Max(ID) + 1 From tbl;
INSERT INTO tbl VALUES (#var,'Record 1')
INSERT INTO tbl VALUES (#var,'Record 2')
INSERT INTO tbl VALUES (#var,'Record 3')
-- Example 2
INSERT INTO #temp VALUES (1,2)
INSERT INTO #temp VALUES (1,2)
INSERT INTO ActualTable (col1, col2, sequence)
SELECT temp.*, (SELECT MAX(ID) + 1 FROM ActualTable)
FROM #temp temp
-- Example 3
DECLARE #var int
INSERT INTO ActualTable (col1, col2, sequence) OUTPUT #var = inserted.sequence VALUES (1, 2, (SELECT MAX(ID) + 1 FROM ActualTable))
The first two examples rely on batch updating. But based on your comment, I have added example 3 which is a single input initially. You can then use the sequence that was inserted to insert the rest of the records. If you have never used an output, please reply in comments and I will expand further.
I would isolate all of the above inside of a transactions.
If you were using SQL Server 2012, you could use the SEQUENCE operator as shown here.
Forgive me if syntax errors, don't have SSMS installed
I have a deadlock problem between 2 processes that insert data in the same table
These 2 processes run exactly the same SQL orders on a table with a primary key (identity) and a unique index.
the sequence of SQL order is the following, for each process in an explicit transaction :
begin trans
select CUSTID from CUSTOMERS where CUSTNUMBER='unique value'
------- the row is never found in this case so... insert the data
insert into CUST(CUSTNUMBER) values('unique value')
------- then we must read the value generated for the pk
select CUSTID from CUSTOMERS where CUSTNUMBER='unique value'
commit
each process work on a distinct data set and have no common values for "CUSTNUMBER"
the deadlock occurs in this case :
spid 1 : select custid... for unique value 1
spid 2 : select custid... for unique value 2
spid 1 : insert unique value 1
spid 2 : insert unique value 2
spid 2 : select custid again for value 2 <--- Deadlock Victim !
spid 1 : select custid again for value 1
The deadlock graph show that the problem occurs on the unique index on CUSTNUMBER
The killed process had a lock OwnerMode:X and was RequestMode:S on the unique index for the same HoBt ID.
The winner process was OnwerMode:X and RequestMode:S for the same HoBt ID
I have no idea to explain that, maybe someone can help me ?
try using OUTPUT to get rid of the final SELECT:
begin trans
select CUSTID from CUSTOMERS where CUSTNUMBER='unique value'
------- the row is never found in this case so... insert the data
insert into CUST(CUSTNUMBER) OUTPUT INSERTED.CUSTID values('unique value')
--^^^^^^^^^^^^^^^ will return a result set of CUSTIDs
commit
OR
DECLARE #x table (CUSTID int)
begin trans
select CUSTID from CUSTOMERS where CUSTNUMBER='unique value'
------- the row is never found in this case so... insert the data
insert into CUST(CUSTNUMBER) OUTPUT INSERTED.CUSTID INTO #x values('unique valu')
--^^^^^^^^^^^^^^^^^^^^^^ will store a set of CUSTIDs
-- into the #x table variable
commit
I have no explanation to the deadlock only another way of doing what you are doing using merge and output. It requires that you use SQL Server 2008 (or higher). Perhaps it will take care of your deadlock issue.
declare #dummy int;
merge CUSTOMERS as T
using (select 'unique value') as S(CUSTNUMBER)
on T.CUSTNUMBER = S.CUSTNUMBER
when not matched then
insert (CUSTNUMBER) values(S.CUSTNUMBER)
when matched then
update set #dummy = 1
output INSERTED.CUSTID;
This will return the newly created CUSTID if there was no match and the already existing CUSTID if there where a match for CUSTNUMBER.
It would be best if you post the actual deadlock graph (the .xml file, not the picture!). W/o that noone can be sure, but is likely that you see a case of the read-write deadlock that occurs due to the order of using vs. applying updates to the secondary indexes. I cannot reommend a solution w/o seeing the deadlock graph and the exact table schema (clustered index and all non-clustered indexes).
On a separate note the SELECT->if not exists->INSERT pattern is always wrong under concurrency, there isn't anything to prevent two threads from trying to insert the same row. A much better patter is to simply insert always and catch the duplicate key violation exception that occurs (is also more performant). As for your second SELECT, use OUTPUT clause as other have already suggested. so basically this whole ordeal can be reduced an insert int a try/catch block. MERGE will also work.
An alternative to using output is replacing the last select with a select scope_identity() if the CUSTID column is an identity column.