I have two PostgreSQL databases. I'll send request to one database to do some DML operations , based on success response I'll send request to second database to do some DML operations. If second operation is failed, how can I roolback first DB DML operations.
Hope you all understood my problem.
Standard rollback does not apply since you need the result from statement 1 and use it for 2nd statement. You need to do this in your app outside sql.
For the 'artificial rollback', you need to cache the original values for the row(s) effected by the operation
The flow will work on the following lines:
select * from table where the_Where; //statement 1
update db1.table set updates where the_Where;
--if success--
update db2.table set updates where another_Where;
--if failure--
update table set (data from statement 1) where the_Where;
Related
The coldfusion documentation, (I'm using CF8) states:
Changes to data that is requested
by the queries are not committed to the datasource until all actions within
the transaction block have executed successfully.
But it also states:
In a transaction block, you can write queries to more than one database, but you must commit or roll back a transaction to one database before writing a query to another
I have multiple transactions in my code base which access 2 databases for both selects and update/inserts. The code assumes that all queries will either succeed or they will all be rolled back. But I don't know if that is true based upon the line in the docs that says: "but you must commit or roll back a transaction to one database before writing a query to another".
What is the behavior if a write to the first database succeeds, then the subsequent write to another database fails? Will the first be rolled back?
What the documentation means is that you must put a <cftransaction action="commit"> after the queries to one database before you can move on to using another datasource. It will throw an error if it detects that you have <cfquery> tags with different datasources inside of a transaction without using the commit. See your database documentation for exact transaction support as the CFML via the database driver is only sending in transaction commands on your behalf, it is not responsible for their execution or behavior. Enable JDBC logging in your database to see this in action.
Won't work:
<cftransaction action="begin">
<cfquery datasource="foo">
select * from foo_test
</cfquery>
<cfquery datasource="bar">
select * from bar_test
</cfquery>
</cftransaction>
Will work
<cftransaction action="begin">
<cfquery datasource="foo">
select * from foo_test
</cfquery>
<cftransaction action="commit"><!-- Commit before switching DSNs --->
<cfquery datasource="bar">
select * from bar_test
</cfquery>
</cftransaction>
If you are using three part names for multiple database access through a single datasource, the transaction control will work.
<cftransaction action="begin">
<cfquery datasource="foo">
INSERT INTO foo_test ( id )
VALUES ( 70 )
</cfquery>
<!-- insert into the bar database via the foo datasource --->
<cfquery datasource="foo">
INSERT INTO bar.dbo.bar_test (id )
VALUES ( 'frank' ) <!-- Fails because not an int and the prior insert into foo is rolled back -->
</cfquery>
</cftransaction>
The default behaviour for CFTransaction is that all writes will be rolled back if there is an exception anywhere within the transaction block. So if one query fails, all queries are rolled back.
This is only if the database supports commits and rollbacks based on Transaction Control Language, a subset of SQL.
However you can granulary control how CF transaction works, beyond the default behaviour, including features such as savepoints and nested transactions.
I am using DB2 9.7 FP5 for LUW. I have a table with 2.5 million rows and I want to delete about 1 million rows and this delete operation is distributed across table. I am deleting data with 5 delete statements.
delete from tablename where tableky between range1 and range2
delete from tablename where tableky between range3 and range4
delete from tablename where tableky between range5 and range5
delete from tablename where tableky between range7 and range8
delete from tablename where tableky between range9 and range10
While doing this, first 3 deletes works properly but the 4th fails and DB2 hangs, doing nothing. Below is the process I followed, please help me on this:
1. Set following profile registry parameters: DB2_SKIPINSERTED,DB2_USE_ALTERNATE_PAGE_CLEANING,DB2_EVALUNCOMMITTED,DB2_SKIPDELETED,DB2_PARALLEL_IO
2.Alter bufferpools for automatic storage.
3. Turn off logging for tables (alter table tabname activate not logged initially) and delete records
4. Execute the script with +c to make sure logging is off
What are the best practices to delete such large amount of data? Why its failing when it is deleting data from same table and of same nature?
This is allways tricky task. The size of transaction (e.g. for safe rollback) is limited by the size of transaction log. The transaction log is filled not only by yours sql commands but also by the commands of other users using db in the same moment.
I would suggest using one of/or combination of following methods
1. Commits
Do commmits often - in your case I would put one commit after each delete command
2. Increase the size of transaction log
As I recall default db2 transaction log is not very big. The size of transaction log should be calculated/tuned for each db individually. Reference here and with more details here
3. Stored procedure
Write and call stored procedure which does deletes in blocks, e.g.:
-- USAGE - create: db2 -td# -vf del_blocks.sql
-- USAGE - call: db2 "call DEL_BLOCKS(4, ?)"
drop PROCEDURE DEL_BLOCKS#
CREATE PROCEDURE DEL_BLOCKS(IN PK_FROM INTEGER, IN PK_TO INTEGER)
LANGUAGE SQL
BEGIN
declare v_CNT_BLOCK bigint;
set v_CNT_BLOCK = 0;
FOR r_cur as c_cur cursor with hold for
select tableky from tablename
where tableky between pk_from and pk_to
for read only
DO
delete from tablename where tableky=r_cur.tableky;
set v_CNT_BLOCK=v_CNT_BLOCK+1;
if v_CNT_BLOCK >= 5000 then
set v_CNT_BLOCK = 0;
commit;
end if;
END FOR;
commit;
END#
4. Export + import with replace option
In some cases when I needed to purge very big tables or leave just small amount of records (and had no FK constraints), then I used export + import(replace). The replace import option is very destructive - it purges the whole table before import of new records starts (reference of db2 import command), so be sure what you're doing and make backup before. For such sensitive operations I create 3 scripts and run each separately: backup, export, import. Here is the script for export:
echo '===================== export started ';
values current time;
export to tablename.del of del
select * from tablename where (tableky between 1 and 1000
or tableky between 2000 and 3000
or tableky between 5000 and 7000
) ;
echo '===================== export finished ';
values current time;
Here is the import script:
echo '===================== import started ';
values current time;
import from tablename.del of del allow write access commitcount 2000
-- !!!! this is IMPORTANT and VERY VERY destructive option
replace
into tablename ;
echo '===================== import finished ';
5. Truncate command
Db2 in version 9.7 introduced TRUNCATE statement which:
deletes all of the rows from a table.
Basically:
TRUNCATE TABLE <tablename> IMMEDIATE
I had no experience with TRUNCATE in db2 but in some other engines, the command is very fast and does not use transaction log (at least not in usual manner). Please check all details here or in official documentation. As solution 4, this method too is very destructive - it purges the whole table so be very careful before issuing the command. Ensure previous state with table/db backup doing first.
Note about when to do this
When there are no other users on db, or ensure this by locking the table.
Note about rollback
In transaction db (like db2) rollback can restore db state to the state when transaction started. In methods 1,3 and 4 this can't be achieved, so if you need feature "restoring to the original state", the only option which ensures this is the method nr. 2 - increase transaction log.
delete from ordpos where orderid in ((select orderid from ordpos where orderid not in (select id from ordhdr) fetch first 40000 rows only));
Hoping this will resolve your query :)
It's unlikely that DB2 is "hanging" – more likely it's in the process of doing a Rollback after the DELETE operation filled the transaction log.
Make sure that you are committing after each individual DELETE statement. If you are executing the script using the +c option for the DB2 CLP, then make sure you include an explicit COMMIT statement between each DELETE.
Best practice to delete the data which has millions of rows is to use commit in between the deletes. In your case you can use commit after every delete statement.
What commit does is it will clear the transction logs and make space available for other delte operations to perform.
Alternatively instad of 5 delete statements use loop and pass the delete statement to delete, After one iteration of the loop execute one commit then database will never hang and simultaneously your data will get deleted.
use some thing like this.
while(count<no of records)
delete from (select * from table fetch fist 50000 records only)
commit;
count= total records- no of records.
If SELECT WHERE FETCH FIRST 10 ROWS ONLY can pull-in a few chunk of records,in chunks of 10 for example, then you can feed this as input into another script that will then delete these records. Rinse and repeat...
For the benefit of everyone, here is the link to my developerWorks article on the same problem. I tried different things and the one I shared on this article worked perfectly for me.
Primary Question:
I want to truncate and refresh a table in SQL Server, but want to wait until any queries currently accessing the table to finish. Is this a simple setting in SQL Server, or do I need to create some logic to accomplish it?
Detailed Description:
I have a VB application that sits on about 300 terminals. The application calls a SqlServer(2008 R2) stored procedure ([spGetScreenData]) every 2 minutes to get the latest sales data.
[spGetScreenData] creates a series of temp tables and returns a select query of about 200 rows and 100 columns. It takes about 8 seconds to execute.
My goal is to create a new stored procedure ([spRefreshScreenData]) that executes every two minutes which will refresh the data in a table ([SCREEN_DATA]). I will then change [spGetScreenData] to simply query [SCREEN_DATA].
The job that refreshes [SCREEN_DATA] first sets a flag in a status table to 'RUNNING' while it executes. Once complete, it sets that status to 'COMPLETED'.
[spGetScreenData] checks the status of the flag before querying and waits (for a period of time) until it's ready. Something like...
DECLARE #Condition AS BIT=0
, #Count AS INT=0
, #CycleCount AS INT=10 --10 cycles (20 Seconds)
WHILE #Condition = 0 AND #Count < #CycleCount
BEGIN
SET #Count = #Count + 1
IF EXISTS( SELECT Status
FROM tbl_Process_Status
WHERE Process = 'POS_Table_Refresh'
AND Status='Running')
WAITFOR DELAY '000:00:02' --Wait 2 seconds
ELSE
SET #Condition=1
END
SELECT *
FROM SCREEN_DATA
WHERE (Store=#Store OR #Store IS NULL)
My concern has to do with [spRefreshScreenData]. When [spRefeshScreenData] begins its truncation, there could be dozens of requests for the data currently running.
Will SqlServer simply wait until the request are done before truncating? Is there a setting I have to set to not mess these queries up?
Or do I have to build some mechanism to wait until all requests are completed before starting the truncation?
The job that refreshes [SCREEN_DATA] first sets a flag in a status table to 'RUNNING' while it executes. Once complete, it sets that status to 'COMPLETED'.
[spGetScreenData] checks the status of the flag before querying and waits (for a period of time) until it's ready
Don't. Use app locks. The readers (spGetScreenData) are the app lock in shared mode, the writers (refresh job) requests it X mode. See sp_getapplock.
But even this is no necessary. You can build the new data online, while the queries continue, w/o affecting them using a staging table, ye a different table than the one queried by the apps. When the rebuild is complete simply swap the original tabel with the staging one, using either fast SWITCH operations (see Transferring Data Efficiently by Using Partition Switching) or using the good 'ole sp_rename trick.
This is a general question about how to lock range of values (and nothing else!) when they are not exists in table yet. The trigger for the question was that I want to do "insert if not exists", I don't want to use MERGE because I need to support SQL Server 2005.
In the first connection I:
begin transaction
select data from a table using (SERIALIZABLE, ROWLOCK) + where clause to respecify range
wait...
In the second connection, I insert data to the table with values that do not match the where clause in the first connection
I would expect that the second connection won't be affected by the first one, but it finishes only after I commit (or rollback) the first connection's transaction.
What am I missing?
Here is my test code:
First create this table:
CREATE TABLE test
(
VALUE nvarchar(100)
)
Second, open new query window sql server managements studio and execute the following:
BEGIN TRANSACTION;
SELECT *
FROM test WITH (SERIALIZABLE,ROWLOCK)
WHERE value = N'a';
Third, open another new query window and execute the following:
INSERT INTO test VALUES (N'b');
Notice that the second query doesn't ends until the transaction in the first window ends
You are missing an index on VALUE.
Without that SQL Server has nothing to take a key range lock on and will lock the whole table in order to lock the range.
Even when the index is added however you will still encounter blocking with the scenario in your question. The RangeS-S lock doesn't lock the specific range given in your query. Instead it locks the range between the keys either side of the selected range.
When there are no such keys either side the range lock extends to infinity. You would need to add a value between a and b (for example aa) to prevent this happening in your test and the insert of b being blocked.
See Bonus Appendix: Range Locks in this article for more about this.
In my database I have certain data that is important to the functioning of the app (constants, ...). And I have test data that is being generated by testing the site. As the test data is expendable it delete it regularly. Unfortunately the two types of data occur in the same table so I cannot do a delete from T but I have to do a delete from T where IsDev = 0.
How can I make sure that I do not accidentally delete the non-dev data by forgetting to put the filter in? If that happens I have to restore from a production backup which is wasting my time. I would require some sort of foreign key like behavior that fails a delete when a certain condition is met. This would also be useful to ensure that my code does not do anything harmful due to a bug.
Well, you could use a trigger that throws an exception if any of the records in the deleted meta-table have IsDev = 1.
CREATE TRIGGER TR_DEL_protect_constants ON MyTable FOR DELETE AS
BEGIN
IF EXISTS(SELECT 1 FROM deleted WHERE IsDev <> 0)
BEGIN
ROLLBACK
RAISERROR('Can''t delete constants', 1, 16)
RETURN
END
END
I'm guessing a bit on the syntax, but you get the idea.
I would use a trigger.
keep a backup of the rows you want to retain in a separate admin table
Seems like you need a trigger on delete operation that would look at the row and rollback transaction if it sees that it's a row that should never be deleted.
Also, you might want to read this article: Prevent accidental update or delete commands of all rows in a SQL Server table
Depending on how transparent you want to make this, you could use an INSTEAD OF trigger that will always remember the WHERE for you.
CREATE TRIGGER TR_IODEL_DevOnly ON YourTable
INSTEAD OF DELETE
AS
BEGIN
DELETE FROM t
FROM Deleted d
INNER JOIN YourTable t
ON d.PrimaryKey = t.PrimaryKey
WHERE t.IsDev = 0
END
I suggest that instead of writing the delete statement from scratch every time, just create a stored procedure to do the deletions and execute that.
create procedure ResetT as delete from T where IsDev = 0
You could create an extra column IS_TEST in your tables, rename the TABLE_NAME to TABLE_NAME_BAK, and create a view TABLE_NAME on the TABLE_NAME_BAK so that only rows where IS_TEST was set are displayed in it. Setting IS_TEST to zero for the data you wish to keep, and adding a DEFAULT 1 to the IS_TEST column should complete the job. It is similar to the procedure required for creating 'soft deletes'.