I have 2 sessions, each performs the same tasks but on different tables, as follows
begin tran
update...set...
commit tran
checkpoint
Each update is a large batch. The database is under simple recovery mode. To save t-log from growing too large we issue checkpoint so that t-log truncation can happen.
My question is:
If session A committed the transaction and issued a checkpoint while session B is still in the process of updating will the checkpoint issue by session A wait on session B due to session B's active transaction?
In other words would a checkpoint have to wait for all active transaction to finish? How likely is it for the two sessions to form a deadlock?
Also if two checkpoint commands are issued at the same time what will happen?
Note that the session A updates table_A and session B updates table_B. They never update the same table at any given time.
Also I know that using insert into, rename, drop can achieve faster update. But I am limited not to do so. I just want to know about checkpoint concurrency.
Thanks,
A manual Checkpoint simply tells SQL Server to write in-memory changes to disk. It should have no effect on the size of the log.
If Session A commits and checkpoints while Session B is in a transaction on a different table, these are unrelated events - the Session A checkpoint will go ahead, as will the Session B transaction. Since a manual checkpoint simply forces a write of the in-memory data to disk at a time of the programmer's choosing rather than at a time of SQL Server's choosing, the only perceptible consequence should be slightly degraded performance.
Since checkpoints take effect at the database level, concurrent Checkpoints should have the same effect as one Checkpoint.
Checkpoints have absolutely no relation to the data in the database. They do not cause data changes or changes in visibility.
You are likely degrading performance considerably.
Also, it is unlikely that this solves your log problems because SQL Server by default checkpoints regularly anyway. Learn about the log a little more and you'll surely find a better way to address that. Or ask a question about your log problems.
Related
We have a huge DML script, that opens up a transaction and performs a lot of changes and only then it commits.
So recently, I had triggered this scripts (through an app), and as it was taking quite an amount of time, I had killed the session, which triggered a ROLLBACK.
So the problem is that this ROLLBACK took forever and moreover it was hogging a lot of CPU (100% utilization), and as I was monitoring this session (using exec DMVs), I saw a lot of waits that are IO related (IO_COMPLETION, PAGE_IO_LATCH etc).
So my question is:
1. WHy does a rollback take some much amount of time? Is it because it needs to write every revert change to the LOG file? And the IO waits I saw could be related to IO operation against this LOG file?
2. Are there any online resources that I can find, that explains how ROLLBACK mechanism works?
Thank You
Based on another article on the DBA side of SO, ROLLBACKs are slower for at least two reasons: the original SQL is capable of being multithreaded, where the rollback is single-threaded, and two, a commit confirms work that is already complete, where the rollback not only must identify the log action to reverse, but then target the impacted row.
https://dba.stackexchange.com/questions/5233/is-rollback-a-fast-operation
This is what I have found out about why a ROLLBACK operation in SQL Server could be time-consuming and as to why it could produce a lot of IO.
Background Knowledge (Open Tran/Log mechanism):
When a lot of changes to the DB are being written as part of an open transaction, these changes modify the data pages in memory (dirty pages) and log records (into a structure called LOG BLOCKS) generated are initially written to the buffer pool (In Memory). These dirty pages are flushed to the disk either by a recurring Checkpoint operation or a lazy-write process. In accordance with the write-ahead logging mechanism of the SQL Server, before the dirty pages are flushed the LOG RECORDS describing these changes needs to be flushed to the disk as well.
Keeping this background knowledge in mind, now when a transaction is rolled back, this is almost like a recovery operation, where all the changes that are written to the disk, have to be undone. So, the heavy IO we were experiencing might have happened because of this, as there were lots of data changes that had to be undone.
Information Source: https://app.pluralsight.com/library/courses/sqlserver-logging/table-of-contents
This course has a very deep and detailed explanation of how logging recovery works in SQL Server.
The log is a sequence of log records, which maintains information about update activities on the database. Whenever a transaction starts, reads, writes or commits it registers itself in the log with its particular action. So now when recovering from failure a transaction needs to be undone if the transaction hasn't committed and it needs to be redone if it has committed. My doubt is regarding the logic behind doing this. Why do we need to redo committed transactions?
Reference: Slide 19 - http://codex.cs.yale.edu/avi/db-book/db6/slide-dir/PPT-dir/ch16.ppt
The data changes for a committed transaction, stored in the database buffers of the SGA, are not necessarily written immediately to the datafiles by the database writer (DBWn) background process.
because they are in SGA they are visible to other users but those changes still can be lost after commit if not written to datafiles immediately.
Reference: https://docs.oracle.com/cd/B19306_01/server.102/b14220/transact.htm
Reference for image: https://docs.oracle.com/cd/E17781_01/server.112/e18804/memory.htm#ADMQS174
It may be possible for a transaction T1 that all its log records have been output to stable storage but the actual updates on data are still in main memory. If a failure occurs at this point then redoing this transaction will ensure that all updates which were virtually lost due to failure would now get written to the stable storage.
Is there any way that I can determine a DML was run on current session, and I need to Commit or Rollback the transaction? Efficiently?
The Problem
We have a Transaction Manager in our application. It do the Commit/Rollback on each request. But, sometimes we just query some data, and there's no need to Commit/Rollback. We don't want the overhead of frequent commits when there isn't any change.
Inefficient Solution?
this answer suggest to query dbms_transaction. But I'm not sure that if it have a less overhead than doing frequent commits. (Overhead both on our application and database.)
I'm looking into optimizing throughput in a Java application that frequently (100+ transactions/second) updates data in a Postgresql database. Since I don't mind losing a few transactions if the database crashes, I think that using asynchronous commit could be a good fit.
My only concern is that I don't want a delay after commit until other transactions/queries see my commit. I am using the default isolation level "Read committed".
So my question is: Does using asynchronous commit in Postgresql in any way mean that there will be delays before other transactions see my committed data or before they proceed if they have been waiting for my transaction to complete? (As mentioned before, I don't care if there is a delay before my data is persisted to disk.)
It would seem that this is the behavior you're looking for.
http://www.postgresql.org/docs/current/static/wal-async-commit.html
Selecting asynchronous commit mode means that the server returns success as soon as the transaction is logically completed, before the
WAL records it generated have actually made their way to disk. This
can provide a significant boost in throughput for small transactions.
The WAL is used to provide on-disk data integrity, and has nothing to do about table integrity of a running server; it's only important if the server crashes. Since they specifically mention "as soon as the transaction is logically completed", the documention is indicating that this does not affect table behavior.
When Sql Server issues a checkpoint, does it block every other operation until the checkpoint is completed?
If I understand correctly, when a checkpoint occurs, the server should write all dirty pages.
When it's complete, it will write checkpoint to the transaction log, so in case of any failure it will process only transactions from that point of time (or transactions which already started at time of checkpoint).
How does sql server prevent some non dirty page to become dirty while the checkpoint is in progress?
Does it block all writes until the checkpoint is completed?
Checkpoints do not block writes.
A checkpoint has a start and an end LSN. It guarantees that all pages on disk are at least at the start LSN of the checkpoint. It does not matter if any page is at a later LSN (because it has been written to after the checkpoint has started).
The checkpoint only guarantees a minimum LSN for all pages on disk. It does not guarantee an exact LSN.
This makes sense because you can delete all transaction log records which contain information from LSNs which are earlier than the checkpoint start LSN. That is the purpose of a checkpoint: Allow parts of the log to become inactive.
Checkpoints are not needed for data consistency and correctness. They just free log space and shorten recovery times.
when a checkpoint occurs, the server should write all dirty pages
And that's what it does. However the guarantee given by checkpoint is it writes all the pages that were dirty at the instant the checkpoint started. Any page that got dirty while the checkpoint was making progress may or may not be written, but is sure not guaranteed to be written. What this guarantee offers is an optimization that physical recovery can start REDO from the last checkpoint since everything in the log prior to it is already been applied to the data pages (does not have to be redone). Is even on Wikipedia page for ARIES:
The naive way for checkpointing involves locking the whole database to
avoid changes to the DPT and the TT during the creation of the
checkpoint. Fuzzy logging circumvents that by writing two log records.
One Fuzzy Log Starts Here record and, after preparing the checkpoint
data, the actual checkpoint. Between the two records other logrecords
can be created
usr's answer explains how this is achieved (by using a checkpoint start LSN and end LSN).