I would like to understand how postgreSQL is doing multi query execution for example i have a DB that there are lots of insert queries running for example 20-40 per minute and lots of select queries like 200-300 per minute (simple query by primary key selection).
This type of queries are run on the same table and i'm curious on how postgreSQL is handling these. Is it like when insert query is run table is lock and we have to wait for select queries or it is row locking so that while insert query is in progress select queries can continue and ignore locked rows?
In mysql database there is MyISAM engine that does table lock and innoDB that does row locking i guess...
Postgres implements multiv version concurrency control (MVCC) which means that readers never block writers and writers never block readers.
For normal DML statements Postgres never takes a table lock either so the SELECT queries are never blocked by any of the INSERT statements you are running concurrently.
The Postgres Wiki contains links to more detailed descriptions on how exactly MVCC is implemented and works in Postgres.
Essentially every modern DBMS uses some kind of MVCC these days. Oracle, Firebird and DB2 have "always" been using it. SQL Server introduced it with SQL Server 2005 (although it's still not the default behaviour) and in MySQL the InnoDB engine uses it.
Related
I need some light here. I am working with SQL Server 2008.
I have a database for my application. Each table has a trigger to stores all changes on another database (on the same server) on one unique table 'tbSysMasterLog'. Yes the log of the application its stored on another database.
Problem is, before any Insert/update/delete command on the application database, a transaction its started, and therefore, the table of the log database is locked until the transaction is committed or rolled back. So anyone else who tries to write in any another table of the application will be locked.
So...is there any way possible to disable transactions on a particular database or on a particular table?
You cannot turn off the log. Everything gets logged. You can set to "Simple" which will limit the amount of data saved after the records are committed.
" the table of the log database is locked": why that?
Normally you log changes by inserting records. The insert of records should not lock the complete table, normally there should not be any contention in insertion.
If you do more than inserts, perhaps you should consider changing that. Perhaps you should look at the indices defined on log, perhaps you can avoid some of them.
It sounds from the question that you have a create transaction at the start of your triggers, and that you are logging to the other database prior to the commit transaction.
Normally you do not need to have explicit transactions in SQL server.
If you do need explicit transactions. You could put the data to be logged into variables. Commit the transaction and then insert it into your log table.
Normally inserts are fast and can happen in parallel with out locking. There are certain things like identity columns that require order, but this is very lightweight structure they can be avoided by generating guids so inserts are non blocking, but for something like your log table a primary key identity column would give you a clear sequence that is probably helpful in working out the order.
Obviously if you log after the transaction, this may not be in the same order as the transactions occurred due to the different times that transactions take to commit.
We normally log into individual tables with a similar name to the master table e.g. FooHistory or AuditFoo
There are other options a very lightweight method is to use a trace, this is what is used for performance tuning and will give you a copy of every statement run on the database (including triggers), and you can log this to a different database server. It is a good idea to log to different server if you are doing a trace on a heavily used servers since the volume of data is massive if you are doing a trace across say 1,000 simultaneous sessions.
https://learn.microsoft.com/en-us/sql/tools/sql-server-profiler/save-trace-results-to-a-table-sql-server-profiler?view=sql-server-ver15
You can also trace to a file and then load it into a table, ( better performance), and script up starting stopping and loading traces.
The load on the server that is getting the trace log is minimal and I have never had a locking problem on the server receiving the trace, so I am pretty sure that you are doing something to cause the locks.
I have two applications that do something to the same SQL Server table. One application uses C# SqlBulkCopy to import about two hundred thousand records into the SQL Server table, and the other application queries data from the same SQL Server table.
I find this message - please check the screenshot. The table has one hundred million rows. How can I fix it?
If any transaction is modifying a table and affecting more than 5000 rows, then SQL Server will escalate the locking from row-level locking to an exclusive table lock.
So if your application #1 is bulk-loading 200'000 rows into the table, then that table will be exclusively locked for the duration of the loading process.
Therefore, your application #2 - or any other client - won't be able to query that table, until the loading process is done.
This is normal, documented, expected behavior on the part of SQL Server.
Either make sure you load your data in batches of less than 5000 rows at a time during business hours, or then do the bulk-loading after hours, when no one is being negatively impacted by an exclusive table lock.
It seems like despite the fact we're not using transactions at all we get random deadlock error from SQL Azure.
Are there no transnational situation when SQL Azure can get into a deadlock?
It seems like when we are running a batch of UPDATE queries it acts like the batch is a one big transaction.
All the updates are by id and update a single a line.
There is no such things as "not using transactions". There's always a transaction, wether you start one explicitly or not. Read Tracking down Deadlocks in SQL Database for how to obtain the deadlock graph in SQL Azure. Connect to master and run:
SELECT * FROM sys.event_log
WHERE database_name like '<your db name>'
AND event_type = 'deadlock';
Then analyze the deadlock graph do understand the cause. Most likely you're doing scan because of missing indexes.
When you have concurrent transactions running (either implicit or explicit) you encounter deadlocks. Probably when you you said no transactions that means your transactions are implicit.
I have a PostgreSQL 9.2 database using pgBouncer connection pool in a debian server.
In that database, regular users performs queries over a few tables and I have a cron process fetching data and inserting in a table (using a pg/plsql function which makes some validations before insert).
The problem I have is that when I have a huge load on the cron processes (many inserts), the table get locked and the queries to this table does not respond (or takes a lot of time to respond).
Is there any way to set priorities by stored procedure, database user(cron and queries use different database users) or by type (select has higher priority than insert).
If there is no way to user priority definition in postgreSQL, is there any workaround?
Inserts can wait, but the user queries should not...
The cron process creates and drops a pgbouncer connection per insert. If I use the same connection, the problem is bigger (the queries takes even longer)
Thanks in advance,
Claudio
We are using partitioned views (SQL Server 2008 Standard, partitioned tables are not an option), and they work fine if we consider the partition elimination goal: if we run a query in a partitioned view specifying a clause on the column we choose as the discriminator, we can see from the actual execution plan that only the table related to the specified discriminator value is hit. But we incur in locking problems if there are concurrent INSERT or UPDATE statements, even if those ones are NOT hitting the table selected by the discriminator.
Analyzing the locks I can see that, even if the execution plan shows that only the right one table is read, IS locks are still put on ALL the tables in the partitioned view, and of course if someone else has already put an X locks on one of those the whole query running on the partitioned view gets locked on that one, even if the table with an X upon is not read at all.
Is this a limitation of partitioned view in general, or there is a way to avoid it while sticking with partitioned views? We created the partitioned view and the related things following the SQL Server Books Online recommendations.
Thanks
Wasp
This is by design. Do not lock X entire tables.