I am using PgBouncer in "transaction" mode for connection pooling. Our code already have PreparedStatements used at multiple places.
Even though FAQ on PgBouncer https://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling suggests to use prepareThreshold=0 to disable preparedStatements , but in reality this does not work.
Any solution to this ?
That's old question but I am going to answer it for reference.
Try also adding preparedStatementCacheQueries=0 to your JDBC connection string.
e.g. jdbc:postgresql://{0}:{1}/{2}?user={3}&password={4}&prepareThreshold=0&preparedStatementCacheQueries=0
Related
We are developing a SQL Database, and client connects to server with rpc.
Think about a case: there're Transaction_A(TxnA) & Transaction_B(TxnB). TxnA may be like UPDATE tbl SET a=a+1 WHERE id=1 .
When client gets CONN_TIMEOUT exception, the reason might be server crashed or network issues. Therefore, TxnA may or maynot complete write. And meanwhile TxnB may write to the same table and update table meta (like advancing the sequence number or revision).
So it seems no way to check TxnA completed or not by the client. And the query may not be idempotent, such as the 'UPDATE a=a+1' example above. So it cannot be retried recklessly.
My question: Is there any solution to solve this issue? I'm not sure how other SQL system or storage system work with this issue. Try google some keywords, but fail to get the answers.
Thank you for any inspiration.
[Question posted by a user on YugabyteDB Community Slack]
It seems that connecting to a cluster is “slow” the first time through NpgsqlConnection.
Is this normal PostgreSQL behaviour or is this related to YugabyteDB?
We are using the latest 6.0.0-preview7 nuget to use the multi host connecting string for load balancing.
According to Allow extensibility in type loading and database capabilities · Issue #1486 · npgsql/npgsql, extra calls are made to system tables to map types when the first connection is created. In order to avoid the initial call, you can set connStringBuilder.ServerCompatibilityMode = ServerCompatibilityMode.NoTypeLoading; on the connection string. When setting this property the first call is not made and the connection will be faster.
We have an old database product; the master deployment has always done ALTER Database [databasename] SET CONCAT_NULL_YIELDS_NULL OFF. Since this setting is going away, we want to come up with a testing plan for getting rid of it and tracking down what code depends on it.
With this switch off, the documentation says that SELECT 'abc' + NULL FROM sometable results in abc rather than NULL.
I can easily observe this behavior at the connection level; however the application never sets it at the connection level. That's not the question here.
How can I observe by any SQL statement the effect of turning off CONCAT_NULL_YIELDS_NULL only at the database level using .NET System.Data.SqlClient as the database access client?
We audited the entire codebase and removed all instances of SET commands except for transaction isolation level as these hose things pretty hard. While I tried EXEC to see if it would bypass the driver settings and found it didn't, I'm reasonably certain there's some sufficiently exotic SQL construction that could succeed.
I think I have finally understood what you actually want: to change the connection option without changing it. No, there are no ways to achieve this, otherwise I would have heard about them, and Rutzky would have mentioned them in his answer I referenced in my original response.
There is a strict hierarchy in a way the options can be set:
Instance-wide settings for all databases, using sp_configure 'user options'. Lowest level defaults.
Database-specific options which can be set via ALTER DATABASE. Take precedence over instance defaults.
Explicit SET statements executed on the connection. Override everything.
The advice? Change the application' connection string, if you can, so that it would use another driver that doesn't set / touch this option. Assuming, of course, it wouldn't crash anything, but this is a thing which can be tested relatively quickly.
Also, you may try to change the instance defaults, and see if the driver somehow favours them over #2. It is possible that it can be too smart for its own good.
P.S. For the sake of completeness, the only cases of interdependent options that I know of are listed below:
Shortcut settings, for example ANSI_DEFAULTS. Setting it actually affects several options related to ANSI compatibility;
SET LANGUAGE. When you change the connection language, it also overrides the DATEFIRST and DATEFORMAT options.
P.P.S. I decided to leave my previous answer intact, as it still contains some info which might be useful for people who would understand your question the way I initially did.
Even if your application doesn't specify this option explicitly, different ODBC and OLEDB drivers have different defaults for connection options. Depending on the driver, this particular option can either be set implicitly, or inherited from database / server settings.
In order to see the actual state of this option for connections established by your app, you can setup a Profiler trace which will show you effective connection settings.
In regards to overriding the driver's behaviour setting this option, there is a rather thorough answer for a similar question on DBA StackExchange. There, you can find several possible approaches, and see which one suits you best.
On the SQL side, one possible solution is to replace old style + string concatenations with the newer concat() function which is available since SQL Server 2012. This function skips NULL arguments during concatenation, so the result always looks as if concat_null_yields_null is set to off. Depending on the amount of SQL code modules in your system, it might be quite an undertaking, especially as it's difficult to distinguish between numerical additions and string concatenations with a naked eye.
I would recommend to shop around for some code analysis / refactoring tools that might lend a hand in this. Not sure if SSDT has this capability, but I would start with it first, since it's free (and it might make sense to try out the Visual Studio 2019, which was released 2 days ago - there might be something there). Other than that, well... RedGate, Idera, ApexSQL, you name it. Someone might have done this already.
This can be achieved with this query
IF (SELECT ''+NULL) IS NULL
BEGIN
SET CONCAT_NULL_YIELDS_NULL OFF
END
ELSE
BEGIN
SET CONCAT_NULL_YIELDS_NULL ON
END
Some colleagues were using an Excel file to keep track of some issues, and they have decided to switch to a better system, asking me to setup a Jira project for them and to import all the tickets. A way or the other I have done everything, but the resolution date is now wrong, because it's the one of when I ran the script to import them into Jira. They would like to have the original one, so that they can know when an issue was really fixed. Unfortunately there's no way to change it from Jira's interface, so I have to access the DB directly. The command, for the record, is like:
update jiraissue
set RESOLUTIONDATE = "2015-02-16 14:48:40"
where pkey = "OV001-1";
Now, low-level writes to a database in general are dangerous, and I am wondering whether there can be any risks. Our test server is not available right now, so I'd have to work directly on the production one. One thing I had seen on our test server is that this seemed to work, except that JQL queries such as
resolved < 2015-03-20
are wrong because they still use the old Resolution date. Clearly, I have to reindex; but I'm wondering whether it is safe. Does Jira perform some consistency checks? Like, verifying that a ticket is solved after it is created. In my case, since I have modified the resolution date but not the creation, it is a clear inconsistency. Will Jira complain about this? Is there the risk to corrupt the DB? And if I also modify the creation date, do I have to watch out for other things?
We are using Jira 5.2.11.
I have access to the test server again, and I have tried it. I have modified all the RESOLUTIONDATE fields I had to fix, and when I reloaded the page the new date was there. Jira didn't complain about anything. I reindexed the server, so that queries yield correct results, and I saw no issues. Then I even ran the integrity checks (Administration -> System -> Integrity Checker), and no error was found.
Finally I did the same on the production server, and again everything is running fine.
I can therefore conclude that the operation is not dangerous at all, and it can be done safely.
According to the DBCP Document, BasicDataSource does not support setLoginTimeout(). My question is then how do I set a LoginTimeout for the creation of Connection objects? I know I can set maxWait on the pool, but my understanding is that that'll only be used for when the pool is exhausted and you're waiting for an existing connection to free up. It will not save me from the situation where a new connection needs to be created, but the connection/login into the DB hangs.
Any help is appreciated. Thanks.
Well there is always an option to add correct parameter to the URL. Depending on which DB you are using you can add one of the parameters in JDBC url.
Here is the link that confirms that BasicDataSource does not support loginTimeout
And at the bottom of this blog There is a table which lists URL parameters for connection timeouts.
#Sap is right, you might use the JDBC url in order to add connection properties, for example:
basicDataSource.setUrl("jdbc:postgresql://192.168.0.100:5432/postgres?connectTimeout=TIME_IN_SECONDS");
Where TIME_IN_SECONDS can be any intenger value.
Besides, we can use BasicDataSource#setConnectionProperties. As the parameter says:
The connection properties that will be sent to our JDBC driver when establishing new connections.
Format of the string must be [propertyName=property;]*
NOTE - The "user" and "password" properties will be passed explicitly, so they do not need to be included here.
So, it might be something like (and following the previous example):
basicDataSource.setUrl("jdbc:postgresql://192.168.0.100:5432/postgres");
basicDataSource.setConnectionProperties("connectTimeout=TIME_IN_SECONDS");
P.S:
You can add more properties using ; at the end of the property.