I have found that using PHP to talk to a MSSQL database, we have incredibly slow results using sqlsrv_fetch_array(). Using sqlsrv_query and "Scrollable" => "static" allows us to iterate through a list of 600 items in 35 seconds, while buffered is completed in under 1 second.
Only one thing springs to mind - that this is a remote database on another server, and using a static cursor requires us to fetch a result directly from the database each time.
Or there's something fundamentally broken with these queries.
For reference, the application is in a Microsoft Azure App Service while the database is on a "SQL Instance" in Azure. However I get similar results when I run the app in debugging mode on my local PC.
I'm not too keen on changing things from "defaults".
What causes this issue and what's the fix? Or is to use a buffered cursor the right solution when dealing with lists of data over a remote connection?
I can confirm using a static cursor takes much longer and is subject to the workload on the SQL server. For example using the static cursor the best time I could get when iterating a result set of almost 14,000 records is 14 seconds. If we don't use the static cursor and go with the default we can iterate the same result set sometimes in less than a second. And this is the BEST the static cursor could do. We have examples of it taking 30s, 50s or even over a minute to pull all the rows from the SQL server.
There seems to be a major problem with the php drivers at least up to and including 17.8.1.1 with static cursors.
result of command yum list installed msodbc*
For the record this php is version:
PHP 7.3.23 (cli) (built: Sep 29 2020 08:33:03) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.23, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.3.23, Copyright (c) 1999-2018, by Zend Technologies
running on CentOS Linux release 7.8.2003 (Core)
Related
We recently upgraded our SQL server to 2016 and I turned on QueryStore to do the analysis that it provides. I'm encountering a problem where, even if the time period of the report is Last hour, it will generate a message that says "Couldn't connect to database" even when running it on the database server itself. Sometimes if I keep refreshing the report it will eventually display some data, but it's intermittent at best. I'm running SSMS 17.5 on a sql server 2016 server.
We are having a somewhat similar issue with another program that connects to the database where it will sometimes not be able to connect, but every time I run my queries in SSMS, run reports in SSRS, or even use activity monitor, I never see any connection drops, so I'm not sure if it is related.
Thank you in advance for any help!
I find it works fine with the statistic set to Avg, StdDev, or Total. Max and Min give the error.
I found this happens when the query store runs out of space and gets into cleanup mode.
In database properties in SSMS try playing around with Query Store settings: for how many days it stores the query stats and does it get into "size cleanup" mode. More info on how to keep it adjusted: https://learn.microsoft.com/en-us/sql/relational-databases/performance/best-practice-with-the-query-store?view=sql-server-ver15#Configure
Two month ago I deployed a new VM in Azure. I used the pre-configured "SQL Server 2016 SP1 Standard on Windows Server 2016" with 7 GB of RAM, and I chose the offered option to make backups automatically. Only other things I changed is add it to AD and put some databases (largest of ~2 GB size of backup file)
Now the server is running a service called SqlIaaSExtension.Service which I understand is for doing these backups as well as automated patching. You can find the services description here: MS service description
The problem is, it keeps on building up memory until after some weeks the SQL Server itself fails to execute larger queries. A restart of the SqlIaaSExtension.Service fixes the problem, but this is not at all a sustainable solution.
Does anybody know a working solution other then disabling the service and loosing the functionality altogether?
My setup (german):
I have meanwhile got some Information from Microsoft:
There seems to be an error in the SqlIaaSExtension.Service which is known to MS and will eventually be fixed.
Workaround is:
A: If you don´t need the functionality - remove this service, as indicated in the service description.
B: If you want to keep the functionality - restart the service periodically. Possibly automate via Task-planner.
Updated info from MS 19/07/2017: Error is identified and should be fixed in the next 7-10 Days. A mitigation is restarting the service if necessary.
Updated info from MS 31/07/2017: Error should be fixed in Version 1.2.19.0. This can be checked from the Azure Portal under "extensions" in the VM-Menu.
I am struggling with the above - I've installed PowerQuery (64 bit Excel 2013 setup) and under the database connection options, despite following the instructions here to download the Ngpsql data provider for PostgreSQL:
https://support.office.com/en-ie/article/Connect-to-a-PostgreSQL-database-Power-Query-bf941e52-066f-4911-a41f-2493c39e69e4?ui=en-US&rs=en-IE&ad=IE
I can still only see a limited set of options under under the database list, which do not include a PostgreSQL database. Now having hunted around on the web I found this thread:
https://superuser.com/questions/950100/connect-to-postgresql-database-from-excel-2013-power-query-with-npgsql
Which seems to suggest that the reason I cannot see the Postgresql option is that I am not using an OfficePro installation (think it was home edition).
Does anyone have any pointers - any workaround for this? Or do I really have to get a different version of office to get data from a Postgresql db, short of converting the database into Access or something? Thanks
[I have Office 2013 Pro.]
I had to do this in addition to installing "PowerQuery_2.44.4675.281 (64-bit) [en-us].msi" (I ticked GAC installation on the installation dialog) and "Npgsql-3.2.3.msi". Also, rebooted the machine.
Everything then started to work connection-wise.
But when returning large amounts of data into Excel with Powerquery at times I would get "type cast" errors - I could not map it down to NULLs or anything easy to determine. Powerquery seems a fine tool for some usages and I am sure this error can be fixed with data transformation steps.
If you just want to get the postgres data - you can use VBA + ADO. I have just finished setting it up and it works.
Install "psqlodbc_x64.msi".
Add references to your vba project.
Finally I created the connection with this connecton string (no windows DSN setup required - modify the string below as per your setup):
cnn.Open "Driver={PostgreSQL Unicode(x64)};Server=127.0.0.1;Port=xxxx;UID=postgres;PWD=postgres; Database=db_name;"
I could also successfully add it as a data source using the same connection string.
i have a problem in my delphi application with tadodataset
the simple question after switching from delphi xe2 to xe9 and sql2014
i see in profiler that all of my queries started with SET NO_BROWSETABLE ON this cause recompile stored procedures and functions in sql server
can some one tell me how can i disable this option
and sample code
ADODataSet1.Close; ADODataSet1.CommandText := 'Select * from
mytable'; ADODataSet1.Open;
and result in profiler :
SET NO_BROWSETABLE ON
Select * from mytable
Your question seems to imply that the SET NO_BROWSETABLE ON has only started since changing to a more recent Delphi version.
If that is what you mean, I am not sure your observation is accurate, at least I cannot reproduce any difference in behaviour between an app compiled with the current Delphi version, XE10 Seattle and Delphi 7 from 15 years go regarding NO_BROWSETABLE.
I have an instance of Sql Server 2014 running on this machine, which is Windows 10 Pro 64-bit. If I compile and run a minimal Delphi Seattle project that does a simple select * from sometable using a TAdoConnection and a TAdoQuery, Sql Server's Profiler shows the MDac layer sending a SET NO_BROWSETABLE ON, like you. However, if I compile and run exactly the same project in Delphi 7, I get exactly the same statements shown in the profiler - there is no difference whatsoever if the app is compiled in D7 or Seattle.
This is with the cursor location set to clUseClient. The SET NO_BROWSETABLE ON doesn't occur if I use clUseServer. It also doesn't occur in a minimal DBExpress application using a TSqlConnection and TSqlQuery so maybe that uses a server-side cursor as well.
See here for more info about NO_BROWSETABLE:
https://support.microsoft.com/en-us/kb/885146
I'm using the Microsoft OLE DB Provider for Sql Server. Although this is said to be deprecated by MS, I've always had far less trouble with it than with their provider for ODBC or that "native client" one.
You also might want to take a look at this SO q:
Strange ADO behavior generating unwanted NO_BROWSETABLE / set fmtonly queries in VB6
Btw you mention "xe9". Do you mean XE8 or Delphi Seattle or what?
I have strange effects when retrieving columns of type DATE from SQLServer2008 using the Microsoft JDBC-Driver version 3.0 when running under the official Oracle JDK 1.7.0. Host OS is Windows Server 2003.
All Date columns are retrieved as two days in the past with respect to the value actually stored in the column.
I cooked up a minimal code example the test this out (Test table and data):
CREATE TABLE Java7DateTest (
dateColumn DATE
);
INSERT INTO Java7DateTest VALUES('2011-10-10');
Code:
public class Java7SQLDateTest {
public static void main(final String[] argv) {
try {
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection connection = DriverManager.getConnection(
"jdbc:sqlserver://192.168.0.1:1433;databaseName=dbNameHere",
"user", "password");
PreparedStatement statement = connection.prepareStatement("SELECT * FROM Java7DateTest");
ResultSet resultSet = statement.executeQuery();
while (resultSet.next()) {
final java.sql.Date date = resultSet.getDate("dateColumn");
final String str = resultSet.getString("dateColumn");
System.out.println(date + " (raw: " + str + ")");
}
resultSet.close();
statement.close();
connection.close();
} catch (final Throwable t) {
throw new RuntimeException(t.getMessage(), t);
}
}
}
Running this code on above configuration prints: "2011-10-08 (raw: 2011-10-08)".
Under JRE 1.6.0_27 it prints: "2011-10-10 (raw: 2011-10-10)"
I could not find anything that seems to relate to my problem with google, so I'm assuming that its either something stupid I overlooked or nobody is using Java7 yet.
Can anybody confirm this problem? What are my alternatives if I still want to use Java7?
Edit: The problem occurs even when running with -Xint, so its not caused by Hotspot bugs.
Edit2: Old drivers (Microsoft 1.28) work properly with JDK1.7.0 (we were using that driver until maybe two years ago, I think).
jTDS also works perfectly fine with the example. I am considering switching to jTDS, but I am reluctant to do so because I have not the faintest idea what the effects on our productive environment may be. Ideally it should just work, but that what I believed when I switched my dev box to Java7, too.
There is one pretty fat database in the production environment, that is too big to create a copy of, for testing (or rather our server has so little disk left). So setting up a test environment for that one app is not straigthforward, I would have to stitch up a shrinked database for that.
Edit3: jTDS has its own set of catches attached. I found a behavioral difference that breaks one of our applications. ResultSet.getObject() returns different object types for SmallInt columns depending on driver (Short vs Integer). Also jTDS does not implement JDBC4 Connection interface, Connect.isValid() is not supported.
Edit4: I noticed last week that MSSQL-JDBC 3.0 refuses to connect to any DB after I updated to JDK1.6.0_29. jTDS it is then... we switched the productive server yesterday (I fixed tow places where the application was relying on peculiarities of the driver), and so far we had have no problems.
Thank you for your feedback. The Microsoft JDBC Driver for SQL Server does not yet support JRE 1.7.
We are aware of the getDate issue between our JDBC driver & JRE 1.7 and we are looking into publishing a hotfix to enable customers to move forward with non-production testing of our driver with JRE 1.7.
We will publish a link to the hotfix on our blog once available.
http://blogs.msdn.com/b/jdbcteam/
The hotfix is now available. http://blogs.msdn.com/b/jdbcteam/archive/2012/01/20/hotfix-available-for-date-issue-when-using-jre-1-7.aspx
Our blog also contains information on the known issues with JRE 1.6u29 & 1.6u30.
Shamitha Reddy
Program Manager - Microsoft JDBC Driver for SQL Server
I don't quite have an answer for you. But, I've recreated your situation as you described. It is the same with the jdbc driver v3.101 and v3.202 and v4.ctp3 when run under jdk1.7. However, the v2 driver from MS gives your expected answer both under jdk1.6 and jdk1.7. If you need a quick fix and can move to an older jdbc driver, that may work for you.
Other thoughts are on how the MS jdbc driver handles dates and conversion of Date objects between SQL Server and the jvm. Since the storage of the date is without a time zone, the interpretation of the Date object by the driver is based on the default time zone for the machine running the jdbc driver. For instance, if you store a smalldate of '2011-10-11 12:00' and retrieve it from a machine with the default time zone set to GMT-7 then the resulting UTC time of the Date object would be '2011-10-11 19:00'. It could be that there is some change in jdk1.7 that impacts this conversion process in the driver resulting in a wild offset. You might experiment with the ResultSet.getDate(column, Calendar) method to see if a Calendar with a specific time zone gets you the result you want or helps make sense of why you are seeing the strange offset in the conversion.
I don't have a SQL Server setup, but I can't reproduce your problem with PostgreSQL 9.0 and MySQL 5.1 on Windows 7 x64 with JDK 1.7.0. So JDK 1.7.0 can be excluded from being suspect. I have the impression that the SQL Server JDBC driver is to blame here. I'd suggest to use the jTDS JDBC driver instead. It has always been praised for its better performance and stability as opposed to the MS-provided SQL Server JDBC driver.
Information and download link for the hotpatch from Microsoft Support can be found here:
http://support.microsoft.com/kb/2652061
I was experiencing the same issue, where the date would be off by two days, and this hotpatch fixed it.
This is also an issue in OpenJDK 1.6.0_20. However, the mssql driver works fine with Suns JRE 1.6.0_16.