Querydsl generates invalid SQL on basic aggregate query - sql-server

I am using QueryDSL and Hibernate to query a SqlServer2008 database.
I have a Sessions table that contains a sessionId column. I want to obtain the max sessionId.
I run the following query:
QSessions session = QSessions.sessions;
HibernateSQLQuery query = new HibernateSQLQuery(sessionFactory.openSession(), new SQLServer2005Templates());
query.from(session).list(session.sessionId.max());
My hibernate.cfg.xml:
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.url">jdbc:sqlserver://SVR;databaseName=DB</property>
<property name="hibernate.connection.username">X</property>
<property name="hibernate.connection.password">X</property>
<property name="hibernate.connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
<property name="hibernate.dialect">org.hibernate.dialect.SQLServer2008Dialect</property>
<mapping class="com.coveo.data.Sessions" />
</session-factory>
</hibernate-configuration>
When I run the query, I can see that Hibernate attempts to run the following SQL query:
17:36:10.011 [main] DEBUG org.hibernate.SQL - select max(sessions.*.sessionId) as col__1 from sessions
This is not valid SQL. I would have expected
select max(sessions.sessionId) as col__1 from sessions
When Hibernate attempts to execute the query, I get the following error:
com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near '*'.
I get the same syntax error when attempting to run the invalid SQL query using SQL Management Studio.
Why does Querydsl require a dialect? Isn't Hibernate in charge of generating the SQL?
Why is the generated SQL invalid?
EDIT: I get the same behavior when using .avg(), .min() or another aggregate function.

I ended up switching to jOOQ.
So far, the experience has been very positive. The API is somewhat similar, the documentation is better and it doesn't generate invalid SQL.

There are a few issues here.
You use HibernateSQLQuery which is meant to execute SQL queries through the Hibernate API, please use HibernateQuery for Hibernate ORM queries
You use the Q-type generated for ORM queries in SQL queries. Querydsl should handle this usage better and I created a ticket for it https://github.com/mysema/querydsl/issues/463
Querydsl supports both ORM and SQL based queries for Hibernate and you accidently mixed them.
For normal Hibernate queries your example would be
QSessions session = QSessions.sessions;
HibernateQuery query = new HibernateQuery(sessionFactory.openSession());
query.from(session).list(session.sessionId.max());
I assume that QSessions is based on the Sessions entity.
jOOQ is a very good alternative for the SQL level, but if you need to access also JPA and NoSQL based storages, Querydsl offers more.

Related

Jooq with SQL Server How to select top N results

In our project we are using two databases, one postgres and one mssql. We are using Jooq in order to query these DBs and with postgres everything was pretty straight forward!
But with mssql we are facing some troubles. The task is to select the top 10 values and let's say that we have the following java code:
DSL.using(conn)
.select(USE_CASE.asterisk())
.from(USE_CASE)
.where(USE_CASE.RECORD_ACTIVE.eq(true))
.orderBy(USE_CASE.CREATED_ON.desc())
.limit(10)
.offset(0)
.fetch(new UseCaseMapper()))
This works like a charm with postgres but on mssql we get the following error:
Execution exception[[CompletionException: org.jooq.exception.DataAccessException:
SQL [select "park"."dbo"."use_case".* from "park"."dbo"."use_case" where
"park"."dbo"."use_case"."record_active" = ?
order by "park"."dbo"."use_case"."created_on" desc limit ?];
Incorrect syntax near 'limit'.]]
I know that for mssql the equivalent query would be something like,
select top 10 *
from use_case
where record_active = true
order by created_on desc;
How can I change my java code to get the limit records in mssql?
Correct SQLDialect
The exception message hints at the fact that you're still using the PostgreSQL (or some other non-SQL Server) dialect as your catalogs/schemas/tables/columns are quoted using "double_quotes", instead of [brackets].
Just use SQLDialect.SQLSERVER instead, when running your query on SQL Server. The LIMIT 10 syntax will then be correctly translated to TOP 10.
jOOQ Professional Edition vs jOOQ Open Source Edition
From our posted error messages, it is not quite clear how you really configured your jOOQ integration, but if you're using DSL.using(Connection) without explicitly specifying the SQLDialect, and jOOQ doesn't correctly "guess" the appropriate SQLDialect, this can mostly be because of one of two reasons:
You're using some non-standard JDBC URL, which jOOQ doesn't recognise. It thus uses SQLDialect.DEFAULT, instead, which produces the wrong SQL syntax for your.
You're using the jOOQ Open Source Edition (Maven groupId org.jooq) instead of the jOOQ Professional Edition (Maven groupId org.jooq.pro, to be installed manually, as it is not distributed via Maven Central), which would also lead to jOOQ using SQLDialect.DEFAULT.
Notice: this can also happen by accident, e.g. as a transitive dependency that is being pulled in via Spring Boot or some other framework that already depends on the jOOQ Open Source Edition. In that case, you have to make sure to exclude that transitive dependency from ending up on your classpath. In Maven, you can display all your dependencies using mvn dependency:tree

Simple query generates Error 3078 "The Microsoft Jet database engine cannot find the input table or query"

Any suggestions how to restructure this simple query?
Hitting a SQL Server Database using DAO via VB6 (updated old code to work with new database) and somehow this query in one of the apps is giving fits.
Select I.sType, Count(I.BarcodeID) AS CountOfID
FROM (SELECT DISTINCT sType, BarcodeID From [Ready]) as I
GROUP BY I.sType
ORDER BY sType
I've pasted the query into SQL Enterprise Manager and it runs just fine as expected. It worked fine hitting the original Access DB. But somehow the DAO via ODBC hitting the SQL Server is generating:
"The Microsoft Jet database engine cannot find the input table or query"
Things I've tried:
Removing the ()
Removing "as I" and "I." notation.
Brackets around [Ready] and without (thinking it might be reserved
keyword).
Confirmed that the connection is to the ODBC for Sql Server (this connection is used for other queries too.
Checked SQL profiling tool and the query is making it to the server (edited, I was checking the wrong tool on SQL Express)
EDIT: just to sate everyone's concern that I'm not really hitting SQL Server, I edited the SQL command to an even simpler
SELECT DISTINCT sType, BarcodeID From [Ready]
and it works correctly ON the same connection, so the connection is 100% confirmed to be SQL Server, the error message is in error (reported "Access").
So the problem is with no doubt the query FROM a query.
UPDATE:
Definitely confirmed my suspicions that the query, albeit a normal one, is not working with ADO-> ODBC-> Sql Server.
The first step SQL is doing is trying to validate the columns, etc.. of the "from" table (which in this case is not a table, but a query itself).
When I run a basic SELECT DISTINCT sType, BarcodeID From [Ready] SQL checks the columns, keys, indexes, etc of the table called "Ready" (exec sp_special_columns N'Ready',NULL,NULL,N'V',N'T',N'U'), then proceeds to return the results.
When I use a subquery, SQL is firing the same check on the table def, but for a table called SELECT DISTINCT sType, BarcodeID From [Ready] which of course does not exist and it returns an error (exec sp_special_columns N'SELECT DISTINCT sType, BarcodeID From [Ready]',NULL,NULL,N'V',N'T',N'U'). VB6/ADO is reporting correctly that SQL says the table is not found. So apparently this query from a query stumps the capabilities of ADO->ODBC...
since the issue is clearly the Select FROM (Select From) structure of querying a query being incompatible with ADO->ODBC->SQL Server, my solution was simply to make the subquery a view in SQL and alter the query to select from that.

SQL Azure V12 BACPAC import error: "The internal target platform type SqlAzureV12DatabaseSchemaProvider does not support schema file version '3.3'"

Until a very few days ago I was able to import a V12 BACPAC from Azure to my local server with SQL Server 2014 SP1 CU6 (12.0.4449.0).
But now, when I try to import the BACPAC, my SQL Server Management Studio 2014 says:
"Internal Error. The internal target platform type SqlAzureV12DatabaseSchemaProvider does not support schema file version '3.3'. (File: D:\MyDB.bacpac) (Microsoft.Data.Tools.Schema.Sql)"
I think I've the latest SQL Server 2014 SP1 version with all the latest updates (build 12.0.4449.0) but still I get this error.
Please help!
Thanks
Fix: To resolve, use the latest SSMS Preview which installs the most up to date DacFx version. This understands how to process the latest features, notably Database Scoped Configuration Options. Once this is installed you can Import inside SSMS or using SqlPackage from the “C:\Program Files (x86)\Microsoft SQL Server\130\DAC\bin” location if you prefer command line tools.
Alternatively, execute the following command on the Azure DB to set MaxDop value back to default since it appears the issue is that this has been changed to 1. Future exports should now produce bacpacs that can be understood by the 2014 client tools, assuming no other new Azure features have been added to the DB.
ALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 0
Root cause / why does this happen: The root cause is that your database have non-default values for 1 or more Database Scoped Configuration options. As these were only added very recently, older versions of the tools do not understand how to deploy them and so DacFx blocks. These are the only properties/objects with that high a schema version. Basically any time you see an error like “does not support schema file version '3.3'” it means you need to upgrade. One possible cause is if the database was migrated from AzureV1 -> AzureV12, which sets the MaxDop option to 1 from its default of 0.
Notes: It's strongly recommended that you use the latest SSMS and keep it up to date via the built-in update notifications if you're working with Azure. it will ensure that you avoid running into issues like this one. Generally if you only use the SQL Server 2014 surface area you should be able to use older tools when re-importing, but with the huge number of recent advancements in Azure SQL DB cases like this will crop up more and more often where the new tools are required in order to perform as expected.
For reference, I’m including the Database Scoped Configuration options and their default values below. If any of these properties are non-default on the DB when exporting the schema version gets bumped so that old tools do not break.
<!-- Database Scoped Configurations-->
<Property Name="MaxDop" Type="System.Int32" DefaultValue="0" />
<Property Name="MaxDopForSecondary" Type="System.Int32?" DefaultValue="null"/>
<Property Name="LegacyCardinalityEstimation" Type="System.Boolean" DefaultValue="false" />
<Property Name="LegacyCardinalityEstimationForSecondary" Type="System.Boolean?" DefaultValue="null" />
<Property Name="ParameterSniffing" Type="System.Boolean" DefaultValue="true" />
<Property Name="ParameterSniffingForSecondary" Type="System.Boolean?" DefaultValue="null" />
<Property Name="QueryOptimizerHotfixes" Type="System.Boolean" DefaultValue="false" />
<Property Name="QueryOptimizerHotfixesForSecondary" Type="System.Boolean?" DefaultValue="null" />
The simple "Alter" solution given by Kevin (ALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 0) seems to be the fast solution to resolve the crisis for anyone having customer-down issues. Never mind about installing the latest DAC or SQL Server 2016, it's not necessary to resolve the immediate issue, PLUS all that is in preview status (beta). Hardly something you want to introduce into a production environment right now
This apparently only happened to us if we had a v11 database pending auto update by MSFT set for this last weekend. For those database upgrades we canceled and applied the upgrade ourselves, the Max Degree Of Parallelism field appears not to have gotten set to 0, and this error occurred. We have about 300 db's and noticed this as the pattern
FYI: You can check for that problem value with this SQL query
SELECT [dbscm].[value] AS [MaxDop],
[dbscm].[value_for_secondary] AS [MaxDopForSecondary],
[dbscl].[value] AS [LegacyCardinalityEstimation],
[dbscl].[value_for_secondary] AS
[LegacyCardinalityEstimationForSecondary],
[dbscp].[value] AS [ParameterSniffing],
[dbscp].[value_for_secondary] AS
[ParameterSniffingForSecondary],
[dbscq].[value] AS [QueryOptimizerHotfixes],
[dbscq].[value_for_secondary] AS
[QueryOptimizerHotfixesForSecondary]
FROM [sys].[databases] [db] WITH (NOLOCK)
LEFT JOIN [sys].[database_scoped_configurations] AS [dbscm] WITH
(NOLOCK) ON [dbscm].[name] = N'MAXDOP'
LEFT JOIN [sys].[database_scoped_configurations] AS [dbscl] WITH
(NOLOCK) ON [dbscl].[name] = N'LEGACY_CARDINALITY_ESTIMATION'
LEFT JOIN [sys].[database_scoped_configurations] AS [dbscp] WITH
(NOLOCK) ON [dbscp].[name] = N'PARAMETER_SNIFFING'
LEFT JOIN [sys].[database_scoped_configurations] AS [dbscq] WITH
(NOLOCK) ON [dbscq].[name] = N'QUERY_OPTIMIZER_HOTFIXES'
WHERE [db].[name] = DB_NAME();
I was facing the same issue while I was importing an export from azure to my local MSSQLLocalDB instance (for local debugging).
I did not want to touch the azure db neither wanted to download the latest preview.
So What I did was as follows On my local db:
Executed the alter query setting the value for MAXDOP to 1
ALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 1
Imported the bacpac, which ran successfully.
Rest the value of MAXDOP to 0
ALTER DATABASE SCOPED CONFIGURATION SET MAXDOP = 0
Hope it helps somebody in similar use case

Query TimeOut Hints not working in Eclipse Link 2.6

Query TimeOut Hints not working in Eclipse Link 2.6 when used with MSSQL Server DB
Tried with below options.
1.Adding hints to the Query Itself
Query query = em.createNamedQuery(AlertCategoryType.FIND_ACTIVE_TYPES);
query.setParameter("isInactive", Boolean.FALSE);
query.setHint(QueryHints.JDBC_TIMEOUT, "1");
Added Eclipse link hints to Named query
#NamedQuery(name = "AlertCategoryType.findByIsInactive", query = "SELECT a FROM AlertCategoryType a", hints={#QueryHint(name="eclipselink.jdbc.timeout", value="50")})
Added JPA hints to Named query
#NamedQuery(name = "AlertCategoryType.findByIsInactive", query = "SELECT a FROM AlertCategoryType a", hints={#QueryHint(name="eclipselink.jdbc.timeout", value="50")})
Tried adding property in persistence.xml
None of the above hints worked. Is there anything else missing out?
Please suggest any work around to work query time out when using any persistent provider with MSSQL server DB.
Thanks,
Vittal
If none of the above works with MS SQL Server try with javax.persistence.query.timeout. Keep in mind that neither a persistence provider nor database is obliged to support it.
As specified in JPA 2.0 Specification, chapter 3.8.9. Query Hints:
Portable applications should not rely on this hint. Depending on the
persistence provider and database in use, the hint may or may not be
observed.
If you are not tied to JPA 2.0 you can try with pure JDBC instead as it seems to be supported:
Is setQueryTimeout for JDBC driver 2.0 on CallableStatement supported?
MSDN > JDBC Driver API Reference > setQueryTimeout Method (SQLServerStatement)
Sample JDBC Driver Applications > Connection URL Sample

EclipseLink generated SQL doesn't include pagination

(My environment: Windows 7 x64 and Server 2008, EclipseLink 2.5.2, I've tried the following with JTDS and the MS JDBC driver, and with MS SQL Server Express 2008 and 2012.)
I'm doing a paginated query in a table with anywhere from 5-50 million rows, with about 5 million rows matching the filter criteria. The following is the paginated query I'm using to display this data in the UI 25 rows at a time. The pagination is working properly - the list contains just 25 rows for each page. However, the query takes 24 seconds to return 25 rows, which seems long.
My goal is to log the generated SQL, so I can see exactly how JPA is accomplishing the pagination in SQL Server 2008 vs 2012. But the generated SQL doesn't include anything to do with pagination, which makes me wonder what else I'm not seeing in the generated SQL.
The query:
CriteriaBuilder cb = JPA.em().getCriteriaBuilder();
CriteriaQuery<RGHICarrierPull> cq = cb.createQuery(RGHICarrierPull.class);
Root<RGHICarrierPull> from = cq.from(RGHICarrierPull.class);
CriteriaQuery<RGHICarrierPull> select = cq.select(from);
// Add filter/sort predicates to "predicates"
...
select.where(predicates);
// Apply pagination
records.setFirstResult((page-1)*limit);
records.setMaxResults(limit);
// Get data
List<RGHICarrierPull> lst = records.getResultList();
To log the generated SQL programatically:
// Log the sql for this query
Session session = JPA.em().unwrap(JpaEntityManager.class).getActiveSession();
DatabaseQuery databaseQuery = ((EJBQueryImpl)records).getDatabaseQuery();
databaseQuery.prepareCall(session, new DatabaseRecord());
System.out.println(databaseQuery.getSQLString());
The logged SQL:
SELECT t1.SHIPTO_ZIP, t1.WHSE, t1.ANYNBR1, t1.ANYTEXT1, t1.CREATE_DATE_TIME, t1.
MOD_DATE_TIME, t1.PULL_TIME, t1.PULL_TIME_AMPM, t1.PULL_TRLR_CODE, t1.USER_ID,
1.SHIP_VIA FROM Ship_Via t0, RGHI_Carrier_Pull t1 WHERE ((t1.WHSE = 'WHSE1') AND
(t0.SHIP_VIA = t1.SHIP_VIA)) ORDER BY t0.SHIP_VIA ASC, t1.SHIPTO_ZIP ASC
Obviously, this is not a paginated query, so if I run this query directly, it runs for over a minute and returns all 5 million rows. I get the same results if I use persistence.xml settings to log all JPA queries, and also if I log the SQL from MS SQL Server.
Is this the actual generated SQL? I see two possibilities:
This is the full generated SQL, but EclipseLink is doing something else to accomplish the pagination.
EclipseLink is logging this generated SQL prior to the pagination stuff being added to it
Try setting the log level in EclipseLink to Finest, and check the database platform that is being used. EclipseLink logging will also show what is sending to the database. This should log the same SQL as what you get from the getSQLString(), but allows you to validate you are executing the correct api, and the initial start up logging will show if the platform being used matches your database, otherwise it will need to be specified using the target-database property: http://www.eclipse.org/eclipselink/documentation/2.4/jpa/extensions/p_target_database.htm
EclipseLink will use pagination within the generated SQL as described here http://wiki.eclipse.org/EclipseLink/Examples/JPA/Pagination if the plaform supports it, otherwise it resorts to using JDBC api to limit the results sent across, and then jump to the first result in the returned resultset, which is less efficient then if pagination is done entirely in the database.
EclipseLink (at least up to the current version 2.6.1) supports neither SQL Server 2012 pagination in the form of OFFSET-FETCH syntax nor older SQL Server TOP syntax, check out EclipseLink Database Support. Instead EclipseLink internally uses JDBC feature Statement.setMaxRows() which essentially discards excessive rows from the returning ResultSet. And there is no plans to support that in the future versions so far.
You can try to implement this manually extending the SQLServerPlatform and overriding method printSQLSelectStatement() similarly to how it is done in PostgreSQLPlatform.printSQLSelectStatement(). The working prototype is here: https://github.com/roman-sinyakov/eclipselink/blob/master/SQLServer2012Platform.java.

Resources