Pagination with EclipseLink, SQL Server and spring data - sql-server

I am currently trying to find out what the best pagination option is with EclipseLink and SQL Server.
I saw that for SQL Server there is no special support for pagination with EclipseLink. If I have the following data model, findAll creates a set of select (n + 1) for each subentity. That doesn't look efficient at first.
public class Foo {
#OneToMany( fetch = FetchType.LAZY )
private Set<Bar> bars = Sets.newHashSet();
}
public class Bar {
#OneToMany( fetch = FetchType.LAZY )
private Set<FooBar> fooBars = Sets.newHashSet();
}
public class FooBar {
String name;
}
So for every bar and fooBar there is a separate select.
Now I would like to discuss a few solutions. My current suggestions are. If a good possibility has been found, I can see how it is best implemented in spring data.
Chunking using Ids with IN
Query query = em.createQuery("SELECT f.id FROM Foo f ORDER BY f.id OFFSET 10 ROWS FETCH NEXT
10 ROWS ONLY");
List<Integer> ids = query.getResultList();
Query query = em.createQuery("SELECT DISTINCT f FROM foo f"
+ " LEFT JOIN FETCH f.bar bars "
+ " LEFT JOIN FETCH bar.fooBars foobars "
+ " WHERE t.id in (:ids) " )
query.setParameter("ids", ids.subList(5, 10));
List<Foo> foos = query.getResultList();
Using IN statement for every subselect (like hibernate) instead of single select
Using Batch with QueryHints IN or JOIN?
https://www.eclipse.org/eclipselink/documentation/2.5/jpa/extensions/q_batch.htm
Another possibility?

Related

How to select unrelated entities with a single query

Let's imagine we have 5 tables that have no relationship between each other but all of them share the same column. Let's name the tables ClojureConf, KotlinConf, ScalaConf, GroovyConf, JavaConf. They all have a column UserId. The number and data types of other columns are different in each of them. A given User may have attended zero or more conferences.
The task is to just select all the records from each of the 5 tables for a given UserId, convert them to DTOs and return as json.
Currently the code does 5 trips to the database to get a list of results from each table.
Is there a library support in hibernate/jpa what would make a single trip to a database? The goal is to improve performance.
Is it possible to define a projection for entities that would look similar to this:
interface ConfAttended {
List<ClojureConf> getClojureConfs();
List<KotlinConf> getKotlinConfs();
List<ScalaConf> getScalaConfs();
List<GroovyConf> getGroovyConfs();
List<JavaConf> getJavaConfs();
}
and a repository that would select and map the results in one go
interface ConfAttendedDAO extends JpaRepository<User, Long> {
#Query("SELECT c, k, s, g, j FROM ClojureConf c " +
"JOIN KotlinConf k ON c.UserId = k.UserId " +
"JOIN ScalaConf s ON c.UserId = s.UserId " +
"JOIN GroovyConf g ON c.UserId = g.UserId " +
"JOIN JavaConf j ON c.UserId = j.UserId " +
"WHERE c.UserId = :userId")
ConfAttended findByUserIdForProjection(#Param("userId") long userId);
}
?
I ended up having a query like this:
interface ConfAttendedDAO extends JpaRepository<User, Long> {
#Query("SELECT c, k, s, g, j FROM User u " +
"LEFT JOIN ClojureConf c ON c.UserId = :userId " +
"LEFT JOIN KotlinConf k ON k.UserId = :userId " +
"LEFT JOIN ScalaConf s ON s.UserId = :userId " +
"LEFT JOIN GroovyConf g ON g.UserId = :userId " +
"LEFT JOIN JavaConf j ON j.UserId = :userId " +
"WHERE u.Id = :userId")
List<Object[]> findAllByUserId(#Param("userId") long userId);
}
Hibernate takes care of mapping rows to entities. Each Object[] has all 5 entities (or nulls) as its elements. Selecting from User is to force the query to return results. Otherwise if first table returns nothing - the whole query returns nothing. Another downside is that if one table has 10 results and another has 1, the table with less results gets them duplicated.
As for performance (the sole point of doing all this), getting and processing results is 4-5 times faster than with 5 separate SELECTs.

Get random result from JPQL query over large table

I'm currently using JPQL queries to retrieve information from a database. The purpose of the project is testing a sample environments through randomness with different elements so I need the queries to retrieve random single results all through the project.
I am facing that JPQL does not implement a proper function for random retrieval and postcalculation of random takes too long (14 seconds for the attached function to return a random result)
public Player getRandomActivePlayerWithTransactions(){
List<Player> randomPlayers = entityManager.createQuery("SELECT pw.playerId FROM PlayerWallet pw JOIN pw.playerId p"
+ " JOIN p.gameAccountCollection ga JOIN ga.iDAccountStatus acs"
+ " WHERE (SELECT count(col.playerWalletTransactionId) FROM pw.playerWalletTransactionCollection col) > 0 AND acs.code = :status")
.setParameter("status", "ACTIVATED")
.getResultList();
return randomPlayers.get(random.nextInt(randomPlayers.size()));
}
As ORDER BY NEWID() is not allowed because of JPQL restrictions I have tested the following inline conditions, all of them returned with syntax error on compilation.
WHERE (ABS(CAST((BINARY_CHECKSUM(*) * RAND()) as int)) % 100) < 10
WHERE Rnd % 100 < 10
FROM TABLESAMPLE(10 PERCENT)
Have you consider to generate a random number and skip to that result?
I mean something like this:
String q = "SELECT COUNT(*) FROM Player p";
Query query=entityManager.createQuery(q);
Number countResult=(Number) query.getSingleResult();
int random = Math.random()*countResult.intValue();
List<Player> randomPlayers = entityManager.createQuery("SELECT pw.playerId FROM PlayerWallet pw JOIN pw.playerId p"
+ " JOIN p.gameAccountCollection ga JOIN ga.iDAccountStatus acs"
+ " WHERE (SELECT count(col.playerWalletTransactionId) FROM pw.playerWalletTransactionCollection col) > 0 AND acs.code = :status")
.setParameter("status", "ACTIVATED")
.setFirstResult(random)
.setMaxResults(1)
.getSingleResult();
I have figured it out. When retrieving the player I was also retrieving other unused related entity and all the entities related with that one and so one.
After adding fetch=FetchType.LAZY (don't fetch entity until required) to the problematic relation the performance of the query has increased dramatically.

Not retrieving any data from a search query

I need to search all the usernames and for the application to print out all the usernames containting the search string. I think I have done this but when I execute searchByString(), I just get no results.
AbstractFacade
/* trying out a search function */
public List<T> searchByString(String string) {
System.out.println("in SearchByString");
return getEntityManager().createNamedQuery("Userdetails.findByUsername").setParameter("username", "%" + string + "%").getResultList();
}
Userdetails.java this is where the query is done
#Entity
#Table(name = "USERDETAILS")
#XmlRootElement
#NamedQueries({
#NamedQuery(name = "Userdetails.findAll", query = "SELECT u FROM Userdetails u"),
#NamedQuery(name = "Userdetails.findById", query = "SELECT u FROM Userdetails u WHERE u.id = :id"),
#NamedQuery(name = "Userdetails.findByUsername", query = "SELECT u FROM Userdetails u WHERE u.username = :username")})
I can not get a filled list. I am wondering if the query is running right and retriving anything, so I want to be able to see what it retrieves when it is run, so I can look at why nothing gets displayed to the user.
Your query parameter looks like you want a like clause, but your named query uses =. Change
SELECT u FROM Userdetails u WHERE u.username = :username
to
SELECT u FROM Userdetails u WHERE u.username like :username
and it should work.

JPA Entity use index hint

Is it possible to specify a database index hint on a play framework Entity query.
My code looks like:
public static List<Transaction> findOnInactive(Date date) {
return Transaction.find(
"date = ? and account in ( select d.acctNb from account d "
+ " where d.date = ? and (d.inactive = true or d.blocked = true)"
+ " group by d.acctNb )", date, date).fetch();
}
Running the generated query takes 20 sec. However running the same query manually with
select * from transaction with (INDEX(_dta_index_k1_1)) ...
only take 1 sec. Anyway I could specify the index hint in my JPA query?
You need to use native SQL query, something like this:
return JPA.em().createNativeQuery(
"select * from transaction with (INDEX(_dta_index_k1_1)) ...",
Transaction.class).getResultList();

Converting complex sql stored proc into linq

I'm using Linq to Sql and have a stored proc that won't generate a class. The stored proc draws data from multiple tables into a flat file resultset.
The amount of data returned must be as small as possible, the number of round trips to the Sql Server need to be limited, and the amount of server-side processing must be limited as this is for an ASP.NET MVC project.
So, I'm trying to write a Linq to Sql Query however am struggling to both replicate and limit the data returned.
Here's the stored proc that I'm trying to convert:
SELECT AdShops.shop_id as ID, Users.image_url_75x75, AdShops.Advertised,
Shops.shop_name, Shops.title, Shops.num_favorers as hearts, Users.transaction_sold_count as sold,
(select sum(L4.num_favorers) from Listings as L4 where L4.shop_id = L.shop_id) as listings_hearts,
(select sum(L4.views) from Listings as L4 where L4.shop_id = L.shop_id) as listings_views,
L.title AS listing_title, L.price as price, L.listing_id AS listing_id, L.tags, L.materials, L.currency_code,
L.url_170x135 as listing_image_url_170x135, L.url AS listing_url, l.views as listing_views, l.num_favorers as listing_hearts
FROM AdShops INNER JOIN
Shops ON AdShops.shop_id = Shops.shop_id INNER JOIN
Users ON Shops.user_id = Users.user_id INNER JOIN
Listings AS L ON Shops.shop_id = L.shop_id
WHERE (Shops.is_vacation = 0 AND
L.listing_id IN
(
SELECT listing_id
FROM (SELECT l2.user_id , l2.listing_id, RowNumber = ROW_NUMBER() OVER (PARTITION BY l2.user_id ORDER BY NEWID())
FROM Listings l2
INNER JOIN (
SELECT user_id
FROM Listings
GROUP BY
user_id
HAVING COUNT(*) >= 3
) cnt ON cnt.user_id = l2.user_id
) l2
WHERE l2.RowNumber <= 3 and L2.user_id = L.user_id
)
)
ORDER BY Shops.shop_name
Now, so far I can return a flat file but am not able to limit the number of listings. Here's where I'm stuck:
Dim query As IEnumerable = From x In db.AdShops
Join y In (From y1 In db.Shops
Where y1.Shop_name Like _Search + "*" AndAlso y1.Is_vacation = False
Order By y1.Shop_name
Select y1) On y.Shop_id Equals x.shop_id
Join z In db.Users On x.user_id Equals z.User_id
Join l In db.Listings On l.Shop_id Equals y.Shop_id
Select New With {
.shop_id = y.Shop_id,
.user_id = z.user_id,
.listing_id = l.Listing_id
} Take 24 ' Fields ommitted for briefity...
I assume to select a random set of 3 listings per shop, I'd need to use a lambda expression however am not sure how to do this. Also, need to add in somewhere consolidated totals for listing fieelds against individual shops...
Anyone have any thoughts?
UPDATE:
Here's the current solution that I'm looking at:
Result class wrapper:
Public Class NewShops
Public Property Shop_id As Integer
Public Property listing_id As Integer
Public Property tl_listing_hearts As Integer?
Public Property tl_listing_views As Integer?
Public Property listing_creation As Date
End Class
Linq + code:
Using db As New Ads.DB(Ads.DB.Conn)
Dim query As IEnumerable(Of IGrouping(Of Integer, NewShops)) =
(From x In db.AdShops
Join y In (From y1 In db.Shops
Where (y1.Shop_name Like _Search + "*" AndAlso y1.Is_vacation = False)
Select y1
Skip ((_Paging.CurrentPage - 1) * _Paging.ItemsPerPage)
Take (_Paging.ItemsPerPage))
On y.Shop_id Equals x.shop_id
Join z In db.Users On x.user_id Equals z.User_id
Join l In db.Listings On l.Shop_id Equals y.Shop_id
Join lt In (From l2 In db.Listings _
Group By id = l2.Shop_id Into Hearts = Sum(l2.Num_favorers), Views = Sum(l2.Views), Count() _
Select New NewShops With {.tl_listing_views = Views,
.tl_listing_hearts = Hearts,
.Shop_id = id})
On lt.Shop_id Equals y.Shop_id
Select New NewShops With {.Shop_id = y.Shop_id,
.tl_listing_views = lt.tl_listing_views,
.tl_listing_hearts = lt.tl_listing_hearts,
.listing_creation = l.Creation,
.listing_id = l.Listing_id
}).GroupBy(Function(s) s.Shop_id).OrderByDescending(Function(s) s(0).tl_listing_views)
Dim Shops as New Dictionary(Of String, List(Of NewShops))
For Each item As IEnumerable(Of NewShops) In query
Shops.Add(item(0).shop_name, (From i As NewShops In item
Order By i.listing_creation Descending
Select i Take 3).ToList)
Next
End Using
Anyone have any other suggestions?
From the looks of that SQL and code, I'd not be turning it into LINQ queries. It'll just obfuscate the logic and probably take you days to get it correct.
If SQLMetal doesn't generate it properly, have you considered using the ExecuteQuery method of the DataContext to return a list of the items you're after?
Assuming that your sproc you're trying to convert is called sp_complicated, and takes in one parameter, something like the following should do the trick
Protected Class TheResults
Public Property ID as Integer
Public Property image_url_75x75 as String
'... and so on and so forth for all the returned columns. Be careful with nulls
End Class
'then, when you want to use it
Using db As New Ads.DB(Ads.DB.Conn)
dim results = db.ExecuteQuery(Of TheResults)("exec sp_complicated {0}", _Search)
End Using
Before you freak out, that's not susceptible to SQL Injection. L2SQL uses proper SQLParameters, as long as you use the squigglies and don't just concatenate the strings yourself.

Resources