I'm using Spring Boot with Ojdbc8 18.3.0.0.0
With Hikari Datasource and JPA, all query work fine.
But now I need to set Query timeout for all database query
I was try many way:
javax.persistence.query.timeout=1000
spring.transaction.default-timeout=1
spring.jdbc.template.query-timeout=1
spring.jpa.properties.hibernate.c3p0.timeout=1
spring.jpa.properties.javax.persistence.query.timeout=1
Config Class:
#Configuration
public class JPAQueryTimeout {
#Value("${spring.jpa.properties.javax.persistence.query.timeout}")
private int queryTimeout;
#Bean
public PlatformTransactionManager transactionManager() throws Exception {
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setDefaultTimeout(queryTimeout); //Put 1 seconds timeout
return txManager;
}
}
Query:
List<Integer> llll = manager.createNativeQuery("select test_sleep(5) from dual")
.setHint("javax.persistence.query.timeout", 1).getResultList();
The database task take 5 second before return value, but in all of case, no error occor.
Could anyone tell me how to set query timeout?
You can try using the simplest solution, that is using the timeout value within #Transactional;
#Transactional(timeout = 1) // in seconds (so that is 1 second timeout)
public Foo runQuery(String id) {
String result = repo.findById(id);
// other logic
}
Be aware that the method annotated with #Transactional must be public for it to work properly
Related
I'm having trouble with transactions in Spring Boot using #Transactional annotation. The latest Spring is connected to a MS SQL Database.
I have following service, which periodically executes transactional method according to some criteria:
#Service
public class SomeService {
SomeRepository repository;
public SomeService(SomeRepository someRepository) {
this.repository = someRepository;
}
#Scheduled(fixedDelayString="${property}") //10 seconds
protected scheduledIteration() {
if(something) {
insertDataInNewTransaction(getSomeData());
}
}
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
protected void insertDataInNewTransaction(List<Data> data) {
//insert data to db
repository.saveAll(data);
//call verify proc
repository.verifyData();
}
}
The algorithm supposed to process data, insert them into table and perform check (db procedure). If the procedure throws an exception, the transaction should be rollbacked. I'm sure, that the procedure does not perform commit of the transaction.
The problem I'm facing is, that calling the method does not begin new transaction (or does but it's auto-commited), because I've tried following:
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
protected void insertDataInNewTransaction(List<Data> data) {
int counter = 0;
for(Data d : data) {
repository.save(d);
counter++;
//test
if(counter == 10) {
throw new Exception("test");
}
}
}
After the test method is executed, the first 10 rows remain in the table, where they were supposed to be rollbacked. During debugging I've noticed, that calling repository.save() in the loop inserts to the table outside transaction, because I can see the row from DB IDE while debugger sitting on next row. This gave me an idea, that the problem is caused by auto-commit, as it's MS SQL default. So I have tried to add following properties, but without any difference:
spring.datasource.hikari.auto-commit=false
spring.datasource.auto-commit=false
Is there anything I'm doing wrong?
If you use Spring Proxy AOP, then you need to turn the method insertDataInNewTransaction as public.
Remember that if the method is public, but it is invoked from the same bean, it will not create a new transaction (because spring proxies won't be call).
Short answer:
#Transactional(propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
public void insertDataInNewTransaction(List<Data> data) {
//insert data to db
repository.saveAll(data);
//call verify proc
repository.verifyData();
}
But if you really need a new separate transaction use Propagation.REQUIRES_NEW instead of Propagation.REQUIRED.
I am working on a Spring project. I want to use scheduler in it and want to schedule it on a variable date. This date has to be taken from database. Is it possible to fetch data from database before server is getting started?
Two solutions come to my mind:
#PostConstruct annotated method of some #Component:
#Component
public class MyBean
{
#PostConstruct
public void init()
{
// Blocking DB call could go here
}
}
Application Events. For the ApplicationReadyEvent:
#Component
public class ApplicationReadyEventListener implements ApplicationListener<ApplicationReadyEvent>
{
#Override
public void onApplicationEvent(ApplicationReadyEvent event)
{
// DB call could go here
//
// Notice that if this is a web services application, it
// would potentially be serving requests while this method
// is being executed
}
}
Could someone please tell me why this spring transaction is not rolling back appropriately?
The error I get is this:
org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.transaction.PlatformTransactionManager' available
This is my repository with a save transaction that will intentionally fail:
#Repository
public class TransactionalRepository {
private final PlayerRepository playerRepository;
#Autowired
public TransactionalRepository(PlayerRepository playerRepository) {
this.playerRepository = playerRepository;
}
public Player saveSuccess(Player player) {
return playerRepository.save(player);
}
#Transactional
public Player saveFail(Player player) {
player.setName("FAIL"); // should not be saved in DB if transaction rollback is successful
player = playerRepository.save(player);
throw new IllegalStateException("intentionally fail transaction");
}
}
And here is the test:
#RunWith(SpringRunner.class)
#SpringBootTest
public class MongoTransactionApplicationTests {
#Autowired
public TransactionalRepository playerRepository;
#Test
public void contextLoads() {
Player player = new Player();
player.setId(UUID.randomUUID().toString());
final String PLAYER_NAME = "new-"+player.getId().subSequence(0,8);
player.setName(PLAYER_NAME);
player = playerRepository.saveSuccess(player);
try {
player = playerRepository.saveFail(player);
} catch (IllegalStateException e) {
// this is supposed to fail
}
Assert.assertEquals(PLAYER_NAME, player.getName());
}
}
Download all the code here if you want to see it run
Unlike other implementations the Spring Data MongoDB module does not by default register a PlatformTransactionManager if none is present. This is up to the users configuration, to avoid errors with non MongoDB 4.x servers as well as projects already using #Transactional along with a non MongoDB specific transaction manager implementation. Please refer to the reference documentation for details.
Just add a MongoTransactionManager to your configuration.
#Bean
MongoTransactionManager txManager(MongoDbFactory dbFactory) {
return new MongoTransactionManager(dbFactory);
}
You might also want to check out the Spring Data Examples and have a look at the one for MongoDB transactions.
I have a webjob that calls a long running stored procedure that keeps timing out. Can anyone help please?
The web job is called using the following code:
static void Main()
{
ApplicationDbContext context = new ApplicationDbContext();
context.Database.CommandTimeout = 6000;
context.PopulateJobTypeDescendants();
}
The method on the context (ApplicationDbContext) is shown below:
public void PopulateJobTypeDescendants()
{
Database.ExecuteSqlCommand("PopulateJobTypeDescendants");
}
The following exception is raised when the web job is run. We have read that it could be related to the plan/DTUs on the server so we went from S1 -> S3, this still didn't solve the issue and the process bombs out after 45 seconds. The strange thing is that if I connect to azure sql db from SSMS and call the stored procedure it works fine.
[07/11/2016 22:25:02 > e2cf50: ERR ] Unhandled Exception:
System.Data.SqlClient.SqlException: Timeout expired. The timeout
period elapsed prior to completion of the operation or the server is
not responding. This failure occurred while attempting to connect to
the routing destination. The duration spent while attempting to
connect to the original server was - [Pre-Login] initialization=14;
handshake=26; [Login] initialization=0; authentication=0; [Post-Login]
complete=1; ---> System.ComponentModel.Win32Exception: The wait
operation timed out
The connection string is shown below:
<add name="TempsContext" connectionString="Server=tcp:[XXX],1433;Database=temps_testing;User ID=[XXX];Password=[XXX];Trusted_Connection=False;Encrypt=True;Connection Timeout=600;" providerName="System.Data.SqlClient" />
It is possible that this is caused by some inconsistencies in how EF propagates the value of CommandTimeout to the commands its creates, e.g. to do database initialization or to obtain versioning information from the server.
It should be possible to use command interception as a workaround, e.g.:
using System.Data.Common;
using System.Data.Entity;
using System.Data.Entity.Infrastructure.Interception;
namespace CommandTimeOutBug
{
class Program
{
static void Main(string[] args)
{
DbInterception.Add(new MyInterceptor());
using (var context = new ApplicationDbContext())
{
context.Database.CommandTimeout = 6000;
context.PopulateJobTypeDescendants();
}
}
}
public class ApplicationDbContext : DbContext
{
public void PopulateJobTypeDescendants()
{
Database.ExecuteSqlCommand("PopulateJobTypeDescendants");
}
}
public class MyInterceptor: DbCommandInterceptor
{
public override void NonQueryExecuting(DbCommand command,
DbCommandInterceptionContext<int> interceptionContext)
{
command.CommandTimeout = 6000;
base.NonQueryExecuting(command, interceptionContext);
}
public override void ReaderExecuting(DbCommand command,
DbCommandInterceptionContext<DbDataReader> interceptionContext)
{
command.CommandTimeout = 6000;
base.ReaderExecuting(command, interceptionContext);
}
public override void ScalarExecuting(DbCommand command,
DbCommandInterceptionContext<object> interceptionContext)
{
command.CommandTimeout = 6000;
base.ScalarExecuting(command, interceptionContext);
}
}
}
I have created a bug at https://github.com/aspnet/EntityFramework6/issues/24 to track this.
Indeed it looks like a performance issue. Could you please reply me offline at mihaelab at micrososft dot com with server, database names details and full call stack of the exception?
Thanks,
Mihaela
We are having problems with our solrServer client's connection pool running out of connections in no time, even when using a pool of several hundred (we've tried 1024, just for good measure).
From what I've read, the following exception can be caused by not using a singleton HttpSolrServer object. However, see our XML config below, as well:
Caused by: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:232)
at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:199)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:455)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:448)
XML Config:
<solr:solr-server id="solrServer" url="http://solr.url.domain/"/>
<solr:repositories base-package="de.ourpackage.data.solr" multicore-support="true"/>
At this point, we are at a loss. We are running a web application on a tomcat7. Whenever a user requests a new website, we send one or more request to the Solr Server, requesting whatever we need, which are usually single entries or page of 20 (using Spring Data).
As for the rest of our implementation, we are using an abstract SolrOperationsrepository class, which is extended by each of our repositories (one repository for each core).
The following is how we set our solrServer. I suspect we are doing something fundamentally wrong here, which is why our connections are overflowing. According to the logs, they are always being returned into the pool, btw.
private SolrOperations solrOperations;
#SuppressWarnings("unchecked")
public final Class<T> getEntityClass() {
return (Class<T>)((ParameterizedType)getClass().getGenericSuperclass()).getActualTypeArguments()[0];
}
public final SolrOperations getSolrOperations() {
/*HttpSolrServer solrServer = (HttpSolrServer)solrOperations.getSolrServer();
solrServer.getHttpClient().getConnectionManager().closeIdleConnections(500, TimeUnit.MILLISECONDS);*/
logger.info("solrOperations: " + solrOperations);
return solrOperations;
}
#Autowired
public final void setSolrServer(SolrServer solrServer) {
try {
String core = SolrServerUtils.resolveSolrCoreName(getEntityClass());
SolrTemplate template = templateHolder.get(core);
/*solrServer.setConnectionTimeout(500);
solrServer.setMaxTotalConnections(2048);
solrServer.setDefaultMaxConnectionsPerHost(2048);
solrServer.getHttpClient().getConnectionManager().closeIdleConnections(500, TimeUnit.MILLISECONDS);*/
if ( template == null ) {
template = new SolrTemplate(new MulticoreSolrServerFactory(solrServer));
template.setSolrCore(core);
template.afterPropertiesSet();
logger.debug("Creating new SolrTemplate for core '" + core + "'");
templateHolder.put(core, template);
}
logger.debug("setting SolrServer " + template);
this.solrOperations = template;
} catch (Exception e) {
logger.error("cannot set solrServer...", e);
}
}
The code that is commented out has been mostly used for testing purposes. I also read somewhere else that you cannot manipulate the solrServer object on-the-fly. Which begs the question, how do I set a timeout/poolsize in the XML config?
The implementation of a repository looks like this:
#Repository(value="stellenanzeigenSolrRepository")
public class StellenanzeigenSolrRepositoryImpl extends SolrOperationsRepository<Stellenanzeige> implements StellenanzeigenSolrRepositoryCustom {
...
public Query createQuery(Criteria criteria, Sort sort, Pageable pageable) {
Query resultQuery = new SimpleQuery(criteria);
if ( pageable != null ) resultQuery.setPageRequest(pageable);
if ( sort != null ) resultQuery.addSort(sort);
return resultQuery;
}
public Page<Stellenanzeige> findBySearchtext(String searchtext, Pageable pageable) {
Criteria searchtextCriteria = createSearchtextCriteria(searchtext);
Query query = createQuery(searchtextCriteria, null, pageable);
return getSolrOperations().queryForPage(query, getEntityClass());
}
...
}
Can any of you point to mistakes that we've made, that could possibly lead to this issue? Like I said, we are at a loss. Thanks in advance, and I will, of course update the question as we make progress or you request more information.
The MulticoreServerFactory always returns an object of HttpClient, that only ever allows 2 concurrent connections to the same host, thus causing the above problem.
This seems to be a bug with spring-data-solr that can be worked around by creating a custom factory and overriding a few methods.
Edit: The clone method in MultiCoreSolrServerFactory is broken. This hasn't been corrected yet. As some of my colleagues have run into this issue recently, I will post a workaround here - create your own class and override one method.
public class CustomMulticoreSolrServerFactory extends MulticoreSolrServerFactory {
public CustomMulticoreSolrServerFactory(final SolrServer solrServer) {
super(solrServer);
}
#Override
protected SolrServer createServerForCore(final SolrServer reference, final String core) {
// There is a bug in the original SolrServerUtils.cloneHttpSolrServer()
// method
// that doesn't clone the ConnectionManager and always returns the
// default
// PoolingClientConnectionManager with a maximum of 2 connections per
// host
if (StringUtils.hasText(core) && reference instanceof HttpSolrServer) {
HttpClient client = ((HttpSolrServer) reference).getHttpClient();
String baseURL = ((HttpSolrServer) reference).getBaseURL();
baseURL = SolrServerUtils.appendCoreToBaseUrl(baseURL, core);
return new HttpSolrServer(baseURL, client);
}
return reference;
}
}