SymmetircDS: Extract Frequency by Channel - symmetricds

I'm trying to limit the extract frequency and found two ways in user guide (https://www.symmetricds.org/doc/3.9/html/user-guide.html#_extract_frequency_by_channel). None of them are working.
1) By setting extract_period_millis.
update sym_channel set extract_period_millis = 30000, last_update_time=CURRENT_TIMESTAMP where channel_id='channel_bps2swd'
After that, I was receiving the following error message on console:
[server] - SymmetricServlet - Error while processing GET request for node: 0001 at 127.0.0.1 with path: /server/pull
org.jumpmind.db.sql.SqlException: Failed to execute sql: null
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:302)
at org.jumpmind.db.sql.JdbcSqlReadCursor.<init>(JdbcSqlReadCursor.java:120)
at org.jumpmind.db.sql.JdbcSqlTemplate.queryForCursor(JdbcSqlTemplate.java:140)
at org.jumpmind.db.sql.AbstractSqlTemplate.query(AbstractSqlTemplate.java:199)
at org.jumpmind.db.sql.AbstractSqlTemplate.query(AbstractSqlTemplate.java:195)
at org.jumpmind.db.sql.AbstractSqlTemplate.query(AbstractSqlTemplate.java:185)
at org.jumpmind.db.sql.AbstractSqlTemplate.query(AbstractSqlTemplate.java:121)
at org.jumpmind.symmetric.service.impl.ConfigurationService.getNodeChannels(ConfigurationService.java:436)
at org.jumpmind.symmetric.service.impl.ConfigurationService.getSuspendIgnoreChannelLists(ConfigurationService.java:531)
at org.jumpmind.symmetric.web.PullUriHandler.handlePull(PullUriHandler.java:112)
at org.jumpmind.symmetric.web.PullUriHandler.handleWithCompression(PullUriHandler.java:100)
at org.jumpmind.symmetric.web.AbstractCompressionUriHandler.handle(AbstractCompressionUriHandler.java:84)
at org.jumpmind.symmetric.web.SymmetricServlet.service(SymmetricServlet.java:114)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:833)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
at org.jumpmind.symmetric.web.HttpMethodFilter.doFilter(HttpMethodFilter.java:62)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:561)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:334)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:104)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:243)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:679)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:597)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Parameter arg '0001' type: -2147483648 caused exception: Invalid parameter index 1.
at org.jumpmind.db.sql.JdbcSqlTemplate.setValues(JdbcSqlTemplate.java:1027)
at org.jumpmind.db.sql.JdbcSqlReadCursor.<init>(JdbcSqlReadCursor.java:92)
... 44 more
Caused by: java.sql.SQLException: Invalid parameter index 1.
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.getParameter(JtdsPreparedStatement.java:543)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setParameter(JtdsPreparedStatement.java:612)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setString(JtdsPreparedStatement.java:892)
at org.apache.commons.dbcp.DelegatingPreparedStatement.setString(DelegatingPreparedStatement.java:135)
at org.apache.commons.dbcp.DelegatingPreparedStatement.setString(DelegatingPreparedStatement.java:135)
at org.springframework.jdbc.core.StatementCreatorUtils.setValue(StatementCreatorUtils.java:453)
at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValueInternal(StatementCreatorUtils.java:241)
at org.springframework.jdbc.core.StatementCreatorUtils.setParameterValue(StatementCreatorUtils.java:172)
at org.jumpmind.db.sql.JdbcSqlTemplate.setValues(JdbcSqlTemplate.java:1023)
... 45 more
[server] - NodeConcurrencyInterceptor - Error building response headers
I had to set back extract_period_millis = 0 to solve this error message.
Note: Table SYM_NODE_CHANNEL_CTL is empty
2) By setting start/endtime to on table SYM_NODE_GROUP_CHANNEL_WND for associated channel.
It didn't have any effects. Changes were synced shortly after they have occured.
System:
Symmetric 3.9.12, Windows 10 x64, mssql
Any solutions to set the sync frequency?

If your just looking to change the sync frequency you could use the following parameters as they are both defaulted to 1 min (60000ms).
job.pull.period.time.ms
job.push.period.time.ms

In class
org.jumpmind.symmetric.service.impl.ConfigurationServiceSqlMap.java
add this
putSql("selectNodeChannelControlLastExtractTimeSql" ,"select channel_id,last_extract_time from sym_node_channel_ctl where node_id=?");

Related

Exporting data using solr streaming expressions is randomly not working

I am using solr 7.4 version. I am trying to export data using streaming expressions.
Following is the simple curl query that i am using. I am indexing my data and testing the following query.
curl --data-urlencode 'expr=search(collection1,
q="text:solar",
fl="id",
sort="id asc",
qt="/export")' http://localhost:8983/solr/collection1/stream
when i run above query with initial few documents indexed, it is working fine. After indexing few thousand documents (around , i am getting the following error for the same query.
EXCEPTION":"java.util.concurrent.ExecutionException: java.io.IOException: --> http://localhost:8983/solr/collection1/: An exception has occurred on the server, refer to server log for details."
I've looked into my solr logs and the following is the error info
java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> http://localhost:8983/solr/collection1/: An exception has occurred on the server, refer to server log for details.
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397)
at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83)
at org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539)
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181)
at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209)
at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325)
at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120)
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: --> http://localhost:8983/solr/collection1/: An exception has occurred on the server, refer to server log for details.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394)
... 49 more
Caused by: java.io.IOException: --> http://localhost:8983/solr/collection1/: An exception has occurred on the server, refer to server log for details.
at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:225)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507)
at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.io.IOException: JSONTupleStream: expected OBJECT_START but got EOF
at org.apache.solr.client.solrj.io.stream.JSONTupleStream.expect(JSONTupleStream.java:99)
at org.apache.solr.client.solrj.io.stream.JSONTupleStream.advanceToDocs(JSONTupleStream.java:179)
at org.apache.solr.client.solrj.io.stream.JSONTupleStream.next(JSONTupleStream.java:77)
at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:194)
... 8 more
Is it because new documents have something unusual in their content? Even if that is the case, i am requesting only one field which is id and there is nothing unusual about it.
Also, to mention, if i give q="*:*" it is working fine (which is kind of weird)
Any idea what what is issue here ?

Solr-cloud - no servers hosting shard: shard1

I have created 6 collections, each collection is having 3 shards and 2 replica (solr version 5.5.0). For a few days my setup was working fine. But after some days when I restated solr started getting error.
Getting error
PKIAuthenticationPlugin Decryption failed
Server 1:
81245736 ERROR (qtp668849042-17075) [ ] o.a.s.s.PKIAuthenticationPlugin Decryption failed , key must be wrong
java.security.InvalidKeyException: No installed provider supports this key: (null)
at javax.crypto.Cipher.chooseProvider(Cipher.java:893)
at javax.crypto.Cipher.init(Cipher.java:1249)
at javax.crypto.Cipher.init(Cipher.java:1186)
at org.apache.solr.util.CryptoKeys.decryptRSA(CryptoKeys.java:277)
at org.apache.solr.security.PKIAuthenticationPlugin.parseCipher(PKIAuthenticationPlugin.java:172)
at org.apache.solr.security.PKIAuthenticationPlugin.decipherHeader(PKIAuthenticationPlugin.java:159)
at org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:118)
at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:264)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
81245736 WARN (qtp668849042-17075) [ ] o.a.s.s.PKIAuthenticationPlugin Failed to decrypt header, trying after refreshing the key
81245747 ERROR (qtp668849042-17075) [ ] o.a.s.s.PKIAuthenticationPlugin Exception trying to get public key from : http://SorServer2:8983/solr
org.noggit.JSONParser$ParseException: JSON Parse Error: char=<,position=0 BEFORE='<' AFTER='html> <head> <meta http-equiv="Content-'
at org.noggit.JSONParser.err(JSONParser.java:356)
at org.noggit.JSONParser.handleNonDoubleQuoteString(JSONParser.java:712)
at org.noggit.JSONParser.next(JSONParser.java:886)
at org.noggit.JSONParser.nextEvent(JSONParser.java:930)
at org.noggit.ObjectBuilder.<init>(ObjectBuilder.java:44)
at org.noggit.ObjectBuilder.getVal(ObjectBuilder.java:37)
at org.apache.solr.common.util.Utils.fromJSON(Utils.java:98)
at org.apache.solr.security.PKIAuthenticationPlugin.getRemotePublicKey(PKIAuthenticationPlugin.java:202)
at
at java.lang.Thread.run(Thread.java:745)
81245748 ERROR (qtp668849042-17075) [ ] o.a.s.s.PKIAuthenticationPlugin Decryption failed , key must be wrong
This issue gets resolved some time when I restart all solr node manually. But after some time I get the same error.

org.hibernate.jdbc.AbstractBatcher : exception clearing maxRows/queryTimeout

I am getting the below exception when my application tries to query MS SQL server for a functionality.
Referring to other suggestions on this forum, I have already updated my tomcat configuration (context.xml) for the impacted resource to values shown below.
But it didn't help.
An observation on this, if the functionality is repeatedly invoked, then it works in 1 out of 4/5 attempts.
Also, when it works, the specific item it fails other time is executed within 40 seconds. (Usually 36-39 seconds)
However, if at all it touches the 40 seconds mark, the exception is thrown and functionality fails.
Resource Config
<Resource
minEvictableIdleTimeMillis="60000"
maxActive="100"
auth="Container"
testOnBorrow="true"
password="*******"
testOnReturn="false"
testWhileIdle="true"
url="jdbc:sqlserver://*****"
maxWait="10000"
maxIdle="30"
name="jdbc/CiteMod"
username="********"
driverClassName="com.microsoft.sqlserver.jdbc.SQLServerDriver"
type="javax.sql.DataSource"
validationQuery="Select 1"
timeBetweenEvictionRunsMillis="1800000">
Exception Log
15:25:15.259(07/11) WARN org.hibernate.jdbc.AbstractBatcher : exception clearing maxRows/queryTimeout
com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:388)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.checkClosed(SQLServerStatement.java:978)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getMaxRows(SQLServerStatement.java:1011)
at org.apache.tomcat.dbcp.dbcp.DelegatingStatement.getMaxRows(DelegatingStatement.java:237)
at org.apache.tomcat.dbcp.dbcp.DelegatingStatement.getMaxRows(DelegatingStatement.java:237)
at org.hibernate.jdbc.AbstractBatcher.closeQueryStatement(AbstractBatcher.java:298)
at org.hibernate.jdbc.AbstractBatcher.closeQueryStatement(AbstractBatcher.java:234)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1863)
at org.hibernate.loader.Loader.doQuery(Loader.java:718)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:270)
at org.hibernate.loader.Loader.doList(Loader.java:2294)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2172)
at org.hibernate.loader.Loader.list(Loader.java:2167)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:316)
at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1832)
at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165)
at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:179)
at com.lexisnexis.citemod.metadata.service.ExportFilesService.verifyTokenizeResolutionFileName(ExportFilesService.java:594)
at com.lexisnexis.citemod.metadata.service.ExportFilesService.verifyTokenizeResolutionFile(ExportFilesService.java:551)
at com.lexisnexis.citemod.metadata.web.action.export.ExportFilesAction.listVerifyTokenizeResolution(ExportFilesAction.java:496)
at sun.reflect.GeneratedMethodAccessor402.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:269)
at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:170)
at org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.java:146)
at org.apache.struts.chain.commands.servlet.ExecuteAction.execute(ExecuteAction.java:58)
at org.apache.struts.chain.commands.AbstractExecuteAction.execute(AbstractExecuteAction.java:67)
at org.apache.struts.chain.commands.ActionCommandBase.execute(ActionCommandBase.java:51)
at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:191)
at org.apache.commons.chain.generic.LookupCommand.execute(LookupCommand.java:305)
at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:191)
at org.apache.struts.chain.ComposableRequestProcessor.process(ComposableRequestProcessor.java:283)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:462)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
:

Gerrit: Internal Server Error with java.sql.SQLRecoverableException: Closed Connection

From time to time we get Internal Server errors when we push to Gerrit. In the logs we see that the connection the connection is closed before the transaction is completed. (See log below)
We're using an Oracle DB with the following settings:
[database]
type = ORACLE
poolmaxidle = 16
database = db/ReviewDB
port = 1521
instance = infrapd1
hostname = HOSTNAME
connectionpool = true
username = USERNAME
poollimit = 32
dataSourceInterceptorClass = com.googlesource.gerrit.plugins.javamelody.MonitoringDataSourceInterceptor
In the logs we see errors like these when we get an Internal Server error:
[2015-02-13 15:49:51,505] ERROR com.google.gerrit.server.git.ReceiveCommits : Can't insert change/patchset for PROJECT
com.google.gerrit.server.git.InsertException: Error inserting change/patchset
at com.google.gerrit.server.git.ReceiveCommits$1.apply(ReceiveCommits.java:258)
at com.google.gerrit.server.git.ReceiveCommits$1.apply(ReceiveCommits.java:249)
at com.google.common.util.concurrent.Futures$MappingCheckedFuture.mapException(Futures.java:1672)
at com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:85)
at com.google.gerrit.server.git.ReceiveCommits.insertChangesAndPatchSets(ReceiveCommits.java:756)
at com.google.gerrit.server.git.ReceiveCommits.processCommands(ReceiveCommits.java:564)
at com.google.gerrit.server.git.AsyncReceiveCommits$Worker.run(AsyncReceiveCommits.java:90)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at com.google.gerrit.server.util.RequestScopePropagator$5.call(RequestScopePropagator.java:222)
at com.google.gerrit.server.util.RequestScopePropagator$4.call(RequestScopePropagator.java:201)
at com.google.gerrit.server.util.ThreadLocalRequestScopePropagator$1.call(ThreadLocalRequestScopePropagator.java:55)
at com.google.gerrit.server.util.RequestScopePropagator$1.call(RequestScopePropagator.java:98)
at com.google.gerrit.server.util.RequestScopePropagator$2.run(RequestScopePropagator.java:131)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:364)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.ExecutionException: com.google.gwtorm.server.OrmException: Cannot set auto commit mode
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
at com.google.common.util.concurrent.AbstractCheckedFuture.checkedGet(AbstractCheckedFuture.java:78)
... 17 more
Caused by: com.google.gwtorm.server.OrmException: Cannot set auto commit mode
at com.google.gwtorm.jdbc.JdbcSchema.rollback(JdbcSchema.java:80)
at com.google.gerrit.server.change.ChangeInserter.insert(ChangeInserter.java:177)
at com.google.gerrit.server.git.ReceiveCommits$CreateRequest.insertChange(ReceiveCommits.java:1585)
at com.google.gerrit.server.git.ReceiveCommits$CreateRequest.access$800(ReceiveCommits.java:1520)
at com.google.gerrit.server.git.ReceiveCommits$CreateRequest$1.call(ReceiveCommits.java:1551)
at com.google.gerrit.server.git.ReceiveCommits$CreateRequest$1.call(ReceiveCommits.java:1547)
at com.google.gerrit.server.util.RequestScopePropagator$1.call(RequestScopePropagator.java:96)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
at com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:58)
at com.google.gerrit.server.git.ReceiveCommits$CreateRequest.insertChange(ReceiveCommits.java:1546)
at com.google.gerrit.server.git.ReceiveCommits.insertChangesAndPatchSets(ReceiveCommits.java:752)
... 16 more
Caused by: java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.setAutoCommit(PhysicalConnection.java:3790)
at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:371)
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:328)
at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at net.bull.javamelody.JdbcWrapper$ConnectionInvocationHandler.invoke(JdbcWrapper.java:188)
at net.bull.javamelody.JdbcWrapper$DelegatingInvocationHandler.invoke(JdbcWrapper.java:285)
at com.sun.proxy.$Proxy17.setAutoCommit(Unknown Source)
at com.google.gwtorm.jdbc.JdbcSchema.rollback(JdbcSchema.java:78)
... 28 more
Any suggestions on how to resolve this? Any advise on how to improve the DB settings or debug the error? Any pointers are much appreciated.

How to solve Dataconfig error when running full import on solr instance?

I created solr instance and I am trying to run full import but i am getting Data config problem.
Url is:http://localhost:8080/apache-solr-4.0.0/collection1/dataimport?command=full-import&clean=true
5008data-config.xmlfull-importData Config problem: The processing instruction target matching "[xX][mM][lL]" is not allowed.org.apache.solr.handler.dataimport.DataImportHandlerException: Data Config problem: The processing instruction target matching "[xX][mM][lL]" is not allowed.
at org.apache.solr.handler.dataimport.DataImporter.loadDataConfig(DataImporter.java:233)
at org.apache.solr.handler.dataimport.DataImporter.maybeReloadConfiguration(DataImporter.java:131)
at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:167)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:279)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)
at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.xml.sax.SAXParseException; systemId: solrres:/data-config.xml; lineNumber: 2; columnNumber: 6; The processing instruction target matching "[xX][mM][lL]" is not allowed.
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1375)
at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanPIData(XMLScanner.java:662)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanPIData(XMLDocumentFragmentScannerImpl.java:979)
at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanPI(XMLScanner.java:630)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:913)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:607)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:116)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:240)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:300)
at org.apache.solr.handler.dataimport.DataImporter.loadDataConfig(DataImporter.java:224)
... 31 more
500
can any one help me on this?
Thanks in Advance.....
A couple of resources (here and here) seem to indicate that this error can be caused by whitespace preceding '<?xml ...' in the XML being read.

Resources