Http Parser full exception in jetty 8 solr (v 4.5) server - solr

Question

I have a standalone solr(4.5) server which is running on top of jetty 8 .I have an application server on apache tomcat server which is a customer facing node.
Application server connects to standalone solr server to fetch the search result.I send POST request as the query to SOLR is huge but I get the below WARN message on jetty solr server:
WARN org.eclipse.jetty.http.HttpParser â HttpParser Full for server1:8983 <->server 2:99988
On the tomcat application server I get the below error message :
SEVERE: Servlet.service() for servlet [DispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.apache.solr.common.SolrException: No live SolrServers available to handle this request] with root cause
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | java.net.SocketException: Broken pipe
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at java.net.SocketOutputStream.socketWrite0(Native Method)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:147)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.io.AbstractSessionOutputBuffer.writeLine(AbstractSessionOutputBuffer.java:246)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.io.HttpRequestWriter.writeHeadLine(HttpRequestWriter.java:56)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.io.HttpRequestWriter.writeHeadLine(HttpRequestWriter.java:44)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.io.AbstractMessageWriter.write(AbstractMessageWriter.java:90)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.AbstractHttpClientConnection.sendRequestHeader(AbstractHttpClientConnection.java:258)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.conn.DefaultClientConnection.sendRequestHeader(DefaultClientConnection.java:271)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestHeader(ManagedClientConnectionImpl.java:203)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:221)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
INFO | jvm 1 | main | 2017/03/30 03:30:46.402 | at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
INFO | jvm 1 | main | 2017/03/30 03:30:46.403 | at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:467)
INFO | jvm 1 | main | 2017/03/30 03:30:46.403 | at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
INFO | jvm 1 | main | 2017/03/30 03:30:46.403 | at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
I have tried increasing the "requestHeaderSize" , and "maxFormContentSize" in jetty.xml but no luck.

Answer 1

First: Jetty 8 is EOL (End of Life), consider upgrading to something supported and stable.
The HttpParser Full is from excessively large request entities (which is the Request URI line and Request Headers).
If this error is from the server side, then its the Request headers.
See https://stackoverflow.com/a/16015332/775715 for advice on configuring it properly on the server side. (Hint: its a Connector setting. So if you have 2 connectors, you have 2 configurations to change)
maxFormContentSize is for the request body content on POST requests, and has no effect on request uri or request headers. The HttpParser Full is not going to be triggered for excessive request body content, so ignore this aspect of the problem, focus on the request URI and request headers.
If this error is from the client side, then its the Response headers.
Pay attention to what's generating those request URI and request headers, as that's the culprit! The default setting is specifically designed for maximum compatibility on the general internet, if you have to increase the default settings then you either have something seriously wrong with your request URI or request headers, or you are using the API improperly (such as sending documents via POST/GET uri strings, and not request body content)