gzip compression not working in Solr 5.1 - solr

Question

I'm trying to apply gzip compression in Solr 5.1. I understand that Running Solr on Tomcat is no longer supported from Solr 5.0, so I've tried to implement it in Solr.
I've downloaded jetty-servlets-9.3.0.RC0.jar and placed it in my webapp\WEB-INF folder, and have added the following in webapp\WEB-INF\web.xml:
<filter>
<filter-name>GzipFilter</filter-name>
<filter-class>org.eclipse.jetty.servlets.GzipFilter</filter-class>
<init-param>
<param-name>methods</param-name>
<param-value>GET,POST</param-value>
<param-name>mimeTypes</param-name>
<param-value>text/html,text/plain,text/xml,text/json,text/javascript,text/css,application/xhtml+xml,application/javascript,image/svg+xml,application/json,application/xml; charset=UTF-8</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>GzipFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
However, when I start Solr and check the browser, there's no gzip compression, and I only get the following at the Response Headers output:
Content-Type:text/plain;charset=UTF-8
Transfer-Encoding:chunked
Is there anything which I configure wrongly or might have missed out? I'm also running zookeeper-3.4.6.

Answer 1

Download a kosher version of Jetty that closely matches what is currently in Solr: http://www.eclipse.org/jetty/download.html
Extract that .zip to a location you will discard.
Copy out these files to your copy of jetty in Solr (located in path-to-solr/server/):
modules/gzip.mod
etc/gzip.xml
Modify modules/gzip.mod:
#
# GZIP module
# Applies GzipHandler to entire server
#
[depend]
server
[xml]
etc/jetty-gzip.xml
[ini-template]
## Minimum content length after which gzip is enabled
jetty.gzip.minGzipSize=2048
## Check whether a file with *.gz extension exists
jetty.gzip.checkGzExists=false
## Gzip compression level (-1 for default)
jetty.gzip.compressionLevel=-1
## User agents for which gzip is disabled
jetty.gzip.excludedUserAgent=.*MSIE.6\.0.*
Modify etc/gzip.xml:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<!-- =============================================================== -->
<!-- Mixin the GZIP Handler -->
<!-- This applies the GZIP Handler to the entire server -->
<!-- If a GZIP handler is required for an individual context, then -->
<!-- use a context XML (see test.xml example in distribution) -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<Call name="insertHandler">
<Arg>
<New id="GzipHandler" class="org.eclipse.jetty.server.handler.gzip.GzipHandler">
<Set name="minGzipSize">
<Property name="jetty.gzip.minGzipSize" deprecated="gzip.minGzipSize" default="2048"/>
</Set>
<Set name="checkGzExists">
<Property name="jetty.gzip.checkGzExists" deprecated="gzip.checkGzExists" default="false"/>
</Set>
<Set name="compressionLevel">
<Property name="jetty.gzip.compressionLevel" deprecated="gzip.compressionLevel" default="-1"/>
</Set>
<Set name="excludedAgentPatterns">
<Array type="String">
<Item>
<Property name="jetty.gzip.excludedUserAgent" deprecated="gzip.excludedUserAgent" default=".*MSIE.6\.0.*"/>
</Item>
</Array>
</Set>
<Set name="includedMethods">
<Array type="String">
<Item>GET</Item>
<Item>POST</Item>
</Array>
</Set>
<Set name="includedPaths"><Array type="String"><Item>/*</Item></Array></Set>
<Set name="excludedPaths"><Array type="String"><Item>*.gz</Item></Array></Set>
<Call name="addIncludedMimeTypes"><Arg><Array type="String">
<Item>text/html</Item>
<Item>text/plain</Item>
<Item>text/xml</Item>
<Item>application/xml</Item><!-- IMPORTANT - DO NOT FORGET THIS LINE -->
<Item>application/xhtml+xml</Item>
<Item>text/css</Item>
<Item>application/javascript</Item>
<Item>image/svg+xml</Item>
</Array></Arg></Call>
<!--
<Call name="addExcludedMimeTypes"><Arg><Array type="String"><Item>some/type</Item></Array></Arg></Call>
-->
</New>
</Arg>
</Call>
</Configure>
Here's the part that should make you cringe a bit.
Modify bin\solr.cmd
...
set "SOLR_JETTY_CONFIG=--module=http,gzip"
...
set "SOLR_JETTY_CONFIG=--module=https,gzip"
...
Notice that --module=http was already there. Just add ",gzip" so it matches the lines above.
I would prefer finding a better way of specifying the gzip module to load, but I don't know how. If you know how, please reply to this answer and tell me how because I hate modifying a script that comes with a product--it's a maintenance nightmare and, well, I think you get the picture.
After this, restart the solr server and gzip should now be enabled--at least for &wt=xml, which is sent back as Content-Type: application/xml.
You can add what you need to the etc/gzip.xml and restart the solr server for it to recognize your changes.
I tested with and without compression of 1000 documents. For me, it was the difference between 3.8 MB and 637 KB.