Restricting IP addresses for Jetty and Solr - solr

I'm setting up Solr using Jetty. I would like to restrict access to only a few IP addresses. It doesn't seem immediately obvious that this can be done using Jetty. Is it possible and if so, how?

Solr 4.2.1 uses Jetty 8.1.8. Jetty 8 (as noted by jonas789) doesn't support .htaccess. Instead, it uses IPAccessHandler, which doesn't have great documentation available. I had to play with it quite a bit to get it work, so I'm posting an updated solution here.
IPAccessHandler manages a blacklist and a whitelist, accepts arbitrary ranges of IPs, and supports attaching specific URI paths to each white/black -list entry. IPAccessHandler also subclasses HandlerWrapper, which turns out to be important.
The solr app still lives in a WebAppContext (as in Lyndsay's solution), but a WebAppContext is now governed by a ContextHandler, which resides in a ContextHandlerCollection occupying the first handler slot in the server. To stop requests from the wrong IP from getting to the app, we need to wrap it inside an IPAccessHandler somewhere along that path. IPAccessHandler behaves oddly if it's in the wrong spot: I tried inserting it before the context handlers and it gave 403 Forbidden to the wrong machines, threw NullPointerException tantrums with no additional error messages, all sorts of nonsense. I finally got it to work by wrapping the ContextHandlerCollection itself, at the server level.
Go to etc/jetty.xml and scroll to the handlers section. Then wrap the existing ContextHandlerCollection item as follows:
<!-- =========================================================== -->
<!-- Set handler Collection Structure -->
<!-- =========================================================== -->
<Set name="handler">
<New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<Set name="handlers">
<Array type="org.eclipse.jetty.server.Handler">
<Item>
<!-- here begins the new stuff -->
<New class="org.eclipse.jetty.server.handler.IPAccessHandler">
<Call name="addWhite">
<Arg>xxx.xxx.xxx.xxx</Arg>
</Call>
<Set name="handler">
<!-- here's where you put what was there before: -->
<New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection"/>
</Set>
</New>
<!-- here ends the new stuff -->
</Item>
<Item>
<New id="DefaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler"/>
</Item>
<Item>
<New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler"/>
</Item>
</Array>
</Set>
</New>
</Set>
Resources:
http://comments.gmane.org/gmane.comp.java.jetty.support/6066
http://wiki.eclipse.org/Jetty#Configuration_Reference
http://wiki.eclipse.org/Jetty/Reference/jetty.xml_syntax
http://download.eclipse.org/jetty/stable-8/apidocs/org/eclipse/jetty/server/handler/IPAccessHandler.html

I found the solution.
Firstly, extract the contents of solr.war in the example/webapps folder.
Then create a file called .htaccess and place it in the example/webapps/solr folder (the one you just extracted) containing the following:
<Limit>
satisfy all
order deny,allow
deny from all
allow from xxx.xxx.xxx.xxx
</Limit>
In example/etc/ edit the jetty.xml file and comment out the org.mortbay.jetty.deployer.WebAppDeployer part. Then finally create a folder in example/ called contexts (if one does not yet exist) and add a file called solr.xml to it containing:
<Configure id="solr" class="org.mortbay.jetty.webapp.WebAppContext">
<Set name="resourceBase"><SystemProperty name="jetty.home" default="."/>/webapps/solr</Set>
<Set name="contextPath">/solr</Set>
<Call name="setSecurityHandler">
<Arg>
<New class="org.mortbay.jetty.security.HTAccessHandler">
<Set name="protegee">
<Ref id="solr"/>
</Set>
</New>
</Arg>
</Call>
</Configure>
Then start up your new secure solr!

Related

How to get file uploaded time in wso2 esb sequence

I am using WSO2 inbound endpoint to fetch a file from an FTP server. And I know how to get the file name back. Now my question is how to get the file uploaded time back (or the last modified time)?
This is the code to get the file name.
<property expression="get-property('transport', 'FILE_NAME')" name="ftp.var.filename"
xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ns2="http://org.apache.synapse/xsd"/>
I think there should be a similar code to get the timestamp of the file.
With the following property, you will be able to get the last modified time of the file polled from the inbound endpoint.
`<property expression="get-property('transport', 'LAST_MODIFIED')" name="ftp.var.last.modified.time" xmlns:ns="http://org.apache.synapse/xsd"/>`
Add this to the relevant sequence to further process and following is a sample sequence in which the file name and the last modified time is logged.
<?xml version="1.0" encoding="UTF-8"?>
<sequence name="fileSequence" onError="fault" xmlns="http://ws.apache.org/ns/synapse">
<log level="custom">
<property expression="get-property('transport', 'FILE_NAME')"
name="ftp.var.filename" xmlns:ns="http://org.apache.synapse/xsd"/>
<property
expression="get-property('transport', 'LAST_MODIFIED')"
name="ftp.var.last.modified.time" xmlns:ns="http://org.apache.synapse/xsd"/>
</log>
</sequence>
Please check whether this meets your requirement and please refer [1] to further clarify this.
[1]-https://github.com/wso2/wso2-synapse/blob/master/modules/transports/core/vfs/src/main/java/org/apache/synapse/transport/vfs/VFSTransportListener.java#L767

gzip compression not working in Solr 5.1

I'm trying to apply gzip compression in Solr 5.1. I understand that Running Solr on Tomcat is no longer supported from Solr 5.0, so I've tried to implement it in Solr.
I've downloaded jetty-servlets-9.3.0.RC0.jar and placed it in my webapp\WEB-INF folder, and have added the following in webapp\WEB-INF\web.xml:
<filter>
<filter-name>GzipFilter</filter-name>
<filter-class>org.eclipse.jetty.servlets.GzipFilter</filter-class>
<init-param>
<param-name>methods</param-name>
<param-value>GET,POST</param-value>
<param-name>mimeTypes</param-name>
<param-value>text/html,text/plain,text/xml,text/json,text/javascript,text/css,application/xhtml+xml,application/javascript,image/svg+xml,application/json,application/xml; charset=UTF-8</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>GzipFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
However, when I start Solr and check the browser, there's no gzip compression, and I only get the following at the Response Headers output:
Content-Type:text/plain;charset=UTF-8
Transfer-Encoding:chunked
Is there anything which I configure wrongly or might have missed out? I'm also running zookeeper-3.4.6.
Download a kosher version of Jetty that closely matches what is currently in Solr: http://www.eclipse.org/jetty/download.html
Extract that .zip to a location you will discard.
Copy out these files to your copy of jetty in Solr (located in path-to-solr/server/):
modules/gzip.mod
etc/gzip.xml
Modify modules/gzip.mod:
#
# GZIP module
# Applies GzipHandler to entire server
#
[depend]
server
[xml]
etc/jetty-gzip.xml
[ini-template]
## Minimum content length after which gzip is enabled
jetty.gzip.minGzipSize=2048
## Check whether a file with *.gz extension exists
jetty.gzip.checkGzExists=false
## Gzip compression level (-1 for default)
jetty.gzip.compressionLevel=-1
## User agents for which gzip is disabled
jetty.gzip.excludedUserAgent=.*MSIE.6\.0.*
Modify etc/gzip.xml:
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "http://www.eclipse.org/jetty/configure_9_3.dtd">
<!-- =============================================================== -->
<!-- Mixin the GZIP Handler -->
<!-- This applies the GZIP Handler to the entire server -->
<!-- If a GZIP handler is required for an individual context, then -->
<!-- use a context XML (see test.xml example in distribution) -->
<!-- =============================================================== -->
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<Call name="insertHandler">
<Arg>
<New id="GzipHandler" class="org.eclipse.jetty.server.handler.gzip.GzipHandler">
<Set name="minGzipSize">
<Property name="jetty.gzip.minGzipSize" deprecated="gzip.minGzipSize" default="2048"/>
</Set>
<Set name="checkGzExists">
<Property name="jetty.gzip.checkGzExists" deprecated="gzip.checkGzExists" default="false"/>
</Set>
<Set name="compressionLevel">
<Property name="jetty.gzip.compressionLevel" deprecated="gzip.compressionLevel" default="-1"/>
</Set>
<Set name="excludedAgentPatterns">
<Array type="String">
<Item>
<Property name="jetty.gzip.excludedUserAgent" deprecated="gzip.excludedUserAgent" default=".*MSIE.6\.0.*"/>
</Item>
</Array>
</Set>
<Set name="includedMethods">
<Array type="String">
<Item>GET</Item>
<Item>POST</Item>
</Array>
</Set>
<Set name="includedPaths"><Array type="String"><Item>/*</Item></Array></Set>
<Set name="excludedPaths"><Array type="String"><Item>*.gz</Item></Array></Set>
<Call name="addIncludedMimeTypes"><Arg><Array type="String">
<Item>text/html</Item>
<Item>text/plain</Item>
<Item>text/xml</Item>
<Item>application/xml</Item><!-- IMPORTANT - DO NOT FORGET THIS LINE -->
<Item>application/xhtml+xml</Item>
<Item>text/css</Item>
<Item>application/javascript</Item>
<Item>image/svg+xml</Item>
</Array></Arg></Call>
<!--
<Call name="addExcludedMimeTypes"><Arg><Array type="String"><Item>some/type</Item></Array></Arg></Call>
-->
</New>
</Arg>
</Call>
</Configure>
Here's the part that should make you cringe a bit.
Modify bin\solr.cmd
...
set "SOLR_JETTY_CONFIG=--module=http,gzip"
...
set "SOLR_JETTY_CONFIG=--module=https,gzip"
...
Notice that --module=http was already there. Just add ",gzip" so it matches the lines above.
I would prefer finding a better way of specifying the gzip module to load, but I don't know how. If you know how, please reply to this answer and tell me how because I hate modifying a script that comes with a product--it's a maintenance nightmare and, well, I think you get the picture.
After this, restart the solr server and gzip should now be enabled--at least for &wt=xml, which is sent back as Content-Type: application/xml.
You can add what you need to the etc/gzip.xml and restart the solr server for it to recognize your changes.
I tested with and without compression of 1000 documents. For me, it was the difference between 3.8 MB and 637 KB.

Spring data #transactional not rolling back with SQL Server and after runtimeexception

I've enabled my spring application to use transactions and annotated my service method accordingly but the changes to my DB persist when a RuntimeException is thrown.
My Spring configuration looks like this:
<!-- Data Source. -->
<jee:jndi-lookup id="dataSource" jndi-name="java:/jdbc/BeheermoduleDS"/>
<!-- JPA Entity Manager. -->
<jee:jndi-lookup id="entityManagerFactory" jndi-name="java:/jpa/BeheermoduleDS"/>
<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<tx:annotation-driven transaction-manager="txManager" />
My datasource configuration in my jboss' configuration file looks like this:
<datasource jta="true" jndi-name="java:/jdbc/BeheermoduleDS" pool-name="BeheermoduleDS" enabled="true" use-java-context="true" use-ccm="true">
<connection-url>jdbc:sqlserver://localhost:1433;databaseName=Gebruikers;</connection-url>
<driver>sqljdbc</driver>
<security>
<user-name>jboss</user-name>
<password>*****</password>
</security>
</datasource>
My Service method looks like this:
#Transactional
public void authorise(Gebruiker user) {
user.setStatus(GebruikerStatus.Actief.name());
gebruikerRepo.save(user);
if (true) {
throw new RuntimeException("Exception happened just like that");
}
// does more stuff here that is never reached
}
My repository extends a spring data repository and looks like this:
public interface GebruikerRepository extends PagingAndSortingRepository<Gebruiker, Long>, QueryDslPredicateExecutor<Gebruiker> {
}
The transaction is thrown and caught by a controller which just shows a message to the user that an exception occurred. When I check my SQL Server DB, the change made to the user status have been commited.
Weren't they supposed to have been rolled back with the RuntimeException?
After turning debug on for org.springframework.transaction.interceptor I saw that no transactions are being started for my service method, but they are for a bunch of JpaRepository methods.
Also, this is how my persistence.xml looks like:
<persistence-unit name="BeheermodulePU" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<non-jta-data-source>java:/jdbc/BeheermoduleDS</non-jta-data-source>
Judging from the symptoms you describe you are scanning for the same classes twice. You probably have the same <context:component-scan /> in both the configuration of the ContextLoaderListener and DispatcherServlet.
You want the ContextLoaderListener to scan for everything but #Controller and the DispatcherServlet only for #Controllers. Leading to something like this.
For the ContextLoaderListener
<!-- Load everything except #Controllers -->
<context:component-scan base-package="com.myapp">
<context:exclude-filter expression="org.springframework.stereotype.Controller" type="annotation"/>
</context:component-scan>
For the DispatcherServlet
<!-- Load everything except #Controllers -->
<context:component-scan base-package="com.myapp" use-default-filters="false">
<context:include-filter expression="org.springframework.stereotype.Controller" type="annotation"/>
</context:component-scan>
See also #Service are constructed twice for another sample and broader explanation.

JPA does not write to table

I have the following JPA code, with all the values checked (ticket contains a valid bean, it ends without exception, etc.) It is executed, it does not throw any exceptions, yet in the end no data is written into the table.
I tried also retrieving a bean from the table, it also "works" (it is empty, so no data is returned).
The setup is
JBoss 6.1 Final
SQLServer 2008 Express (driver SQL JDBC 3 from MS)
The persistence code:
public String saveTicket() {
System.out.println("Controller saveTicket() ");
EntityManagerFactory factory = Persistence.createEntityManagerFactory("GesMan"); /* I know it would be better to share a single instance of factory, this is just for testing */
EntityManager entityMan = factory.createEntityManager();
entityMan.persist(this.ticket);
entityMan.close();
}
The persistence unit is
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="GesMan" transaction-type="JTA">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>java:/GesManDS</jta-data-source>
<class>es.caib.gesma.gesman.Ticket</class>
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.SQLServerDialect"/>
<property name="hibernate.transaction.manager_lookup_class"
value="org.hibernate.transaction.JBossTransactionManagerLookup"/>
<property name="hibernate.show_sql" value="true"/>
</properties>
</persistence-unit>
</persistence>
The datasource
<datasources>
<local-tx-datasource>
<jndi-name>GesManDS</jndi-name>
<connection-url>jdbc:sqlserver://spsigeswnt14.caib.es:1433;DatabaseName=TEST_GESMAN</connection-url>
<driver-class>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class>
<user-name>thisis</user-name>
<password>notthepassword</password>
<check-valid-connection-sql>SELECT * FROM dbo.Ticket</check-valid-connection-sql>
<metadata>
<type-mapping>MS SQLSERVER</type-mapping>
</metadata>
</local-tx-datasource>
</datasources>
call entityMan.flush() or transaction.commit() befor closing it, otherwise it will discard all changes queued on close.
In the end it looks like I was using the wrong approach.... In JBoss you can`t (better said, I could not get to) access JPA directly (as you would do in JSE).
I ended creating an EJB (with transactions) and passing all JPA logic there.
PS: Of course, if I am wrong please tell me (now it is more of an academic issue, but still I want to know)

iBatis - Why is sqlMapConfig.xml unable to find the sql maps defined in it?

I have a sqlMapConfig.xml that has three SQLMaps defined in it.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sqlMapConfig
PUBLIC "-//ibatis.apache.org//DTD SQL Map Config 2.0//EN"
"http://ibatis.apache.org/dtd/sql-map-config-2.dtd">
<sqlMapConfig>
<!-- Statement namespaces are required for Ibator -->
<settings enhancementEnabled="true" useStatementNamespaces="true"/>
<!-- Setup the transaction manager and data source that are
appropriate for your environment
-->
<transactionManager type="JDBC">
<dataSource type="SIMPLE" >
<property name="JDBC.Driver"
value="com.mysql.jdbc.Driver"/>
<property name="JDBC.ConnectionURL"
value="jdbc:mysql://localhost:3306/sug"/>
<property name="JDBC.Username"
value="root"/>
<property name="JDBC.Password"
value="admin"/>
</dataSource>
</transactionManager>
<!-- SQL Map XML files should be listed here -->
<sqlMap resource="com/tatakelabs/dbmaps/categories_SqlMap.xml" />
<sqlMap resource="com/tatakelabs/dbmaps/pro_SqlMap.xml" />
<sqlMap resource="com/tatakelabs/dbmaps/pro_category_SqlMap.xml" />
</sqlMapConfig>
I get a runtime error - Cause: java.io.IOException: Could not find resource com/tatakelabs/dbmaps/categories_SqlMap.xml
categories_SqlMap.xml is present in that location. I tried changing the location of the map xml, but that did not help. sqlMapConfig.xml validates against the DTD. categories_SqlMap.xml also validates against the right DTD. I am at my wits end trying to figure out why it can't find the resource. The sqlMap files are generated by iBator.
This was happening because the sqlmap file location was not getting copied to target. Added a copy goal and that fixed it.
I had the same problem. It appears the problem lies with the location of the config file. Thus, its in relation of the project resource structure.
I moved the config file in the same package as the mapper classes and it worked. In this case try moving all the resources to this package and update the resource attributes to:
<sqlMap resource="categories_SqlMap.xml" />
<sqlMap resource="pro_SqlMap.xml" />
<sqlMap resource="pro_category_SqlMap.xml" />
Solved it.
I moved the xml file to where the Pojo was located and provided the path as follows:
<sqlMap resource="com/heena/ibatis/model/jsr/jsr.xml" />
And it worked.
place it ...src\java\abc.xml under the Source Packages directory.
If you are using Spring, you can use a SqlMapClientFactoryBean specifying property "mappingLocations". In this property you can specify a generic path, such as "com/tatakelabs/dbmaps/*_SqlMap.xml" or with a variable such as ${mapfiles}, that will be resolved by Spring as an array of file names. This lets you omit sqlMap element in sqlMapConfig. This technique will run with iBatis 2.3.4 on. However sql-map-config-2.dtd is also contained inside iBatis.jar, so you can experience some parsing errors, as /com/ibatis/sqlmap/engine/builder/xml/sql-map-config-2.dtd may have a bug. In this case you may want to replace it inside the jar with the one from the URL:
http://ibatis.apache.org/dtd/sql-map-config-2.dtd.

Resources