Tomcat executor with runnable while(true) loop is only run once. Why? - jakarta-mail

I am trying to implement a javax.mail.event.MessageCountListener in Tomcat. When I start the application the contextInitialized method seems to run and the mailbox is read. However, I see the log message "Idling" only once. I would expect that it would idle constantly and invoke the AnalyzerService() when an email is received or deleted.
Update: Found that the idle() method is not returning. It runs untill com.sun.mail.iap.ResponseInputStream.readResponse(ByteArray ba) method where it runs into a while loop where it never gets out.
Am I misusing the idle() method for something I should not do? Is this a bug in com.sun.mail.iap package?
The AnalyzerContextListener.java:
import com.sun.mail.imap.IMAPStore;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import javax.mail.Folder;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.event.MessageCountListener;
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
public class AnalyzerContextListener implements ServletContextListener {
private ExecutorService executorService;
private final String username = "myemail#gmail.com";
private final String password = "mypassword";
private final String mailhost = "imap.gmail.com";
private final String foldername = "INBOX";
#Override
public void contextInitialized(ServletContextEvent sce) {
final ServletContext servletContext = sce.getServletContext();
executorService = Executors.newFixedThreadPool(3);
Session session = Session.getInstance(new Properties());
try {
final IMAPStore store = (IMAPStore) session.getStore("imaps");
store.connect(mailhost, username, password);
final Folder folder = store.getFolder(foldername);
if (folder == null) {
servletContext.log("Folder in mailbox bestaat niet.");
return;
}
folder.open(Folder.READ_ONLY);
MessageCountListener countListener = new AnalyzerService();
folder.addMessageCountListener(countListener);
Runnable runnable = new Runnable() {
#Override
public void run() {
while (true) {
try {
servletContext.log("Aantal berichten in folder: " + folder.getMessageCount());
servletContext.log("Idling");
store.idle();
} catch (MessagingException ex) {
servletContext.log(ex.getMessage());
return;
}
}
}
};
executorService.execute(runnable);
servletContext.log("Executorservice gestart");
} catch (MessagingException ex) {
servletContext.log(ex.getMessage());
}
}
#Override
public void contextDestroyed(ServletContextEvent sce) {
sce.getServletContext().log("Context wordt vernietigd");
executorService.shutdown();
sce.getServletContext().log("Executorservice gestopt");
}
}
The AnalyzerService.java:
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.event.MessageCountEvent;
import javax.mail.event.MessageCountListener;
class AnalyzerService implements MessageCountListener {
public AnalyzerService() {
}
#Override
public void messagesAdded(MessageCountEvent event) {
Message[] addedMessages = event.getMessages();
for (Message message : addedMessages) {
try {
System.out.println(message.getSubject());
} catch (MessagingException ex) {
System.out.println(ex.getMessage());
}
}
}
#Override
public void messagesRemoved(MessageCountEvent event) {
Message[] removedMessages = event.getMessages();
for (Message message : removedMessages) {
try {
System.out.println(message.getSubject());
} catch (MessagingException ex) {
System.out.println(ex.getMessage());
}
}
}
}

while (true) {
try {
servletContext.log("Aantal berichten in folder: " + folder.getMessageCount());
servletContext.log("Idling");
store.idle();
} catch (MessagingException ex) {
servletContext.log(ex.getMessage());
return;
}
}
has exactly 2 3 possibilities to end earlier than never run only once.
The loop actually ends either:
Through the explicit return in case of a MessagingException. Look at your logs, there either a message or something strange like "null". Consider using a proper stacktrace log (.log(String message, Throwable throwable)) since Exception#getMessage() is often empty or not telling you much.
Through any unchecked exception. You should notice that in some log though since uncaught exceptions via executorService.execute should invoke the nearest uncaught exeption handler which is generally bad. See Choose between ExecutorService's submit and ExecutorService's execute
The loop stops executing after it logs "Idling"
store.idle() never returns. (every other line of code could do that theoretically as well, e.g. the folder.getMessageCount() call in a 2nd iteration but that's very unlikely)
Regarding No 3 - the documentation
Use the IMAP IDLE command (see RFC 2177), if supported by the server, to enter idle mode so that the server can send unsolicited notifications without the need for the client to constantly poll the server. Use a ConnectionListener to be notified of events. When another thread (e.g., the listener thread) needs to issue an IMAP comand for this Store, the idle mode will be terminated and this method will return. Typically the caller will invoke this method in a loop.
If the mail.imap.enableimapevents property is set, notifications received while the IDLE command is active will be delivered to ConnectionListeners as events with a type of IMAPStore.RESPONSE. The event's message will be the raw IMAP response string. Note that most IMAP servers will not deliver any events when using the IDLE command on a connection with no mailbox selected (i.e., this method). In most cases you'll want to use the idle method on IMAPFolder.
That sounds like this method is not designed to return any time soon. In your case never since you don't issue any commands towards the server after you enter idle. Besides that
folder.idle() could be what you should actually do
I guess the documentation is wrong, however ConnectionListener and MessageCountListener are two different things.

Related

Hystrix Circuit breaker not opening the circuit

I am implementing Circuit breaker using Hystrix in my Spring boot application, my code is something like below:
#service
public class MyServiceHandler {
#HystrixCommand(fallbackMethod="fallback")
public String callService() {
// if(remote service is not reachable
// throw ServiceException
}
public String fallback() {
// return default response
}
}
// In application.properties, I have below properties defined:
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=10000
hystrix.command.default.circuitBreaker.requestVolumeThreshold=3
hystrix.command.default.circuitBreaker.sleepWindowInMilliseconds=30000
hystrix.threadpool.default.coreSize=4
hystrix.threadpool.default.metrics.rollingStats.timeInMilliseconds=200000
I see that the fallback() is getting called with each failure of callService(). However, the circuit is not opening after 3 failures. After 3 failures, I was expecting that it will directly call fallback() and skip callService(). But this is not happening. Can someone advise what I am doing wrong here?
Thanks,
B Jagan
Edited on 26th July to add more details below:
Below is the actual code. I played a bit further with this. I see that the Circuit opens as expected on repeated failured when I call the remote service directly in the RegistrationHystrix.registerSeller() method. But, when I wrap the remote service call within Spring retry template, it keeps going into fallback method, but circuit never opens.
#Service
public class RegistrationHystrix {
Logger logger = LoggerFactory.getLogger(RegistrationHystrix.class);
private RestTemplate restTemplate;
private RetryTemplate retryTemplate;
public RegistrationHystrix(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(1000l);
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(3);
retryTemplate.setRetryPolicy(retryPolicy);
}
#HystrixCommand(fallbackMethod = "fallbackForRegisterSeller", commandKey = "ordermanagement")
public String registerSeller(SellerDto sellerDto) throws Exception {
String response = retryTemplate.execute(new RetryCallback<String, Exception>() {
#Override
public String doWithRetry(RetryContext context) {
logger.info(String.format("Retry count %d", context.getRetryCount()));
return restTemplate.postForObject("/addSeller", sellerDto, String.class);
}
});
return response;
}
public List<SellerDto> getSellersList() {
return restTemplate.getForObject("/sellersList", List.class);
}
public String fallbackForRegisterSeller(SellerDto sellerDto, Throwable t) {
logger.error("Inside fall back, cause - {}", t.toString());
return "Inside fallback method. Some error occured while calling service for seller registration";
}
}
Below is the service class which in turn calls the above Hystrix wrapped service. This class in turn is invoked by a controller.
#Service
public class RegistrationServiceImpl implements RegistrationService {
Logger logger = LoggerFactory.getLogger(RegistrationServiceImpl.class);
private RegistrationHystrix registrationHystrix;
public RegistrationServiceImpl(RegistrationHystrix registrationHystrix) {
this.registrationHystrix = registrationHystrix;
}
#Override
public String registerSeller(SellerDto sellerDto) throws Exception {
long start = System.currentTimeMillis();
String registerSeller = registrationHystrix.registerSeller(sellerDto);
logger.info("add seller call returned in - {}", System.currentTimeMillis() - start);
return registerSeller;
}
So, I am trying to understand why the Circuit breaker is not working as expected when using it along with Spring RetryTemplate.
You should be using metrics.healthSnapshot.intervalInMilliseconds while testing. I guess you are executing all 3 request within default 500 ms and hence the circuit isn't getting open. You can either decrease this interval or you may put a sleep between the 3 requests.

How do I tell Camel that I will end the message processing myself

I am writing a component (an endpoint) that will receive the Camel Exchange like this:
from("file|activemq|whatever").to(myEndpoint);
Upon receiving, I want it to pass the exchange to a set of subroutines, which may work asynchronously, and which will eventually decide that they have finished, possibly composed a response in an Out message of the Exchange. All may happen outside the Camel Context, I am working only with the Exchange object.
Then my subroutines should invoke something that will tell Camel that it should propagate the response back, do other stuff as per the source and middle components requirements (for example, if it is a file:/, rename a file) and consider the routing ow this Exchange completed.
I was thinking that I would invoke the Exchange's Uint of Work done method.
Unfortunately I am noticing that Camel still tries to end the exchange by itself too, in wrong time and state. For example, for file source, it fails to rename the file which already has been removed.
Here is some of my code:
Here I define an endpoint:
_proceeder = new DefaultEndpoint() {
private final String _defaultUri = "rex:producer-" + UUID.randomUUID().toString();
#Override
protected String createEndpointUri() {
return _defaultUri;
}
#Override
public Producer createProducer() throws Exception {
return new DefaultAsyncProducer(this) {
#Override
public boolean process(final Exchange exchange1, final AsyncCallback callback) {
final ExchangeWrapper exchange = new ExchangeWrapper(_uri, exchange1, MessageSystem.this);
_LOG.debug("Got input for {}. Processing...", _uri);
exchange._taken(); // 1. all subsequent will increase by 1
/// some majick....
final boolean done = exchange._released(); // if all were released immediately, itll be 0 and sent back now. otherwise the last to release will send it back.
if (done) {
_LOG.debug("Processed input for {} synchronously", _uri);
//callback.done(true);
} else {
_LOG.debug("Processed input for {} asynchronously, awaiting response", _uri);
//exchange1.addOnCompletion(new Synchronization() {
// #Override
// public void onComplete(Exchange exchange) {
// onFailure(exchange);
// }
//
// #Override
// public void onFailure(Exchange exchange) {
// callback.done(false);
// }
//});
}
return done;
}
};
}
#Override
public Consumer createConsumer(Processor processor) throws Exception {
throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
}
#Override
public boolean isSingleton() {
return true;
}
};
_proceeder.setCamelContext(context);
Needless to say that I don't understand why I am given an AsyncCallback in my DefaultAsyncProducer.process() method; regardless of me calling its done() method, the system doesn't see this and still ends the exchange by itself once more. But it is not the question.
here is the ExchangeWrapper _released and _done methods:
private void _done() throws Exception {
UnitOfWork uow = _exchange.getUnitOfWork();
uow.done(_exchange);
//try{
// uow.stop();
//}catch(Exception e){
//
//}
_exchange.setUnitOfWork(null);
}
private boolean _released() {
final boolean ret;
final int cnt;
final int trancnt;
synchronized (_exchange) {
cnt = _exchange.getProperty("rex.takenCount", Integer.class) - 1;
_exchange.setProperty("rex.takenCount", cnt);
trancnt = _exchange.getProperty("rex.takenAsTransient", Integer.class);
}
if (_LOG.isDebugEnabled()) {
_LOG.debug("Input for {} released. {} times left, {} transient", new Object[]{_exchange.getProperty("rex.uri", String.class), cnt, trancnt});
}
if (cnt <= 0 || cnt <= trancnt) {
if (_LOG.isDebugEnabled()) {
_LOG.debug("Message for {} is processed by all non-transient receivers. Setting done...", new Object[]{_exchange.getProperty("rex.uri", String.class)});
}
_done();
ret = true;
if (_LOG.isDebugEnabled()) {
_LOG.debug("Message for {} is set done", new Object[]{_exchange.getProperty("rex.uri", String.class)});
}
} else {
ret = false;
}
return ret;
}
So basically I wrap the Exchange to keep state and decide when the processing should be stopped.
While digging into the Camel internals I've seen some similar counters that keep track of how many times the Exchange has been taken for processing, but I'd like to be in control, thus my own wrapper.
So what should I call instead of
_exchange.getUnitOfWork().done(_exchange);
to tell the Camel Internal Processor and others that there is no need to mark the exchange done because I am doing it?
My latest finding is to call uow.stop(); so that it clears all the 'after' processors etc, but I suddenly understood that I may try and hack Camel myself for a long time, but it's better to ask people who know exactly what to do without trying and guessing.
These are the examples of my Routes:
RouteBuilder rb = new RouteBuilder(_context) {
#Override
public void configure() throws Exception {
if (_tokenizer != null) {
from(_uri).split().method(_tokenizer, "tokenizeReader").streaming().to(_proceeder);
} else {
from(_uri).to(_proceeder);
}
}
};
If I could avoid building routes, instantiating endpoints and producers, and employ standalone processors, I'd happily do, but I don't want to ditch what the marvelous Camel project has to offer in terms of splitting, streaming, marshalling etc etc; and all of this seems to be built around routes.
May be I am not clear with what are you trying to achieve with this, but let me try.
Upon receiving, I want it to pass the exchange to a set of
subroutines, which may work asynchronously, and which will eventually
decide that they have finished
So for this you can write a processor and configure it at the end of your route .Within your processor you can use a thread pool, submit to it the subroutine tasks, wait for their completion and decide if you want to change the message body ( correct way explained here with a good diagram explaining flow of an exchange through a route) and camel will automatically take care of returning the response to the caller of the route based on exchange pattern. For example in your case if the route begins from file/activemq route then it is event based/one way exchange and no response will be sent to the caller as there is no caller client as such. It will be just an event which will initiate the exchange.
Update :
For using the async processing feature in camel for enhanced scalability take a look at this example with code from the highly recommended Camel in Action book

ChannelFactory method call increse memory

I have an winform application which consumes windows service, i user ChannelFactory
to connect to service, problem is when i call service method using channel the memory usage increase and after
method execute memory not go down(even after form close), i call GC.Collect but no change
channel Create class
public class Channel1
{
List<ChannelFactory> chanelList = new List<ChannelFactory>();
ISales salesObj;
public ISales Sales
{
get
{
if (salesObj == null)
{
ChannelFactory<ISales> saleschannel = new ChannelFactory<ISales>("SalesEndPoint");
chanelList.Add(saleschannel);
salesObj = saleschannel.CreateChannel();
}
return salesObj;
}
}
public void CloseAllChannels()
{
foreach (ChannelFactory chFac in chanelList)
{
chFac.Abort();
((IDisposable)chFac).Dispose();
}
salesObj = null;
}
}
base class
public class Base:Form
{
public Channel1 channelService = new Channel1();
public Channel1 CHANNEL
{
get
{
return channelService;
}
}
}
winform class
Form1:Base
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
_dt = CHANNEL.Sales.GetDatatable();
gvAccounts.DataSource = _dt;
}
catch (Exception ex)
{
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
finally
{
CHANNEL.CloseAllChannels();
_dt.Dispose();
//GC.Collect();
}
}
You're on the right track in terms of using ChannelFactory<T>, but your implementation is a bit off.
ChannelFactory<T> creates a factory for generating channels of type T. This is a relatively expensive operation (as compared to just creating a channel from the existing factory), and is generally done once per life of the application (usually at start). You can then use that factory instance to create as many channels as your application needs.
Generally, once I've created the factory and cached it, when I need to make a call to the service I get a channel from the factory, make the call, and then close/abort the channel.
Using your posted code as a starting point, I would do something like this:
public class Channel1
{
ChannelFactory<ISales> salesChannel;
public ISales Sales
{
get
{
if (salesChannel == null)
{
salesChannel = new ChannelFactory<ISales>("SalesEndPoint");
}
return salesChannel.CreateChannel();
}
}
}
Note that I've replaced the salesObj with salesChannel (the factory). This will create the factory the first time it's called, and create a new channel from the factory every time.
Unless you have a particular requirement to do so, I wouldn't keep track of the different channels, especially if follow the open/do method/close approach.
In your form, it'd look something like this:
private void btnView_Click(object sender, EventArgs e)
{
DataTable _dt = new DataTable();
try
{
gvAccounts.AutoGenerateColumns = false;
ISales client = CHANNEL.Sales
_dt = client.GetDatatable();
gvAccounts.DataSource = _dt;
((ICommunicationObject)client).Close();
}
catch (Exception ex)
{
((ICommunicationObject)client).Abort();
MessageBox.Show("Error Occurred while processing...\n" + ex.Message, "Warning", MessageBoxButtons.OK, MessageBoxIcon.Warning);
}
}
The code above gets a new ISales channel from the factory in CHANNEL, executes the call, and then closes the channel. If an exception happens, the channel is aborted in the catch block.
I would avoid using Dispose() out of the box on the channels, as the implementation in the framework is flawed and will throw an error if the channel is in a faulted state. If you really want to use Dispose() and force the garbage collection, you can - but you'll have to work around the WCF dispose issue. Google will give you a number of workarounds (google WCF Using for a start).

What does 'moveFailed' really do?

I want to create a file input that behaves as follows:
Process the exchange
Attempt to copy the input file to a shared drive
If step (2) fails (e.g. share is down) then move to local file instead
Following the doc the 'moveFailed' parameter allows to "set a different target directory when moving files after processing (configured via move defined above) failed". So this sounds like the moveFailed would cover step (3).
The following test, however fails...what am I doing wrong ? I am using camel 2.10.0.fuse.
package sandbox.camel;
import java.io.File;
import org.apache.camel.Endpoint;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.junit.Test;
public class MoveFailedTest extends org.apache.camel.test.junit4.CamelTestSupport {
private String failedDir = "move-failed";
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("file:tmp/prepare").to("file:tmp/input");
from("file:tmp/input?move=/doesnotexist&moveFailed=" + failedDir).to("file:tmp/output");
}
};
}
#Test
public void test_move() throws Exception {
// arrange
File moveFailedDir = new File("tmp/input/" + failedDir);
moveFailedDir.mkdirs();
File[] failedCount1 = moveFailedDir.listFiles();
failedCount1 = failedCount1 == null ? new File[0] : failedCount1;
String messagePayload = "Hello";
Endpoint input = getMandatoryEndpoint("file:tmp/prepare");
MockEndpoint output = getMockEndpoint("mock:file:tmp/output");
output.setMinimumExpectedMessageCount(1);
output.expectedBodiesReceived(messagePayload);
// act
template.asyncSendBody(input, messagePayload);
Thread.sleep(3000);
// assert: only 1 output
assertMockEndpointsSatisfied();
// assert: renamed failed, hence input file was moved to 'movefailed' directory
File[] failedCount2 = moveFailedDir.listFiles();
assertEquals("No file appeared in 'movefailed' directory", failedCount1.length + 1, failedCount2.length);
}
}
Your test is most likely wrong. The autocreate option is default true, which means directories is created if needed.

How to properly canalize multithreaded message flow in a single threaded service?

In a WPF application, I have a 3rd party library that is publishing messages.
The messages are like :
public class DialectMessage
{
public string PathAndQuery { get; private set; }
public byte[] Body { get; private set; }
public DialectMessage(string pathAndQuery, byte[] body)
{
this.PathAndQuery = pathAndQuery;
this.Body = body;
}
}
And I setup the external message source from my app.cs file :
public partial class App : Application
{
static App()
{
MyComponent.MessageReceived += MessageReceived;
MyComponent.Start();
}
private static void MessageReceived(Message message)
{
//handle message
}
}
These messages can be publishing from multiple thread at a time, making possible to call the event handler multiple times at once.
I have a service object that have to parse the incoming messages. This service implements the following interface :
internal interface IDialectService
{
void Parse(Message message);
}
And I have a default static instance in my app.cs file :
private readonly static IDialectService g_DialectService = new DialectService();
In order to simplify the code of the parser, I would like to ensure only one message at a time is parsed.
I also want to avoid locking in my event handler, as I don't want to block the 3rd party object.
Because of this requirements, I cannot directly call g_DialectService.Parse from my message event handler
What is the correct way to ensure this single threaded execution?
My first though is to wrap my parsing operations in a Produce/Consumer pattern. In order to reach this goal, I've try the following :
Declare a BlockingCollection in my app.cs :
private readonly static BlockingCollection<Message> g_ParseOperations = new BlockingCollection<Message>();
Change the body of my event handler to add an operation :
private static void MessageReceived(Message message)
{
g_ParseOperations.Add(message);
}
Create a new thread that pump the collection from my app constructor :
static App()
{
MyComponent.MessageReceived += MessageReceived;
MyComponent.Start();
Task.Factory.StartNew(() =>
{
Message message;
while (g_ParseOperations.TryTake(out message))
{
g_DialectService.Parse(message);
}
});
}
However, this code does not seems to work. The service Parse method is never called.
Moreover, I'm not sure if this pattern will allow me to properly shutdown the application.
What have I to change in my code to ensure everything is working?
PS: I'm targeting .Net 4.5
[Edit] After some search, and the answer of ken2k, i can see that I was wrongly calling trytake in place of take.
My updated code is now :
private readonly static CancellationTokenSource g_ShutdownToken = new CancellationTokenSource();
private static void MessageReceived(Message message)
{
g_ParseOperations.Add(message, g_ShutdownToken.Token);
}
static App()
{
MyComponent.MessageReceived += MessageReceived;
MyComponent.Start();
Task.Factory.StartNew(() =>
{
while (!g_ShutdownToken.IsCancellationRequested)
{
var message = g_ParseOperations.Take(g_ShutdownToken.Token);
g_DialectService.Parse(message);
}
});
}
protected override void OnExit(ExitEventArgs e)
{
g_ShutdownToken.Cancel();
base.OnExit(e);
}
This code acts as expected. Messages are processed in the correct order. However, as soon I exit the application, I get a "CancelledException" on the Take method, even if I just test the IsCancellationRequested right before.
The documentation says about BlockingCollection.TryTake(out T item):
If the collection is empty, this method immediately returns false.
So basically your loop exits immediately. What you may want is to call the TryTake method with a timeout parameter instead, and exit your loop when a mustStop variable becomes true:
bool mustStop = false; // Must be set to true on somewhere else when you exit your program
...
while (!mustStop)
{
Message yourMessage;
// Waits 500ms if there's nothing in the collection. Avoid to consume 100% CPU
// for nothing in the while loop when the collection is empty.
if (yourCollection.TryTake(out yourMessage, 500))
{
// Parses yourMessage here
}
}
For your edited question: if you mean you received a OperationCanceledException, that's OK, it's exactly how methods that take a CancellationToken object as parameter must behave :) Just catch the exception and exit gracefully.

Resources