handling events that do not generate alerts in Apache Flink - apache-flink

Question

I'm using Flink CEP and I need to handle even events that do not generate alerts. Please how can I do it?
I'm consuming events from rabbitMq and have defined some patterns. Now what I need to do is to send all the events received in another Queue to a distant API. I'm new at Flink so I followed the example in the documentation. When I try to send them after matching the received events with the patterns defined I only get those how match with the patterns. What I want to do is just to put an attribute to true in my events for example and send them all to the output queue.
public static void cep() throws Exception {
/**
* RabbitMQ connection
*/
final RMQConnectionConfig connectionConfig = new RMQConnectionConfig.Builder()
.setHost(HOST)
.setPort(PORTS[RD.getValue()])
.setUserName("guest")
.setPassword("guest")
.setVirtualHost("/")
.build();
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
/**
* Retrieve data inputEventstream from rabbitMQ
*/
final DataStream<String> inputEventstream = env
.addSource(new RMQSource<>(
connectionConfig, // config for the RabbitMQ connection
"input", // name of the RabbitMQ queue to consume
true, // use correlation ids; can be false if only at-least-once is required
new SimpleStringSchema())) // deserialization schema to turn messages into Java objects
.setParallelism(1);
/**
* Change DataStream<String> to DataStream<MonitoringEvent> where
* MonitoringEvent refer to a class which modelize our event.
*/
DataStream<MonitoringEvent> inputEventStreamClean = inputEventstream.flatMap(new Tokenizer());
Pattern<MonitoringEvent, ?> warningPattern = Pattern.<MonitoringEvent>begin("start")
.subtype(MonitoringEvent.class)
.where(new SimpleCondition<MonitoringEvent>() {
#Override
public boolean filter(MonitoringEvent value) {
return Integer.parseInt(value.getAncienneChute())>=CHUTE_GRAVE;
}
}).or(new SimpleCondition<MonitoringEvent>() {
#Override
public boolean filter(MonitoringEvent value) {
return value.isChaiseRoulante();
}
}).or(new SimpleCondition<MonitoringEvent>() {
#Override
public boolean filter(MonitoringEvent value) {
return value.isDeambulateur();
}
}).or(new SimpleCondition<MonitoringEvent>() {
#Override
public boolean filter(MonitoringEvent value) {
return value.isDeambulateur();
}
})
.or(new SimpleCondition<MonitoringEvent>() {
#Override
public boolean filter(MonitoringEvent value) {
return EntityManager.getInstance().hasCurrentYearFallTwice(value.getIdClient());
}
});
//PatternStream<MonitoringEvent> fallPatternStream = CEP.pattern(inputEventStreamClean.keyBy("idClient"), warningPattern);
inputEventStreamClean.print();
// Create a pattern stream from our warning pattern
PatternStream<MonitoringEvent> tempPatternStream = CEP.pattern(
inputEventStreamClean.keyBy("idClient"),
warningPattern);
DataStream<FallWarning> warnings = tempPatternStream.select(
(Map<String, List<MonitoringEvent>> pattern) -> {
MonitoringEvent first = (MonitoringEvent) pattern.get("start").get(0);
return new FallWarning(first.getIdClient(), Integer.valueOf(first.getAncienneChute()));
}
);
// Alert pattern: Two consecutive temperature warnings appearing within a time interval of 20 seconds
Pattern<FallWarning, ?> alertPattern = Pattern.<FallWarning>begin("start");
// Create a pattern stream from our alert pattern
PatternStream<FallWarning> alertPatternStream = CEP.pattern(
//warnings.keyBy("idClient"),
warnings,
alertPattern);
// Generate alert
DataStream<Alert> alerts = alertPatternStream.flatSelect(
(Map<String, List<FallWarning>> pattern, Collector<Alert> out) -> {
FallWarning first = pattern.get("start").get(0);
if (first.idNiveauUrgence>=CHUTE_GRAVE && (first.isChaiseRoulante() || first.isDeambulateur() || first.isFracture())) {
out.collect(new Alert(first.idClient));
}
});
// Print the warning and alert events to stdout
warnings.print();
alerts.print(); // here I can send them to RabbitMq
env.execute();
}

Answer 1

You just need to add a Sink to your "alert" DataStream like
alert.addSink(new RMQSink<String>(
connectionConfig, // config for the RabbitMQ connection
"queueName", // name of the RabbitMQ queue to send messages to
new SimpleStringSchema())); // serialization schema to turn Java objects to messages
per example at
https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/connectors/rabbitmq.html