Evaluate only the latest window for event time based sliding windows - apache-flink

Question

I would like to process events in EvenTime using sliding windows. The sliding interval is 24 hours and increment is 30 minutes. The problem is that below code is producing 48 calculations for each event. In our case events are coming in order so we need only the latest window to be evaluated.
Thanks,
Dejan
public static void processEventsa(
DataStream<Tuple2<String, MyEvent>> events) throws Exception {
events.assignTimestampsAndWatermarks(new MyWatermark()).
keyBy(0).
timeWindow(Time.hours(windowSizeHour), Time.seconds(windowSlideSeconds)).
apply(new WindowFunction<Tuple2<String, MyEvent>, Tuple2<String, MyEvent>, Tuple, TimeWindow>() {
#Override
public void apply(Tuple key, TimeWindow window, Iterable<Tuple2<String, MyEvent>> input,
Collector<Tuple2<String, MyEvent>> out) throws Exception {
for (Tuple2<String, MyEvent> record : input) {
}
}
});
}
public class MyWatermark implements
AssignerWithPunctuatedWatermarks<Tuple2<String, MyEvent>> {
#Override
public long extractTimestamp(Tuple2<String, MyEvent> event, long previousElementTimestamp) {
return event.f1.eventTime;
}
#Override
public Watermark checkAndGetNextWatermark(Tuple2<String, MyEvent> event, long previousElementTimestamp) {
return new Watermark(event.f1.eventTime);
}
}
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

Answer 1

The problem was in watermark. AssignerWithPeriodicWatermarks should be used
public class MyWatermark implements
AssignerWithPeriodicWatermarks<Tuple2<String, MyEvent>> {
private final long maxTimeLag = 5000;
#Override
public long extractTimestamp(Tuple2<String, MyEvent> event, long previousElementTimestamp) {
try {
return event.f1.eventTime;
}
catch(NullPointerException ex) {}
return System.currentTimeMillis() - maxTimeLag;
}
#Override
public Watermark getCurrentWatermark() {
return new Watermark(System.currentTimeMillis() - maxTimeLag);
}
}