I created a Timer event handler per the code below in a class object.:
public class MyTimerClass
{
private Timer _timeoutTimer;
private void Constructor()
{
_timeoutTimer = new Timer(TimerHandler, null, 0, 1000);
}
private void TimerHandler()
{
if (Something)
LogMessage(LogLevel.Error, "Timeout Waiting...");
}
}
I created a LogMessageHandler event with delegate in the same class object to handle the timer event, and other log events:
public delegate void LogMessageHandler(LogLevel logLevel, string message);
public event LogMessageHandler OnLogMessage;
private void LogMessage(LogLevel logLevel, string message)
{
if (OnLogMessage != null)
OnLogMessage(logLevel, message);
}
In another class, I would like to handle the log message derived for the timer. Here is my subscription to the OnLogMessage event and the class that handles the thread:
void InitializeMyTimerClass()
{
try
{
_myTimerClass = new MyTimerClass();
_myTimerClass.OnLogMessage += new LogMessageHandler(UpdateLogMessage);
}
catch (Exception ex)
{
_dialog.ShowException(ex.Message);
}
}
private void UpdateLogMessage(LogLevel newLogLevel, string message)
{
TaskScheduler schedulerForLog = TaskScheduler.FromCurrentSynchronizationContext();
Task.Factory.StartNew(() => TrackResponseMessage.Add(FormatLogLevelToString(newLogLevel) + ": " + message),
CancellationToken.None, TaskCreationOptions.None, schedulerForLog);
}
When I run the code and the timer event occurs, I put a break point where the TaskScheduler is created: TaskScheduler schedulerForLog = TaskScheduler.FromCurrentSynchronizationContext();
The LogLevel and string parameters are passed from MyTimerClass successfully. However, I get a InvalidOperationException when the TaskScheduler attempts to get the current synchronization context. It appears the SynchronizationContext from the timer thread is not acceptable for the TaskScheduler.
Question: Is the timer event passed in a separate thread? What is the best way to handle the timer thread in this case? Can someone provide demo code? ...Thanks!
If your problem is a cross-thread invalid operation, you can use the Application.Current.Dispatcher to invoke the process you want to run in the main thread of the application to avoid the issue:
private void UpdateLogMessage(LogLevel newLogLevel, string message)
{
Application.Current.Dispatcher.BeginInvoke(new Action(()=>{
TaskScheduler schedulerForLog = TaskScheduler.FromCurrentSynchronizationContext();
Task.Factory.StartNew(() => TrackResponseMessage.Add(FormatLogLevelToString(newLogLevel) + ": " + message),
CancellationToken.None, TaskCreationOptions.None, schedulerForLog);
});
}
If the main thread is not the thread you need then you need to get a hold of the dispatcher for the thread you want and invoke from there.
Related
Can someone please help me understand when and how is the window (session) in flink happens? Or how the samples are processed?
For instance, if I have a continuous stream of events flowing in, events being request coming in an application and response provided by the application.
As part of the flink processing we need to understand how much time is taken for serving a request.
I understand that there are time tumbling windows which gets triggered every n seconds which is configured and as soon as the time lapses then all the events in that time window will be aggregated.
So for example:
Let's assume that the time window defined is 30 seconds and if an event arrives at t time and another arrives at t+30 then both will be processed, but an event arrivng at t+31 will be ignored.
Please correct if I am not right in saying the above statement.
Question on the above is: if say an event arrives at t time and another event arrives at t+3 time, will it still wait for entire 30 seconds to aggregate and finalize the results?
Now in case of session window, how does this work? If the event are being processed individually and the broker time stamp is used as session_id for the individual event at the time of deserialization, then the session window will that be created for each event? If yes then do we need to treat request and response events differently because if we don't then doesn't the response event will get its own session window?
I will try posting my example (in java) that I am playing with in short time but any inputs on the above points will be helpful!
process function
DTO's:
public class IncomingEvent{
private String id;
private String eventId;
private Date timestamp;
private String component;
//getters and setters
}
public class FinalOutPutEvent{
private String id;
private long timeTaken;
//getters and setters
}
===============================================
Deserialization of incoming events:
public class IncomingEventDeserializationScheme implements KafkaDeserializationSchema {
private ObjectMapper mapper;
public IncomingEventDeserializationScheme(ObjectMapper mapper) {
this.mapper = mapper;
}
#Override
public TypeInformation<IncomingEvent> getProducedType() {
return TypeInformation.of(IncomingEvent.class);
}
#Override
public boolean isEndOfStream(IncomingEvent nextElement) {
return false;
}
#Override
public IncomingEvent deserialize(ConsumerRecord<byte[], byte[]> record) throws Exception {
if (record.value() == null) {
return null;
}
try {
IncomingEvent event = mapper.readValue(record.value(), IncomingEvent.class);
if(event != null) {
new SessionWindow(record.timestamp());
event.setOffset(record.offset());
event.setTopic(record.topic());
event.setPartition(record.partition());
event.setBrokerTimestamp(record.timestamp());
}
return event;
} catch (Exception e) {
return null;
}
}
}
===============================================
main logic
public class MyEventJob {
private static final ObjectMapper mapper = new ObjectMapper();
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
MyEventJob eventJob = new MyEventJob();
InputStream inStream = eventJob.getFileFromResources("myConfig.properties");
ParameterTool parameter = ParameterTool.fromPropertiesFile(inStream);
Properties properties = parameter.getProperties();
Integer timePeriodBetweenEvents = 120;
String outWardTopicHostedOnServer = localhost:9092";
DataStreamSource<IncomingEvent> stream = env.addSource(new FlinkKafkaConsumer<>("my-input-topic", new IncomingEventDeserializationScheme(mapper), properties));
SingleOutputStreamOperator<IncomingEvent> filteredStream = stream
.assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<IncomingEvent>() {
long eventTime;
#Override
public long extractTimestamp(IncomingEvent element, long previousElementTimestamp) {
return element.getTimestamp();
}
#Override
public Watermark getCurrentWatermark() {
return new Watermark(eventTime);
}
})
.map(e -> { e.setId(e.getEventId()); return e; });
SingleOutputStreamOperator<FinalOutPutEvent> correlatedStream = filteredStream
.keyBy(new KeySelector<IncomingEvent, String> (){
#Override
public String getKey(#Nonnull IncomingEvent input) throws Exception {
return input.getId();
}
})
.window(GlobalWindows.create()).allowedLateness(Time.seconds(defaultSliceTimePeriod))
.trigger( new Trigger<IncomingEvent, Window> (){
private final long sessionTimeOut;
public SessionTrigger(long sessionTimeOut) {
this.sessionTimeOut = sessionTimeOut;
}
#Override
public TriggerResult onElement(IncomingEvent element, long timestamp, Window window, TriggerContext ctx)
throws Exception {
ctx.registerProcessingTimeTimer(timestamp + sessionTimeOut);
return TriggerResult.CONTINUE;
}
#Override
public TriggerResult onProcessingTime(long time, Window window, TriggerContext ctx) throws Exception {
return TriggerResult.FIRE_AND_PURGE;
}
#Override
public TriggerResult onEventTime(long time, Window window, TriggerContext ctx) throws Exception {
return TriggerResult.CONTINUE;
}
#Override
public void clear(Window window, TriggerContext ctx) throws Exception {
//check the clear method implementation
}
})
.process(new ProcessWindowFunction<IncomingEvent, FinalOutPutEvent, String, SessionWindow>() {
#Override
public void process(String arg0,
ProcessWindowFunction<IncomingEvent, FinalOutPutEvent, String, SessionWindow>.Context arg1,
Iterable<IncomingEvent> input, Collector<FinalOutPutEvent> out) throws Exception {
List<IncomingEvent> eventsIn = new ArrayList<>();
input.forEach(eventsIn::add);
if(eventsIn.size() == 1) {
//Logic to handle incomplete request/response events
} else if (eventsIn.size() == 2) {
//Logic to handle the complete request/response and how much time it took
}
}
} );
FlinkKafkaProducer<FinalOutPutEvent> kafkaProducer = new FlinkKafkaProducer<>(
outWardTopicHostedOnServer, // broker list
"target-topic", // target topic
new EventSerializationScheme(mapper));
correlatedStream.addSink(kafkaProducer);
env.execute("Streaming");
}
}
Thanks
Vicky
From your description, I think you want to write a custom ProcessFunction, which is keyed by the session_id. You'll have a ValueState, where you store the timestamp for the request event. When you get the corresponding response event, you calculate the delta and emit that (with the session_id) and clear out state.
It's likely you'd also want to set a timer when you get the request event, so that if you don't get a response event in safe/long amount of time, you can emit a side output of failed requests.
So, with the default trigger, each window is finalized after it's time fully passes. Depending on whether You are using EventTime or ProcessingTime this may mean different things, but in general, Flink will always wait for the Window to be closed before it is fully processed. The event at t+31 in Your case would simply go to the other window.
As for the session windows, they are windows too, meaning that in the end they simply aggregate samples that have a difference between timestamps lower than the defined gap. Internally, this is more complicated than the normal windows, since they don't have defined starts and ends. The Session Window operator gets sample and creates a new Window for each individual sample. Then, the operator verifies, if the newly created window can be merged with already existing ones (i.e. if their timestamps are closer than the gap) and merges them. This finally results with window that has all elements with timestamps closer to each other than the defined gap.
You are making this more complicated than it needs to be. The example below will need some adjustment, but will hopefully convey the idea of how to use a KeyedProcessFunction rather than session windows.
Also, the constructor for BoundedOutOfOrdernessTimestampExtractor expects to be passed a Time maxOutOfOrderness. Not sure why you are overriding its getCurrentWatermark method with an implementation that ignores the maxOutOfOrderness.
public static void main(String[] args) throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<Event> events = ...
events
.assignTimestampsAndWatermarks(new TimestampsAndWatermarks(OUT_OF_ORDERNESS))
.keyBy(e -> e.sessionId)
.process(new RequestReponse())
...
}
public static class RequestReponse extends KeyedProcessFunction<KEY, Event, Long> {
private ValueState<Long> requestTimeState;
#Override
public void open(Configuration config) {
ValueStateDescriptor<Event> descriptor = new ValueStateDescriptor<>(
"request time", Long.class);
requestState = getRuntimeContext().getState(descriptor);
}
#Override
public void processElement(Event event, Context context, Collector<Long> out) throws Exception {
TimerService timerService = context.timerService();
Long requestedAt = requestTimeState.value();
if (requestedAt == null) {
// haven't seen the request before; save its timestamp
requestTimeState.update(event.timestamp);
timerService.registerEventTimeTimer(event.timestamp + TIMEOUT);
} else {
// this event is the response
// emit the time elapsed between request and response
out.collect(event.timestamp - requestedAt);
}
}
#Override
public void onTimer(long timestamp, OnTimerContext context, Collector<Long> out) throws Exception {
//handle incomplete request/response events
}
}
public static class TimestampsAndWatermarks extends BoundedOutOfOrdernessTimestampExtractor<Event> {
public TimestampsAndWatermarks(Time t) {
super(t);
}
#Override
public long extractTimestamp(Event event) {
return event.eventTime;
}
}
I am having problem with the application freezing. Let me explain my scenario, I have a service which does an async call to a database to get a list of items, It is run by a task. Inside this task I have a try catch block, so it looks like this
public Task<List<T>> ComboListAsync(int? id = null, EnumDTO dto = EnumDTO.Default)
{
return Task.Run(() =>
{
using (var context = new ContextService())
{
try
{
return GetComboList(id, dto, context);
}
catch (Exception e)
{
Handler.DatabaseConnectionException();
throw;
}
}
});
}
Then it throws an exception as GetComboList its just this (for the moment)
protected virtual List<T> GetComboList(int? id, EnumDTO dto, ContextService context)
{
throw new NotImplementedException();
}
So the call catches the exception and goes inside here
public void Show(string message)
{
Message = message;
Application.Current.Dispatcher.Invoke(() =>
{
dialogView = new DialogView() {DataContext = this, Owner = Application.Current.MainWindow};
dialogView.ShowDialog();
});
}
Now the Dispatcher freezes the app, I tried to change it to use begin invoke, it does the same. Without the dispatcher I get an error message that the calling thread is not a STA. I simply want to display my message in a dialog window, that there was a problem connecting to a database. Can anyone help?
I looked online and there is many threads about dispatcher, but none actually show a solution that will fix my issue.
Thank you
EDIT
Code which calls the ComboListAsync
protected override void RetrieveRelatedActiveLists()
{
MyCollection = service.ComboListAsync().Result;
}
Its a deadlock because of the calling code is using the .Result.
Using service.ComboListAsync().Result makes the UI thread await for this method to return, when you call Application.Current.Dispatcher.Invoke from within it you are sending a message to the UI thread that is awaiting the return of method itself.
You must await the method service.ComboListAsync() like this:
protected override async void RetrieveRelatedActiveLists()
{
MyCollection = await service.ComboListAsync();
}
I have a WPF window with a button that spawns a BackgroundWorker thread to create and send an email. While this BackgroundWorker is running, I want to display a user control that displays some message followed by an animated "...". That animation is run by a timer inside the user control.
Even though my mail sending code is on a BackgroundWorker, the timer in the user control never gets called (well, it does but only when the Backgroundworker is finished, which kinda defeats the purpose...).
Relevant code in the WPF window:
private void button_Send_Click(object sender, RoutedEventArgs e)
{
busyLabel.Show(); // this should start the animation timer inside the user control
BackgroundWorker worker = new BackgroundWorker();
worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
worker.DoWork += new DoWorkEventHandler(worker_DoWork);
worker.RunWorkerAsync();
}
void worker_DoWork(object sender, DoWorkEventArgs e)
{
this.Dispatcher.Invoke((Action)(() =>
{
string body = textBox_Details.Text;
body += "User-added addtional information:" + textBox_AdditionalInfo.Text;
var smtp = new SmtpClient
{
...
};
using (var message = new MailMessage(fromAddress, toAddress)
{
Subject = subject,
Body = body
})
{
smtp.Send(message);
}
}));
}
Relevant code in the user control ("BusyLabel"):
public void Show()
{
tb_Message.Text = Message;
mTimer = new System.Timers.Timer();
mTimer.Interval = Interval;
mTimer.Elapsed += new ElapsedEventHandler(mTimer_Elapsed);
mTimer.Start();
}
void mTimer_Elapsed(object sender, ElapsedEventArgs e)
{
this.Dispatcher.Invoke((Action)(() =>
{
int numPeriods = tb_Message.Text.Count(f => f == '.');
if (numPeriods >= NumPeriods)
{
tb_Message.Text = Message;
}
else
{
tb_Message.Text += '.';
}
}));
}
public void Hide()
{
mTimer.Stop();
}
Any ideas why it's locking up?
Using Dispatcher.Invoke in your worker_DoWork method is putting execution back on the UI thread, so you are not really doing the work asynchronously.
You should be able to just remove that, based on the code you are showing.
If there are result values that you need to show after the work is complete, put it in the DoWorkEventArgs and you will be able to access it (on the UI thread) in the worker_RunWorkerCompleted handler's event args.
A primary reason for using BackgroundWorker is that the marshalling is handled under the covers, so you shouldn't have to use Dispatcher.Invoke.
I am using a Timer to run an event periodically on a reasonably long interval (2 minutes). This is working fine. However I would like the event to fire immediately when the timer is created (instead of waiting 2 minutes).
Note that I can't do this just by calling the method, since it takes some time to run and would block the application. I need the timer to fire as normal and run the event in a separate thread.
The best way I can think of doing this at the moment is subclassing the timer and creating a TriggerManually method that would do something like this:
Turn auto reset off
Set the interval to 1ms
Enable the timer
This would trigger the elapsed event straight away, and I could put all the settings back to normal.
Seems a bit roundabout though. Is there a better way to do it?
Couldn't you just call your event handler for the elapsed event manually?
Even if you were expecting it to execute on a thread pool thread, you could invoke it.
class Blah
{
private Timer mTimer;
public Blah()
{
mTimer = new Timer(120000);
ElapsedEventHandler handler = new ElapsedEventHandler(Timer_Elapsed);
mTimer.Elapsed += handler;
mTimer.Enabled = true;
//Manually execute the event handler on a threadpool thread.
handler.BeginInvoke(this, null, new AsyncCallback(Timer_ElapsedCallback), handler);
}
private static void Timer_Elapsed(object source, ElapsedEventArgs e)
{
//Do stuff...
}
private void Timer_ElapsedCallback(IAsyncResult result)
{
ElapsedEventHandler handler = result.AsyncState as ElapsedEventHandler;
if (handler != null)
{
handler.EndInvoke(result);
}
}
}
I liked Rob Cooke's answer, so I built a small EagerTimer class that subclasses System.Timers.Timer and adds this functionality. (With hints from these articles)
I know I could use System.Threading.Timer instead, but this is simple and works well in my application.
EagerTimer
/// <summary>
// EagerTimer is a simple wrapper around System.Timers.Timer that
// provides "set up and immediately execute" functionality by adding a
// new AutoStart property, and also provides the ability to manually
// raise the Elapsed event with RaiseElapsed.
/// </summary>
public class EagerTimer : Timer
{
public EagerTimer()
: base() { }
public EagerTimer(double interval)
: base(interval) { }
// Need to hide this so we can use Elapsed.Invoke below
// (otherwise the compiler complains)
private event ElapsedEventHandler _elapsedHandler;
public new event ElapsedEventHandler Elapsed
{
add { _elapsedHandler += value; base.Elapsed += value; }
remove { _elapsedHandler -= value; base.Elapsed -= value; }
}
public new void Start()
{
// If AutoStart is enabled, we need to invoke the timer event manually
if (AutoStart)
{
this._elapsedHandler.BeginInvoke(this, null, new AsyncCallback(AutoStartCallback), _elapsedHandler); // fire immediately
}
// Proceed as normal
base.Start();
}
private void AutoStartCallback(IAsyncResult result)
{
ElapsedEventHandler handler = result.AsyncState as ElapsedEventHandler;
if (handler != null) handler.EndInvoke(result);
}
// Summary:
// Gets or sets a value indicating whether the EagerTimer should raise
// the System.Timers.Timer.Elapsed event immediately when Start() is called,
// or only after the first time it elapses. If AutoStart is false, EagerTimer behaves
// identically to System.Timers.Timer.
//
// Returns:
// true if the EagerTimer should raise the System.Timers.Timer.Elapsed
// event immediately when Start() is called; false if it should raise the System.Timers.Timer.Elapsed
// event only after the first time the interval elapses. The default is true.
[Category("Behavior")]
[DefaultValue(true)]
[TimersDescription("TimerAutoStart")]
public bool AutoStart { get; set; }
/// <summary>
/// Manually raises the Elapsed event of the System.Timers.Timer.
/// </summary>
public void RaiseElapsed()
{
if (_elapsedHandler != null)
_elapsedHandler(this, null);
}
}
Unit Tests
[TestClass]
public class Objects_EagerTimer_Tests
{
private const int TimerInterval = 10; // ms
private List<DateTime> _timerFires = new List<DateTime>();
private DateTime _testStart;
[TestInitialize]
public void TestSetup()
{
_timerFires.Clear();
_testStart = DateTime.Now;
}
[TestMethod]
public void Objects_EagerTimer_WithAutoStartDisabled()
{
// EagerTimer should behave as a normal System.Timers.Timer object
var timer = new EagerTimer(TimerInterval);
timer.AutoReset = false;
timer.Elapsed += timerElapsed;
timer.Start();
// Wait (not enough time for first interval)
Thread.Sleep(5);
Assert.IsFalse(_timerFires.Any());
// Wait a little longer
Thread.Sleep(TimerInterval);
Assert.AreEqual(1, _timerFires.Count);
}
[TestMethod]
public void Objects_EagerTimer_WithAutoStartEnabled()
{
// EagerTimer should fire immediately on Start()
var timer = new EagerTimer(TimerInterval);
timer.AutoReset = false;
timer.AutoStart = true;
timer.Elapsed += timerElapsed;
timer.Start();
// Wait (not enough time for first interval)
Thread.Sleep(5);
Assert.IsTrue(_timerFires.Any());
// Wait a little longer, now it will have fired twice
Thread.Sleep(TimerInterval);
Assert.AreEqual(2, _timerFires.Count);
}
[TestMethod]
public void Objects_EagerTimer_WhenRaisingManually()
{
// EagerTimer should fire immediately on Start()
var timer = new EagerTimer(TimerInterval);
timer.AutoReset = false;
timer.AutoStart = false;
timer.Elapsed += timerElapsed;
Assert.IsFalse(_timerFires.Any());
timer.RaiseElapsed();
Assert.IsTrue(_timerFires.Any());
}
private void timerElapsed(object sender, ElapsedEventArgs e) {
_timerFires.Add(DateTime.Now);
}
}
Could you use a System.Threading.Timer instead ?
It has a constructor that lets you choose the interval as well as the delay (which can be set to 0 to begin immediately).
http://msdn.microsoft.com/en-us/library/2x96zfy7.aspx
I've got the following scenario: when a user moves the mouse out of a popup, I want an animation to happen, and five seconds later, I want to remove a PopUp.
This is the code I expected to do this with is:
private bool leftPopup = false;
public void AnimatePopupOut(object sender, MouseEventArgs e)
{
myAnim.Begin();
(new Thread(new ThreadStart(delayedRemovePopup))).Start();
}
private void delayedRemovePopup()
{
leftPopup = true;
Thread.Sleep(5000);
PopUp.IsOpen = false;
}
The first line, "leftPopup = true" is fine, but the third, "PopUp.IsOpen = false" gives me an access violation exception, probably because this object belongs to the GUI thread. Is there any way I can gain access to the PopUp.IsOpen property? If not, is there another way that I can call an event after some time to do this?
Cheers
Nik
Try using the PopUp.Dispatcher.Invoke(). That will marshal your call back to the UI thread.
Here is a trick I did in WPF. It is ported for use in Silverlight and hangs off of the Dispacher class. I don't know about Maurice's answer because I don't see a "Invoke" method in SL5. I do see the BeginInvoke, which is about usless when it comes to delayed actions.
Usage: You must include the System.Windows namespace in your code file or this extension method won't appear.
// lets say you want to enable a control 1 second after a save event
// lets say you just want to prevent click happy users from going crazy
// This code assumes you disabled the button on the click event
Button b = this.idButton1;
b.Dispatcher.DelayInvoke(TimeSpan.FromSeconds(1), () => { b.Enabled = true; });
That is it. Just one line of code does the trick. Below is the extension class that makes the above code possible.
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Threading;
namespace System.Windows {
public static class DispatcherExtensions {
public static void DelayInvoke<TArg1, TArg2, TArg3>(this Dispatcher dispatcher, TimeSpan delay, Action<TArg1, TArg2, TArg3> action, TArg1 arg1, TArg2 arg2, TArg3 arg3) {
dispatcher.DelayInvoke(delay, (Delegate)action, arg1, arg2, arg3);
}
public static void DelayInvoke<TArg1, TArg2>(this Dispatcher dispatcher, TimeSpan delay, Action<TArg1, TArg2> action, TArg1 arg1, TArg2 arg2) {
dispatcher.DelayInvoke(delay, (Delegate)action, arg1, arg2);
}
public static void DelayInvoke<TArg1>(this Dispatcher dispatcher, TimeSpan delay, Action<TArg1> action, TArg1 arg1) {
dispatcher.DelayInvoke(delay, (Delegate)action, arg1);
}
public static void DelayInvoke(this Dispatcher dispatcher, TimeSpan delay, Action action) {
dispatcher.DelayInvoke(delay, (Delegate)action);
}
public static void DelayInvoke(this Dispatcher dispatcher, TimeSpan delay, Delegate del, params object[] args) {
if (dispatcher == null)
throw new NullReferenceException();
if (delay < TimeSpan.Zero)
throw new ArgumentOutOfRangeException("delay");
if (del == null)
throw new ArgumentNullException("del");
var task = new Task(() => { Thread.Sleep(delay); });
task.ContinueWith((t) => { dispatcher.BeginInvoke(del, args); });
task.Start();
}
}
}