I have a Silverlight application that uses an overridden AudioSink.OnSamples() to record sound, and MediaStreamSource.GetSampleAsync() to play sound.
For instance:
protected override void GetSampleAsync(MediaStreamType mediaStreamType)
{
try
{
logger.LogSampleRequested();
var memoryStream = AudioController == null ? new MemoryStream() : AudioController.GetNextAudioFrame();
timestamp += AudioConstants.MillisecondsPerFrame * TimeSpan.TicksPerMillisecond;
var sample = new MediaStreamSample(
mediaStreamDescription,
memoryStream,
0,
memoryStream.Length,
timestamp, // (DateTime.Now - startTime).Ticks, // Testing shows that incrementing a long with a good-enough value is ~100x faster than calculating the ticks each time.
emptySampleDict);
ReportGetSampleCompleted(sample);
}
catch (Exception ex)
{
ClientLogger.LogDebugMessage(ex.ToString);
}
}
Both of these methods should normally be called every 20 milliseconds, and on most machines, that's exactly what happens. However, on some machines, they get called not every 20 ms, but closer to 22-24 ms. That's troublesome, but with some appropriate buffering, the audio is still more-or-less usable. The bigger problem is that in certain scenarios, such as when the CPU is running close to its limit, the interval between calls rises to as much as 30-35 ms.
So:
(1) Has anyone else seen this?
(2) Does anyone have any suggested workarounds?
(3) Does anyone have any tips for troubleshooting this problem?
For what it's worth, after much investigation, the basic solution to this problem is simply not to use as much CPU. In our case, this meant keeping track of the CPU utilization, and switching to a codec that didn't use as much CPU (G711 vs. Speex) when the CPU starts consistently running at 80% or higher.
Related
I have an Angular 6 application that performs an API call every 10 seconds to update price quotes. The timing of the API calls is manages using RxJS interval().
For some reason, on MS Edge, the timings vary wildly, from a couple of seconds, to minutes. Any ideas what might be the cause?
Here is the code:
const refreshPriceInterval = interval(10000);
let startTime = new Date(), endTime: Date;
refreshPriceInterval.pipe(
startWith(0),
flatMap(() => {
endTime = new Date();
let timeDiff = endTime.getTime() - startTime.getTime(); //in ms
// strip the ms
timeDiff /= 1000;
console.log(timeDiff);
startTime = new Date();
return this.timeseriesService.getQuote(symbol, this.userInfo.apiLanguage);
})
)
Here is the console output:
0.001
18.143
4.111
11.057
13.633
12.895
3.003
12.394
7.336
31.616
20.221
10.461
Is there a way to increase the accuracy?
EDIT:
Performance degrades over time.
Reducing the code in the interval() to only a console.log does not perform any better.
Might be an Angular issue.
It is up to the browser to decide how many CPU cycles are allocated per browser tab. Depending on resources (for instance; battery) or activity (background tab vs foreground tab) your browser page will receive more or less cpu slices.
some background: https://github.com/WICG/interventions/issues/5
This is shipped in Edge as well now (as of EdgeHTML14). The clamping to 1Hz in background tabs, not anything more intensive.
Apart from this fact; you are measuring the latency of your call as well:
timeseriesService.getQuote() so it might also be that this call just takes some time.
It was indeed an Angular issue. Timed processes cause Angular to constantly re-render the app, which can be quite resource intensive.
I used runOutsideAngular() to circumvent this issue. It turned out that I only had to run one function outside of Angular:
this.ngZone.runOutsideAngular(() => {
this.handleMouseEvents();
});
Thanks to Mark van Straten for your input!
I want to create a realtime sine generator using apples core audio framework. I want do do it low level so I can learn and understand the fundamentals.
I know that using PortAudio or Jack would probably be easier and I will use them at some point but I would like to get this to work first so I can be confident to understand the fundamentals.
I literally searched for days now on this topic but no one seems to have ever created a real time wave generator using core audio trying to optain low latency while using C and not Swift or Objective-C.
For this I use a project I set up a while ago. It was first designed to be a game. So after the Application starts up, it will enter a run loop. I thought this would perfectly fit as I can use the main loop to copy samples into the audio buffer and process rendering and input handling as well.
So far I get sound. Sometimes it works for a while then starts to glitch, sometimes it glitches right away.
This is my code. I tried to simplify if and only present the important parts.
I got multiple questions. They are located in the bottom section of this post.
Applications main run loop. This is where it all starts after the window is created and buffers and memory is initialized:
while (OSXIsGameRunning())
{
OSXProcessPendingMessages(&GameData);
[GlobalGLContext makeCurrentContext];
CGRect WindowFrame = [window frame];
CGRect ContentViewFrame = [[window contentView] frame];
CGPoint MouseLocationInScreen = [NSEvent mouseLocation];
BOOL MouseInWindowFlag = NSPointInRect(MouseLocationInScreen, WindowFrame);
CGPoint MouseLocationInView = {};
if (MouseInWindowFlag)
{
NSRect RectInWindow = [window convertRectFromScreen:NSMakeRect(MouseLocationInScreen.x, MouseLocationInScreen.y, 1, 1)];
NSPoint PointInWindow = RectInWindow.origin;
MouseLocationInView= [[window contentView] convertPoint:PointInWindow fromView:nil];
}
u32 MouseButtonMask = [NSEvent pressedMouseButtons];
OSXProcessFrameAndRunGameLogic(&GameData, ContentViewFrame,
MouseInWindowFlag, MouseLocationInView,
MouseButtonMask);
#if ENGINE_USE_VSYNC
[GlobalGLContext flushBuffer];
#else
glFlush();
#endif
}
Through using VSYNC I can throttle the loop down to 60 FPS. The timing is not super tight but it is quite steady. I also have some code to throttle it manually using mach timing which is even more imprecise. I left it out for readability.
Not using VSYNC or using mach timing to get 60 iterations a second also makes the audio glitch.
Timing log:
CyclesElapsed: 8154360866, TimeElapsed: 0.016624, FPS: 60.155666
CyclesElapsed: 8174382119, TimeElapsed: 0.020021, FPS: 49.946926
CyclesElapsed: 8189041370, TimeElapsed: 0.014659, FPS: 68.216309
CyclesElapsed: 8204363633, TimeElapsed: 0.015322, FPS: 65.264511
CyclesElapsed: 8221230959, TimeElapsed: 0.016867, FPS: 59.286217
CyclesElapsed: 8237971921, TimeElapsed: 0.016741, FPS: 59.733719
CyclesElapsed: 8254861722, TimeElapsed: 0.016890, FPS: 59.207333
CyclesElapsed: 8271667520, TimeElapsed: 0.016806, FPS: 59.503273
CyclesElapsed: 8292434135, TimeElapsed: 0.020767, FPS: 48.154209
What is important here is the function OSXProcessFrameAndRunGameLogic. It is called 60 times a second and it is passed a struct containing basic information like a buffer for rendering, keyboard state and a sound buffer which looks like this:
typedef struct osx_sound_output
{
game_sound_output_buffer SoundBuffer;
u32 SoundBufferSize;
s16* CoreAudioBuffer;
s16* ReadCursor;
s16* WriteCursor;
AudioStreamBasicDescription AudioDescriptor;
AudioUnit AudioUnit;
} osx_sound_output;
Where game_sound_output_buffer is:
typedef struct game_sound_output_buffer
{
real32 tSine;
int SamplesPerSecond;
int SampleCount;
int16 *Samples;
} game_sound_output_buffer;
These get set up before the application enters its run loop.
The size for the SoundBuffer itself is SamplesPerSecond * sizeof(uint16) * 2 where SamplesPerSecond = 48000.
So inside OSXProcessFrameAndRunGameLogic is the sound generation:
void OSXProcessFrameAndRunGameLogic(osx_game_data *GameData, CGRect WindowFrame,
b32 MouseInWindowFlag, CGPoint MouseLocation,
int MouseButtonMask)
{
GameData->SoundOutput.SoundBuffer.SampleCount = GameData->SoundOutput.SoundBuffer.SamplesPerSecond / GameData->TargetFramesPerSecond;
// Oszi 1
OutputTestSineWave(GameData, &GameData->SoundOutput.SoundBuffer, GameData->SynthesizerState.ToneHz);
int16* CurrentSample = GameData->SoundOutput.SoundBuffer.Samples;
for (int i = 0; i < GameData->SoundOutput.SoundBuffer.SampleCount; ++i)
{
*GameData->SoundOutput.WriteCursor++ = *CurrentSample++;
*GameData->SoundOutput.WriteCursor++ = *CurrentSample++;
if ((char*)GameData->SoundOutput.WriteCursor >= ((char*)GameData->SoundOutput.CoreAudioBuffer + GameData->SoundOutput.SoundBufferSize))
{
//printf("Write cursor wrapped!\n");
GameData->SoundOutput.WriteCursor = GameData->SoundOutput.CoreAudioBuffer;
}
}
}
Where OutputTestSineWave is the part where the buffer is actually filled with data:
void OutputTestSineWave(osx_game_data *GameData, game_sound_output_buffer *SoundBuffer, int ToneHz)
{
int16 ToneVolume = 3000;
int WavePeriod = SoundBuffer->SamplesPerSecond/ToneHz;
int16 *SampleOut = SoundBuffer->Samples;
for(int SampleIndex = 0;
SampleIndex < SoundBuffer->SampleCount;
++SampleIndex)
{
real32 SineValue = sinf(SoundBuffer->tSine);
int16 SampleValue = (int16)(SineValue * ToneVolume);
*SampleOut++ = SampleValue;
*SampleOut++ = SampleValue;
SoundBuffer->tSine += Tau32*1.0f/(real32)WavePeriod;
if(SoundBuffer->tSine > Tau32)
{
SoundBuffer->tSine -= Tau32;
}
}
}
So when the Buffers are created at start up also Core audio is initialized which I do like this:
void OSXInitCoreAudio(osx_sound_output* SoundOutput)
{
AudioComponentDescription acd;
acd.componentType = kAudioUnitType_Output;
acd.componentSubType = kAudioUnitSubType_DefaultOutput;
acd.componentManufacturer = kAudioUnitManufacturer_Apple;
AudioComponent outputComponent = AudioComponentFindNext(NULL, &acd);
AudioComponentInstanceNew(outputComponent, &SoundOutput->AudioUnit);
AudioUnitInitialize(SoundOutput->AudioUnit);
// uint16
//AudioStreamBasicDescription asbd;
SoundOutput->AudioDescriptor.mSampleRate = SoundOutput->SoundBuffer.SamplesPerSecond;
SoundOutput->AudioDescriptor.mFormatID = kAudioFormatLinearPCM;
SoundOutput->AudioDescriptor.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsNonInterleaved | kAudioFormatFlagIsPacked;
SoundOutput->AudioDescriptor.mFramesPerPacket = 1;
SoundOutput->AudioDescriptor.mChannelsPerFrame = 2; // Stereo
SoundOutput->AudioDescriptor.mBitsPerChannel = sizeof(int16) * 8;
SoundOutput->AudioDescriptor.mBytesPerFrame = sizeof(int16); // don't multiply by channel count with non-interleaved!
SoundOutput->AudioDescriptor.mBytesPerPacket = SoundOutput->AudioDescriptor.mFramesPerPacket * SoundOutput->AudioDescriptor.mBytesPerFrame;
AudioUnitSetProperty(SoundOutput->AudioUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
0,
&SoundOutput->AudioDescriptor,
sizeof(SoundOutput->AudioDescriptor));
AURenderCallbackStruct cb;
cb.inputProc = OSXAudioUnitCallback;
cb.inputProcRefCon = SoundOutput;
AudioUnitSetProperty(SoundOutput->AudioUnit,
kAudioUnitProperty_SetRenderCallback,
kAudioUnitScope_Global,
0,
&cb,
sizeof(cb));
AudioOutputUnitStart(SoundOutput->AudioUnit);
}
The initialization code for core audio sets the render callback to OSXAudioUnitCallback
OSStatus OSXAudioUnitCallback(void * inRefCon,
AudioUnitRenderActionFlags * ioActionFlags,
const AudioTimeStamp * inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList * ioData)
{
#pragma unused(ioActionFlags)
#pragma unused(inTimeStamp)
#pragma unused(inBusNumber)
//double currentPhase = *((double*)inRefCon);
osx_sound_output* SoundOutput = ((osx_sound_output*)inRefCon);
if (SoundOutput->ReadCursor == SoundOutput->WriteCursor)
{
SoundOutput->SoundBuffer.SampleCount = 0;
//printf("AudioCallback: No Samples Yet!\n");
}
//printf("AudioCallback: SampleCount = %d\n", SoundOutput->SoundBuffer.SampleCount);
int SampleCount = inNumberFrames;
if (SoundOutput->SoundBuffer.SampleCount < inNumberFrames)
{
SampleCount = SoundOutput->SoundBuffer.SampleCount;
}
int16* outputBufferL = (int16 *)ioData->mBuffers[0].mData;
int16* outputBufferR = (int16 *)ioData->mBuffers[1].mData;
for (UInt32 i = 0; i < SampleCount; ++i)
{
outputBufferL[i] = *SoundOutput->ReadCursor++;
outputBufferR[i] = *SoundOutput->ReadCursor++;
if ((char*)SoundOutput->ReadCursor >= (char*)((char*)SoundOutput->CoreAudioBuffer + SoundOutput->SoundBufferSize))
{
//printf("Callback: Read cursor wrapped!\n");
SoundOutput->ReadCursor = SoundOutput->CoreAudioBuffer;
}
}
for (UInt32 i = SampleCount; i < inNumberFrames; ++i)
{
outputBufferL[i] = 0.0;
outputBufferR[i] = 0.0;
}
return noErr;
}
This is mostly all there is to it. This is quite long but I did not see a way to present all needed information in a more compact way. I wanted to show all because I am by no means a professional programmer. If there is something you feel is missing, pleas tell me.
My feeling tells me that there is something wrong with the timing. I feel the function OSXProcessFrameAndRunGameLogic sometimes needs more time so that the core audio callback is already pulling samples out of the buffer before it is fully written by OutputTestSineWave.
There is actually more stuff going on in OSXProcessFrameAndRunGameLogic which I did not show here. I am "software rendering" very basic stuff into a framebuffer which is then displayed by OpenGL and I also do keypress checks in there because yeah, its the main function of functionality. In the future this is the place where I would like to handle the controls for multiple oscillators, filters and stuff.
Anyway even if I stop the Rendering and Input handling from being called every iteration I still get audio glitches.
I tried pulling all the sound processing in OSXProcessFrameAndRunGameLogic into an own function void* RunSound(void *GameData) and changed it to:
pthread_t soundThread;
pthread_create(&soundThread, NULL, RunSound, GameData);
pthread_join(soundThread, NULL);
However I got mixed results and was not even sure if multithreading is done like that. Creating and destroying threads 60 times a second didn't seem the way to go.
I also had the idea to let sound processing happen on a completely different thread before the application actually runs into the main loop. Something like two simultaneously running while loops where the first processes audio and the latter UI and input.
Questions:
I get glitchy audio. Rendering and input seem to work correctly but audio sometimes glitches, sometimes it doesn't. From the code I provided, can you maybe see me doing something wrong?
Am I using the core audio technology in a wrong way in order to achieve real time low latency signal generation?
Should I do sound processing in a separate thread like I talked about above? How would threading in this context be done correctly? It would make sense to have a thread only dedicated for sound am I right?
Am I right that the basic audio processing should not be done in the render callback of core audio? Is this function only for outputting the provided sound buffer?
And if sound processing should be done right here, how can I access information like the keyboard state from inside the callback?
Are there any resources you could point me to that I maybe missed?
This is the only place I know where I can get help with this project. I would really appreciate your help.
And if something is not clear to you please let me know.
Thank you :)
In general when dealing with low-latency audio you want to achieve the most deterministic behaviour possible.
This, for example, translates to:
Don't hold any locks on the audio thread (priority inversion)
No memory allocation on the audio thread (takes often too much time)
No file/network IO on the audio thread (takes often too much time)
Question 1:
There are indeed some problems with your code for when you want to achieve continuous, realtime, non-glitching audio.
1. Two different clock domains.
You are providing audio data from a (what I call) a different clock domain than the clock domain asking for data. Clock domain 1 in this case is defined by your TargetFramesPerSecond value, clock domain 2 defined by Core Audio. However, due too how scheduling works you have no guarantee that you thread is finishing in time and on time. You try to target your rendering to n frames per second, but what happens when you don't make it time wise? As far as I can see you don't compensate for the deviation a render cycle took compared to the ideal timing.
The way threading works is that ultimately the OS scheduler decides when your thread is active. There are never guarantees and this causes you render cycles to be not very precise (in term of precision you need for audio rendering).
2. There is no synchronisation between the render thread and the Core Audio rendercallback thread.
The thread where the OSXAudioUnitCallback runs is not the same as where your OSXProcessFrameAndRunGameLogic and thus OutputTestSineWave run. You are providing data from your main thread, and data is being read from the Core Audio render thread. Normally you would use some mutexes to protect you data, but in this case that's not possible because you would run into the problem of priority inversion.
A way of dealing with race conditions is to use a buffer which uses atomic variables to store the usage and pointers of the buffer and let only 1 producer and 1 consumer use this buffer.
Good examples of such buffers are:
https://github.com/michaeltyson/TPCircularBuffer
https://github.com/andrewrk/libsoundio/blob/master/src/ring_buffer.h
3. There are a lot of calls in you audio render thread which prevent deterministic behaviour.
As you wrote you are doing a lot more inside the same audio render thread. Changes are quite high that there will be stuff going on (under the hood) which prevents your thread from being on time. Generally, you should avoid calls which take either too much time or are not deterministic. With all the OpenGL/keypres/framebuffer rendering there is no way to be certain you thread will "arrive on time".
Below are some resources worth looking into.
Question 2:
AFAICT generally speaking, you are using the Core Audio technology correctly. The only problem I think you have is on the providing side.
Question 3:
Yes. Definitely! Although, there are multiple ways of doing this.
In your case you have a normal-priority thread running to do the rendering and a high-performance, realtime thread on which the audio render callback is being called. Looking at your code I would suggest putting the generation of the sine wave inside the render callback function (or call OutputTestSineWave from the render callback). This way you have the audio generation running inside a reliable high prio thread, there is no other rendering interfering with the timing precision and there is no need for a ringbuffer.
In other cases where you need to do "non-realtime" processing to get audiodata ready (think of reading from a file, reading from a network or even from another physical audio device) you cannot run this logic inside the Core Audio thread. A way to solve this is to start a separate, dedicated thread to do this processing. To pass the data to the realtime audio thread you would then make use of the earlier mentioned ringbuffer.
It basically boiles down to two simple goals: for the realtime thread it is necessary to have the audio data available at all times (all render calls), if this failes you will end up sending invalid (or better zeroed) audio data.
The main goal for the secondary thread is to fill up the ringbuffer as fast as possible and to keep the ringbuffer as full as possible. So, whenever there is room to put new audio data into the ringbuffer the thread should do just that.
The size of the ringbuffer in this case will dicate how much tolerance there will be for delay. The size of the ringbuffer will be a balance between certainty (bigger buffer) and latency (smaller buffer).
BTW. I'm quite certain Core Audio has all the facilities to do all this for you.
Question 4:
There are multiple ways of achieving you goal, and rendering the stuff inside the render callback from Core Audio is definitely one of them. The one thing you should keep in mind is that you have to make sure the function returns in time.
For changing parameters to manipulate the audio rendering you'll have to find a way of passing messages which enables the reader (audio renderer function) to get messages without locking and waiting. The way I have done this is to create a second ringbuffer which hold messages from which the audio renderer can consume. This can be as simple as a ringbuffer which hold structs with data (or even pointers to data). As long as you stick to the rules of no locking.
Question 5:
I don't know what resources you are aware of but here are some must-reads:
http://atastypixel.com/blog/four-common-mistakes-in-audio-development/
http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
https://developer.apple.com/library/archive/qa/qa1467/_index.html
You basic problem is that you are trying to push audio from your game loop instead of letting the audio system pull it; e.g. instead of always having (or quickly being able to create *) enough audio samples ready for the amount requested by the audio callback to be pulled by the audio callback. The "always" has to account for enough slop to cover timing jitter (being called late or early or too few times) in your game loop.
(* with no locks, semaphores, memory allocation or Objective C messages)
I have some fairly simple stream code that aggregating data via time windows. The windows are on the large side (1 hour, with a 2 hour bound), and the values in the streams are metrics coming from hundreds of servers. I keep running out of memory, and so I added the RocksDBStateBackend. This caused the JVM to segfault. Next I tried the FsStateBackend. Both of these backends never wrote any data to disk, but simply created a directory with the JobID. I'm running this code in standalone mode, not deployed. Any thoughts as to why the State Backends aren't writing data, and why it runs out of memory even when provided with 8GB of heap?
final SingleOutputStreamOperator<Metric> metricStream =
objectStream.map(node -> new Metric(node.get("_ts").asLong(), node.get("_value").asDouble(), node.get("tags"))).name("metric stream");
final WindowedStream<Metric, String, TimeWindow> hourlyMetricStream = metricStream
.assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<Metric>(Time.hours(2)) { // set how long metrics can come late
#Override
public long extractTimestamp(final Metric metric) {
return metric.get_ts() * 1000; // needs to be in ms since Java epoch
}
})
.keyBy(metric -> metric.getMetricName()) // key the stream so we can run the windowing in parallel
.timeWindow(Time.hours(1)); // setup the time window for the bucket
// create a stream for each type of aggregation
hourlyMetricStream.sum("_value") // we want to sum by the _value
.addSink(new MetricStoreSinkFunction(parameters, "sum"))
.name("hourly sum stream")
.setParallelism(6);
hourlyMetricStream.aggregate(new MeanAggregator())
.addSink(new MetricStoreSinkFunction(parameters, "mean"))
.name("hourly mean stream")
.setParallelism(6);
hourlyMetricStream.aggregate(new ReMedianAggregator())
.addSink(new MetricStoreSinkFunction(parameters, "remedian"))
.name("hourly remedian stream")
.setParallelism(6);
env.execute("flink test");
It is tough to say why you would run out of memory unless you have a very large number of metric names (that is the only explanation I can come up with based on the code you posted).
With respect to the disk writing, RocksDB will actually use a temporary directory by default for its actual database files. You can also pass an explicit directory during configuration. You would do this by calling state.setDbStoragePath(someDirectory)
Somewhat confusingly, the FSStateBackend in fact only writes to disk during checkpointing, it otherwise is entirely heap based. So you likely did not see anything in the directory if you did not have checkpointing enabled. So that would explain why you might still run out of memory when the FSStateBackend is used.
Assuming you do have the RocksDB (or any) state backend working, you can enable checkpointing by doing:
env.enableCheckpointing(5000); // value is in MS, so however frequently you want to checkpoint
env.getCheckpointConfig.setMinPauseBetweenCheckpoints(5000); // this is to help prevent your job from making progress if checkpointing takes a bit. For large state checkpointing can take multiple seconds
This may or may not be a bug, but I would like some help understanding the behavior of Timer.
Here is a test program that sets up Timer.periodic with a duration of 1000 microseconds (1 millisecond). The callback that fires increments a count. Once the count reaches 1000 intervals, the program prints the time elapsed and exits. The point being to get close to 1 second in execution time. Consider the following:
import 'dart:async'
main() {
int count = 0;
var stopwatch = new Stopwatch();
stopwatch.start();
new Timer.periodic(new Duration(microseconds: 1000), (Timer t) {
count++;
if(count == 1000){
print(stopwatch.elapsed);
stopwatch.stop();
}
});
The result is:
0:00:01.002953
That is, just over a second (assuming the remainder is coming from start time of the stopwatch).
However, if you change the resolution to be anything under 1 millisecond e.g. 500 microseconds, the Timer seems to ignore the duration entirely and executes as quickly as possible.
Result being:
0:00:00.008911
I would have expected this to be closer to half a second. Is this an issue with the granularity of the Timer? This issue can also be observed when applying a similar scenario to Future.delayed
The minimal resolution of the timer is 1ms. When asking for a 500ns duration is rounded to 0ms, aka: as fast as possible.
The code is:
int milliseconds = duration.inMilliseconds;
if (milliseconds < 0) milliseconds = 0;
return _TimerFactory._factory(milliseconds, callback, true);
Maybe it should take 1ms as a minimum, if that is its actual minimum, or it should handle microseconds internally, even if it only triggers every 10-15 milliseconds and runs the events pending so far.
If you are in VM it looks like a bug. Please file an issue.
If you are in JS side see the following note on the documentation of the Timer class:
Note: If Dart code using Timer is compiled to JavaScript, the finest granularity available in the browser is 4 milliseconds.
I'm trying to acquire data from an MCU, save them to a file and plot them. The code functions properly for some time, then just hangs randomly (sometimes after 1 sec, sometimes after 1 minute ...!). Also the serialport timeouts are not respected, i.e. I'm not receiving any timeout exceptions. I'm using an FTDI232RL chip. The only time I get a timeout exception is when I unplug it while the program is running.
Code:
private: System::Void START_Click(System::Object^ sender, System::EventArgs^ e) {
seconds=0;
minutes=0;
hours=0;
days=0;
t=0;
if((this->comboBox4->Text == String::Empty)||(this->textBox2->Text == String::Empty)||(this->textBox3->Text == String::Empty)){
this->textBox1->Text="please select port, save file directory and logging interval";
timer1->Enabled=false;
}
else{ // start assigning
w=Convert::ToDouble(this->textBox3->Text);
double q=fmod(w*1000,10);
if(q!=0){
MessageBox::Show("The logging interval must be a multiple of 0.01s");
}
else{
period=static_cast<int>(w*1000);
this->interval->Interval = period;
try{ // first make sure port isn't busy/open
if(!this->serialPort1->IsOpen){
// select the port whose name is in comboBox4 (select port)
this->serialPort1->PortName=this->comboBox4->Text;
//open the port
this->serialPort1->Open();
this->serialPort1->ReadTimeout = period+1;
this->serialPort1->WriteTimeout = period+1;
String^ name_ = this->serialPort1->PortName;
START=gcnew String("S");
this->textBox1->Text="Logging started";
timer1->Enabled=true;
interval->Enabled=true;
myStream=new ofstream(directory,ios::out);
*myStream<<"time(ms);ADC1;ADC2;ADC3;ADC4;ADC5;ADC6;ADC7;ADC8;";
*myStream<<endl;
chart1->Series["ADC1"]->Points->Clear();
chart1->Series["ADC2"]->Points->Clear();
chart1->Series["ADC3"]->Points->Clear();
chart1->Series["ADC4"]->Points->Clear();
chart1->Series["ADC5"]->Points->Clear();
chart1->Series["ADC6"]->Points->Clear();
chart1->Series["ADC7"]->Points->Clear();
chart1->Series["ADC8"]->Points->Clear();
backgroundWorker1->RunWorkerAsync();
}
else
{
this->textBox1->Text="Warning: port is busy or isn't open";
timer1->Enabled=false;
interval->Enabled=false;
}
}
catch(UnauthorizedAccessException^)
{
this->textBox1->Text="Unauthorized access";
timer1->Enabled=false;
interval->Enabled=false;
}
}
}
}
private: System::Void backgroundWorker1_DoWork(System::Object^ sender, System::ComponentModel::DoWorkEventArgs^ e) {
while(!backgroundWorker1->CancellationPending){
if(backgroundWorker1->CancellationPending){
e->Cancel=true;
return;
}
t+=period;
if(t<10*period){
this->chart1->ChartAreas["ChartArea1"]->AxisX->Minimum=0;
this->chart1->ChartAreas["ChartArea1"]->AxisX->Maximum=t+10*period;
}
else {
this->chart1->ChartAreas["ChartArea1"]->AxisX->Minimum=t-10*period;
this->chart1->ChartAreas["ChartArea1"]->AxisX->Maximum=t+10*period;
}
*myStream<<t<<";";
for (int n=0;n<8;n++){
adc_array[n]= this->serialPort1->ReadByte();
}
Array::Copy(adc_array,ADC,8);
for(int f=0; f<8; f++){
*myStream<<ADC[f]<<";";
}
*myStream<<endl;
backgroundWorker1->ReportProgress(t);
}
}
private: System::Void backgroundWorker1_ProgressChanged(System::Object^ sender, System::ComponentModel::ProgressChangedEventArgs^ e) {
chart1->Series["ADC1"]->Points->AddXY(t,ADC[0]);
chart1->Series["ADC2"]->Points->AddXY(t,ADC[1]);
chart1->Series["ADC3"]->Points->AddXY(t,ADC[2]);
chart1->Series["ADC4"]->Points->AddXY(t,ADC[3]);
chart1->Series["ADC5"]->Points->AddXY(t,ADC[4]);
chart1->Series["ADC6"]->Points->AddXY(t,ADC[5]);
chart1->Series["ADC7"]->Points->AddXY(t,ADC[6]);
chart1->Series["ADC8"]->Points->AddXY(t,ADC[7]);
}
the user is allowed to define intervals in seconds for data acquisition (in the code this interval is w after conversion to double). In this case, the program sends a pulse to the MCU requesting a new data transmission. So far, I have been testing this for 1 second intervals (note, during each interval the MCU sends 8 frames, each representing an ADC). However, I need to get this to run for 10ms intervals at some point. Will this be possible? Any idea on how to solve the few problems I mentioned at the beginning?
Thanks in advance
UPDATE
Just to give you an idea of what's happening:
I commented the charting part and ran the program for about 5 minutes, with a reading interval of 1s. So I expected to get around 5x60=300 values in the output file, but I only got 39 (i.e. starting from 1s till 39s). The program was still running, but the data were not getting stored anymore.
Testing was done in release mode and not debug mode. In debug mode, setting a break point under serialport->readbyte(), does not reproduce the problem. My guess is it's a timing issue between program and MCU.
You are making several standard mistakes. First off, do NOT unplug the cable when the port is opened. Many USB emulators don't know how to deal with that, the FTDI driver is particularly notorious about that. They just make the port disappear while it is in use, this invariably gives code that uses the port a severe heart attack. An uncatchable exception is common.
Secondly, you are accessing properties of a class that is not thread-safe in a worker thread. The Chart control was made to be used only in a UI thread, accessing the ChartAreas property in a worker is going to buy you a lot of misery. Getting an InvalidOperationException is pretty typical when you violate threading requirements, it is however not consistently implemented. Nastiness includes random AccessViolationExceptions, corrupted data and deadlock.
Third, you are setting completely unrealistic goals. Pursuing an update every 10 milliseconds is pointless, the human eye cannot perceive that. Anything past 50 milliseconds just turns into a blur. Something that is taken advantage of when you watch a movie in the cinema, it displays at 24 frames per second. The failure mode for that is unpleasant as well, you'll eventually reach a point where you are pummeling the UI thread (or the Chart control) with more updates than it can process. The side effect is that the UI stops painting itself, it is too busy trying to keep up with the deluge of invoke requests. And the amount of memory your program consumes keeps building, the update queue grows without bounds. That does eventually end with an OOM exception, it takes a while to consume 2 jiggabytes however. You will need to prevent this from happening, you need to throttle the rate at which you invoke. A simple thread-safe counter can take care of that.
Forth, you are accessing the data you gather in more than one thread without taking care of thread-safety. The ADC array content is being changed by the worker while the UI thread is reading it. Various amounts of misery from that, bad data at a minimum. A simply workaround is to pass a copy of the data to the ReportProgress method. In general, address these kind of threading problems by using pull instead of push. Get rid of the fire-hose problem by having the UI thread pace the requests instead of trying to have the UI thread keep up.