I made jpeg decoder, but I didn't implement restart markers logic. That is reason why my program don't work on some images (for example images saved with Photoshop: File->Save As->jpeg). I want to implement restart marker logic, but there is no detailed online explanation how restart marker logic works. Please can anyone tell me more about restart markers, or suggest me online resource where I can read more about it. Thx!
Restart markers are quite simple. They were designed to allow resynchronization after an error. Since most JPEG images are transmitted over error-free channels, they're rarely needed. A restart interval is defined with the FFDD marker as a 2-byte number. This tells how many MCUs between restart markers. When you encounter a restart marker (FFD0-FFD7), reset the DC values (Y,Cr,Cb) to 0 and the bitstream is started on a byte boundary (after the FFDx). It's simply a matter of counting through the restart interval over and over as you decode the image. The restart marker values will increment from FFD0 to FFD7 and then start again at FFD0. The marker value itself is not terribly important, but it can indicate if large chunks of data is missing. Here's an example of how I do it in my decoder. I throw away the restart markers in my bitstream reader.
iRestartCount = iRestartInterval;
for (y=0; y<Height_in_MCUs; y++)
{
for (x=0; x<Width_in_MCUs; x++)
{
<decode an MCU>
if (iRestartInterval) // if there is a restart interval defined
{
if (--iRestartCount == 0)
{
iRestartCount = iRestartInterval; // reset restart inverval counter
iDCPred0 = iDCPred1 = iDCPred2 = 0; // reset DC predictors
if (*iBit & 7) // adjust bitstream to start on the next byte boundary
{
*iBit += (8 - (*iBit & 7));
}
} // if restart interval expired
} // if restart interval defined
} // for x
} // for y
Update: Restart markers now serve a new purpose - to allow multi-threaded JPEG encoders and decoders. Since each "strip" of MCUs has its DC values reset at the beginning of each restart interval and starts on a byte boundary, each restart interval can be independently encoded or decoded by a different thread. An encoder can now arbitrarily divide the task into N threads and then 'glue' the data together with restart markers. For decoders, it's not as easy. If restart markers are present, then each interval can be assigned to a different thread. If not present, you can still do some pre-decoding tricks to split the job into multiple threads.
Related
I'd like to loop a file using GStreamer.
Gst.Element playbin = Gst.ElementFactory.make ("uridecodebin", null);
I do this by adding a probe to the playbin's src pad, and listen for EOS messages. Whenever one comes, I repeat the stream by seeking back to the beginning.
Gst.Pad srcpad = playbin.get_request_pad("src_%u");
srcpad.add_probe(Gst.PadProbeType.EVENT_DOWNSTREAM, (pad, info) => {
Gst.Event? event = info.get_event();
if (event != null)
{
if (event.type == Gst.EventType.EOS
|| event.type == Gst.EventType.SEGMENT_DONE)
{
var element = pad.get_parent_element();
element.seek(1.0, Gst.Format.TIME, Gst.SeekFlags.SEGMENT, Gst.SeekType.SET, 0, Gst.SeekType.NONE, 0);
return Gst.PadProbeReturn.HANDLED;
}
}
return Gst.PadProbeReturn.OK;
});
However, when I catch the EOS and seek back to the beginning, I get this error:
wavparse gstwavparse.c:2195:gst_wavparse_stream_data:<wavparse0> Error pushing on srcpad wavparse0:src, reason eos, is linked? = 1
How do I get my playbin element back out of the EOS state so that it can play from the place I seeked to?
I'd like to avoid listening to the pipeline bus because it's quite a complex application and the playbin is quite a few Bins deep.
Admittedly, my testing was performed with python, rather than C but I see no reason why the logic should be different.
Set the player's pipeline state to Gst.State.NULL, wait for a tenth of a second or so for the pipeline to reach that state, then simply play it again, as if starting from scratch after loading it. Normally, by simply setting the pipeline state to Gst.State.PLAYING.
Note:
As far as I'm aware, only local files can be pre-rolled i.e. do the seek and then play, thus if the file is not local you must play and then seek. Always wait for the pipeline to reach the desired state, before the next operation.
To pre-roll a local file, set the pipeline state to paused, then perform the seek, finally set it to playing, always waiting for the previous operation to finish before moving on to the next.
I want to create a realtime sine generator using apples core audio framework. I want do do it low level so I can learn and understand the fundamentals.
I know that using PortAudio or Jack would probably be easier and I will use them at some point but I would like to get this to work first so I can be confident to understand the fundamentals.
I literally searched for days now on this topic but no one seems to have ever created a real time wave generator using core audio trying to optain low latency while using C and not Swift or Objective-C.
For this I use a project I set up a while ago. It was first designed to be a game. So after the Application starts up, it will enter a run loop. I thought this would perfectly fit as I can use the main loop to copy samples into the audio buffer and process rendering and input handling as well.
So far I get sound. Sometimes it works for a while then starts to glitch, sometimes it glitches right away.
This is my code. I tried to simplify if and only present the important parts.
I got multiple questions. They are located in the bottom section of this post.
Applications main run loop. This is where it all starts after the window is created and buffers and memory is initialized:
while (OSXIsGameRunning())
{
OSXProcessPendingMessages(&GameData);
[GlobalGLContext makeCurrentContext];
CGRect WindowFrame = [window frame];
CGRect ContentViewFrame = [[window contentView] frame];
CGPoint MouseLocationInScreen = [NSEvent mouseLocation];
BOOL MouseInWindowFlag = NSPointInRect(MouseLocationInScreen, WindowFrame);
CGPoint MouseLocationInView = {};
if (MouseInWindowFlag)
{
NSRect RectInWindow = [window convertRectFromScreen:NSMakeRect(MouseLocationInScreen.x, MouseLocationInScreen.y, 1, 1)];
NSPoint PointInWindow = RectInWindow.origin;
MouseLocationInView= [[window contentView] convertPoint:PointInWindow fromView:nil];
}
u32 MouseButtonMask = [NSEvent pressedMouseButtons];
OSXProcessFrameAndRunGameLogic(&GameData, ContentViewFrame,
MouseInWindowFlag, MouseLocationInView,
MouseButtonMask);
#if ENGINE_USE_VSYNC
[GlobalGLContext flushBuffer];
#else
glFlush();
#endif
}
Through using VSYNC I can throttle the loop down to 60 FPS. The timing is not super tight but it is quite steady. I also have some code to throttle it manually using mach timing which is even more imprecise. I left it out for readability.
Not using VSYNC or using mach timing to get 60 iterations a second also makes the audio glitch.
Timing log:
CyclesElapsed: 8154360866, TimeElapsed: 0.016624, FPS: 60.155666
CyclesElapsed: 8174382119, TimeElapsed: 0.020021, FPS: 49.946926
CyclesElapsed: 8189041370, TimeElapsed: 0.014659, FPS: 68.216309
CyclesElapsed: 8204363633, TimeElapsed: 0.015322, FPS: 65.264511
CyclesElapsed: 8221230959, TimeElapsed: 0.016867, FPS: 59.286217
CyclesElapsed: 8237971921, TimeElapsed: 0.016741, FPS: 59.733719
CyclesElapsed: 8254861722, TimeElapsed: 0.016890, FPS: 59.207333
CyclesElapsed: 8271667520, TimeElapsed: 0.016806, FPS: 59.503273
CyclesElapsed: 8292434135, TimeElapsed: 0.020767, FPS: 48.154209
What is important here is the function OSXProcessFrameAndRunGameLogic. It is called 60 times a second and it is passed a struct containing basic information like a buffer for rendering, keyboard state and a sound buffer which looks like this:
typedef struct osx_sound_output
{
game_sound_output_buffer SoundBuffer;
u32 SoundBufferSize;
s16* CoreAudioBuffer;
s16* ReadCursor;
s16* WriteCursor;
AudioStreamBasicDescription AudioDescriptor;
AudioUnit AudioUnit;
} osx_sound_output;
Where game_sound_output_buffer is:
typedef struct game_sound_output_buffer
{
real32 tSine;
int SamplesPerSecond;
int SampleCount;
int16 *Samples;
} game_sound_output_buffer;
These get set up before the application enters its run loop.
The size for the SoundBuffer itself is SamplesPerSecond * sizeof(uint16) * 2 where SamplesPerSecond = 48000.
So inside OSXProcessFrameAndRunGameLogic is the sound generation:
void OSXProcessFrameAndRunGameLogic(osx_game_data *GameData, CGRect WindowFrame,
b32 MouseInWindowFlag, CGPoint MouseLocation,
int MouseButtonMask)
{
GameData->SoundOutput.SoundBuffer.SampleCount = GameData->SoundOutput.SoundBuffer.SamplesPerSecond / GameData->TargetFramesPerSecond;
// Oszi 1
OutputTestSineWave(GameData, &GameData->SoundOutput.SoundBuffer, GameData->SynthesizerState.ToneHz);
int16* CurrentSample = GameData->SoundOutput.SoundBuffer.Samples;
for (int i = 0; i < GameData->SoundOutput.SoundBuffer.SampleCount; ++i)
{
*GameData->SoundOutput.WriteCursor++ = *CurrentSample++;
*GameData->SoundOutput.WriteCursor++ = *CurrentSample++;
if ((char*)GameData->SoundOutput.WriteCursor >= ((char*)GameData->SoundOutput.CoreAudioBuffer + GameData->SoundOutput.SoundBufferSize))
{
//printf("Write cursor wrapped!\n");
GameData->SoundOutput.WriteCursor = GameData->SoundOutput.CoreAudioBuffer;
}
}
}
Where OutputTestSineWave is the part where the buffer is actually filled with data:
void OutputTestSineWave(osx_game_data *GameData, game_sound_output_buffer *SoundBuffer, int ToneHz)
{
int16 ToneVolume = 3000;
int WavePeriod = SoundBuffer->SamplesPerSecond/ToneHz;
int16 *SampleOut = SoundBuffer->Samples;
for(int SampleIndex = 0;
SampleIndex < SoundBuffer->SampleCount;
++SampleIndex)
{
real32 SineValue = sinf(SoundBuffer->tSine);
int16 SampleValue = (int16)(SineValue * ToneVolume);
*SampleOut++ = SampleValue;
*SampleOut++ = SampleValue;
SoundBuffer->tSine += Tau32*1.0f/(real32)WavePeriod;
if(SoundBuffer->tSine > Tau32)
{
SoundBuffer->tSine -= Tau32;
}
}
}
So when the Buffers are created at start up also Core audio is initialized which I do like this:
void OSXInitCoreAudio(osx_sound_output* SoundOutput)
{
AudioComponentDescription acd;
acd.componentType = kAudioUnitType_Output;
acd.componentSubType = kAudioUnitSubType_DefaultOutput;
acd.componentManufacturer = kAudioUnitManufacturer_Apple;
AudioComponent outputComponent = AudioComponentFindNext(NULL, &acd);
AudioComponentInstanceNew(outputComponent, &SoundOutput->AudioUnit);
AudioUnitInitialize(SoundOutput->AudioUnit);
// uint16
//AudioStreamBasicDescription asbd;
SoundOutput->AudioDescriptor.mSampleRate = SoundOutput->SoundBuffer.SamplesPerSecond;
SoundOutput->AudioDescriptor.mFormatID = kAudioFormatLinearPCM;
SoundOutput->AudioDescriptor.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsNonInterleaved | kAudioFormatFlagIsPacked;
SoundOutput->AudioDescriptor.mFramesPerPacket = 1;
SoundOutput->AudioDescriptor.mChannelsPerFrame = 2; // Stereo
SoundOutput->AudioDescriptor.mBitsPerChannel = sizeof(int16) * 8;
SoundOutput->AudioDescriptor.mBytesPerFrame = sizeof(int16); // don't multiply by channel count with non-interleaved!
SoundOutput->AudioDescriptor.mBytesPerPacket = SoundOutput->AudioDescriptor.mFramesPerPacket * SoundOutput->AudioDescriptor.mBytesPerFrame;
AudioUnitSetProperty(SoundOutput->AudioUnit,
kAudioUnitProperty_StreamFormat,
kAudioUnitScope_Input,
0,
&SoundOutput->AudioDescriptor,
sizeof(SoundOutput->AudioDescriptor));
AURenderCallbackStruct cb;
cb.inputProc = OSXAudioUnitCallback;
cb.inputProcRefCon = SoundOutput;
AudioUnitSetProperty(SoundOutput->AudioUnit,
kAudioUnitProperty_SetRenderCallback,
kAudioUnitScope_Global,
0,
&cb,
sizeof(cb));
AudioOutputUnitStart(SoundOutput->AudioUnit);
}
The initialization code for core audio sets the render callback to OSXAudioUnitCallback
OSStatus OSXAudioUnitCallback(void * inRefCon,
AudioUnitRenderActionFlags * ioActionFlags,
const AudioTimeStamp * inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList * ioData)
{
#pragma unused(ioActionFlags)
#pragma unused(inTimeStamp)
#pragma unused(inBusNumber)
//double currentPhase = *((double*)inRefCon);
osx_sound_output* SoundOutput = ((osx_sound_output*)inRefCon);
if (SoundOutput->ReadCursor == SoundOutput->WriteCursor)
{
SoundOutput->SoundBuffer.SampleCount = 0;
//printf("AudioCallback: No Samples Yet!\n");
}
//printf("AudioCallback: SampleCount = %d\n", SoundOutput->SoundBuffer.SampleCount);
int SampleCount = inNumberFrames;
if (SoundOutput->SoundBuffer.SampleCount < inNumberFrames)
{
SampleCount = SoundOutput->SoundBuffer.SampleCount;
}
int16* outputBufferL = (int16 *)ioData->mBuffers[0].mData;
int16* outputBufferR = (int16 *)ioData->mBuffers[1].mData;
for (UInt32 i = 0; i < SampleCount; ++i)
{
outputBufferL[i] = *SoundOutput->ReadCursor++;
outputBufferR[i] = *SoundOutput->ReadCursor++;
if ((char*)SoundOutput->ReadCursor >= (char*)((char*)SoundOutput->CoreAudioBuffer + SoundOutput->SoundBufferSize))
{
//printf("Callback: Read cursor wrapped!\n");
SoundOutput->ReadCursor = SoundOutput->CoreAudioBuffer;
}
}
for (UInt32 i = SampleCount; i < inNumberFrames; ++i)
{
outputBufferL[i] = 0.0;
outputBufferR[i] = 0.0;
}
return noErr;
}
This is mostly all there is to it. This is quite long but I did not see a way to present all needed information in a more compact way. I wanted to show all because I am by no means a professional programmer. If there is something you feel is missing, pleas tell me.
My feeling tells me that there is something wrong with the timing. I feel the function OSXProcessFrameAndRunGameLogic sometimes needs more time so that the core audio callback is already pulling samples out of the buffer before it is fully written by OutputTestSineWave.
There is actually more stuff going on in OSXProcessFrameAndRunGameLogic which I did not show here. I am "software rendering" very basic stuff into a framebuffer which is then displayed by OpenGL and I also do keypress checks in there because yeah, its the main function of functionality. In the future this is the place where I would like to handle the controls for multiple oscillators, filters and stuff.
Anyway even if I stop the Rendering and Input handling from being called every iteration I still get audio glitches.
I tried pulling all the sound processing in OSXProcessFrameAndRunGameLogic into an own function void* RunSound(void *GameData) and changed it to:
pthread_t soundThread;
pthread_create(&soundThread, NULL, RunSound, GameData);
pthread_join(soundThread, NULL);
However I got mixed results and was not even sure if multithreading is done like that. Creating and destroying threads 60 times a second didn't seem the way to go.
I also had the idea to let sound processing happen on a completely different thread before the application actually runs into the main loop. Something like two simultaneously running while loops where the first processes audio and the latter UI and input.
Questions:
I get glitchy audio. Rendering and input seem to work correctly but audio sometimes glitches, sometimes it doesn't. From the code I provided, can you maybe see me doing something wrong?
Am I using the core audio technology in a wrong way in order to achieve real time low latency signal generation?
Should I do sound processing in a separate thread like I talked about above? How would threading in this context be done correctly? It would make sense to have a thread only dedicated for sound am I right?
Am I right that the basic audio processing should not be done in the render callback of core audio? Is this function only for outputting the provided sound buffer?
And if sound processing should be done right here, how can I access information like the keyboard state from inside the callback?
Are there any resources you could point me to that I maybe missed?
This is the only place I know where I can get help with this project. I would really appreciate your help.
And if something is not clear to you please let me know.
Thank you :)
In general when dealing with low-latency audio you want to achieve the most deterministic behaviour possible.
This, for example, translates to:
Don't hold any locks on the audio thread (priority inversion)
No memory allocation on the audio thread (takes often too much time)
No file/network IO on the audio thread (takes often too much time)
Question 1:
There are indeed some problems with your code for when you want to achieve continuous, realtime, non-glitching audio.
1. Two different clock domains.
You are providing audio data from a (what I call) a different clock domain than the clock domain asking for data. Clock domain 1 in this case is defined by your TargetFramesPerSecond value, clock domain 2 defined by Core Audio. However, due too how scheduling works you have no guarantee that you thread is finishing in time and on time. You try to target your rendering to n frames per second, but what happens when you don't make it time wise? As far as I can see you don't compensate for the deviation a render cycle took compared to the ideal timing.
The way threading works is that ultimately the OS scheduler decides when your thread is active. There are never guarantees and this causes you render cycles to be not very precise (in term of precision you need for audio rendering).
2. There is no synchronisation between the render thread and the Core Audio rendercallback thread.
The thread where the OSXAudioUnitCallback runs is not the same as where your OSXProcessFrameAndRunGameLogic and thus OutputTestSineWave run. You are providing data from your main thread, and data is being read from the Core Audio render thread. Normally you would use some mutexes to protect you data, but in this case that's not possible because you would run into the problem of priority inversion.
A way of dealing with race conditions is to use a buffer which uses atomic variables to store the usage and pointers of the buffer and let only 1 producer and 1 consumer use this buffer.
Good examples of such buffers are:
https://github.com/michaeltyson/TPCircularBuffer
https://github.com/andrewrk/libsoundio/blob/master/src/ring_buffer.h
3. There are a lot of calls in you audio render thread which prevent deterministic behaviour.
As you wrote you are doing a lot more inside the same audio render thread. Changes are quite high that there will be stuff going on (under the hood) which prevents your thread from being on time. Generally, you should avoid calls which take either too much time or are not deterministic. With all the OpenGL/keypres/framebuffer rendering there is no way to be certain you thread will "arrive on time".
Below are some resources worth looking into.
Question 2:
AFAICT generally speaking, you are using the Core Audio technology correctly. The only problem I think you have is on the providing side.
Question 3:
Yes. Definitely! Although, there are multiple ways of doing this.
In your case you have a normal-priority thread running to do the rendering and a high-performance, realtime thread on which the audio render callback is being called. Looking at your code I would suggest putting the generation of the sine wave inside the render callback function (or call OutputTestSineWave from the render callback). This way you have the audio generation running inside a reliable high prio thread, there is no other rendering interfering with the timing precision and there is no need for a ringbuffer.
In other cases where you need to do "non-realtime" processing to get audiodata ready (think of reading from a file, reading from a network or even from another physical audio device) you cannot run this logic inside the Core Audio thread. A way to solve this is to start a separate, dedicated thread to do this processing. To pass the data to the realtime audio thread you would then make use of the earlier mentioned ringbuffer.
It basically boiles down to two simple goals: for the realtime thread it is necessary to have the audio data available at all times (all render calls), if this failes you will end up sending invalid (or better zeroed) audio data.
The main goal for the secondary thread is to fill up the ringbuffer as fast as possible and to keep the ringbuffer as full as possible. So, whenever there is room to put new audio data into the ringbuffer the thread should do just that.
The size of the ringbuffer in this case will dicate how much tolerance there will be for delay. The size of the ringbuffer will be a balance between certainty (bigger buffer) and latency (smaller buffer).
BTW. I'm quite certain Core Audio has all the facilities to do all this for you.
Question 4:
There are multiple ways of achieving you goal, and rendering the stuff inside the render callback from Core Audio is definitely one of them. The one thing you should keep in mind is that you have to make sure the function returns in time.
For changing parameters to manipulate the audio rendering you'll have to find a way of passing messages which enables the reader (audio renderer function) to get messages without locking and waiting. The way I have done this is to create a second ringbuffer which hold messages from which the audio renderer can consume. This can be as simple as a ringbuffer which hold structs with data (or even pointers to data). As long as you stick to the rules of no locking.
Question 5:
I don't know what resources you are aware of but here are some must-reads:
http://atastypixel.com/blog/four-common-mistakes-in-audio-development/
http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing
https://developer.apple.com/library/archive/qa/qa1467/_index.html
You basic problem is that you are trying to push audio from your game loop instead of letting the audio system pull it; e.g. instead of always having (or quickly being able to create *) enough audio samples ready for the amount requested by the audio callback to be pulled by the audio callback. The "always" has to account for enough slop to cover timing jitter (being called late or early or too few times) in your game loop.
(* with no locks, semaphores, memory allocation or Objective C messages)
I'm trying to acquire data from an MCU, save them to a file and plot them. The code functions properly for some time, then just hangs randomly (sometimes after 1 sec, sometimes after 1 minute ...!). Also the serialport timeouts are not respected, i.e. I'm not receiving any timeout exceptions. I'm using an FTDI232RL chip. The only time I get a timeout exception is when I unplug it while the program is running.
Code:
private: System::Void START_Click(System::Object^ sender, System::EventArgs^ e) {
seconds=0;
minutes=0;
hours=0;
days=0;
t=0;
if((this->comboBox4->Text == String::Empty)||(this->textBox2->Text == String::Empty)||(this->textBox3->Text == String::Empty)){
this->textBox1->Text="please select port, save file directory and logging interval";
timer1->Enabled=false;
}
else{ // start assigning
w=Convert::ToDouble(this->textBox3->Text);
double q=fmod(w*1000,10);
if(q!=0){
MessageBox::Show("The logging interval must be a multiple of 0.01s");
}
else{
period=static_cast<int>(w*1000);
this->interval->Interval = period;
try{ // first make sure port isn't busy/open
if(!this->serialPort1->IsOpen){
// select the port whose name is in comboBox4 (select port)
this->serialPort1->PortName=this->comboBox4->Text;
//open the port
this->serialPort1->Open();
this->serialPort1->ReadTimeout = period+1;
this->serialPort1->WriteTimeout = period+1;
String^ name_ = this->serialPort1->PortName;
START=gcnew String("S");
this->textBox1->Text="Logging started";
timer1->Enabled=true;
interval->Enabled=true;
myStream=new ofstream(directory,ios::out);
*myStream<<"time(ms);ADC1;ADC2;ADC3;ADC4;ADC5;ADC6;ADC7;ADC8;";
*myStream<<endl;
chart1->Series["ADC1"]->Points->Clear();
chart1->Series["ADC2"]->Points->Clear();
chart1->Series["ADC3"]->Points->Clear();
chart1->Series["ADC4"]->Points->Clear();
chart1->Series["ADC5"]->Points->Clear();
chart1->Series["ADC6"]->Points->Clear();
chart1->Series["ADC7"]->Points->Clear();
chart1->Series["ADC8"]->Points->Clear();
backgroundWorker1->RunWorkerAsync();
}
else
{
this->textBox1->Text="Warning: port is busy or isn't open";
timer1->Enabled=false;
interval->Enabled=false;
}
}
catch(UnauthorizedAccessException^)
{
this->textBox1->Text="Unauthorized access";
timer1->Enabled=false;
interval->Enabled=false;
}
}
}
}
private: System::Void backgroundWorker1_DoWork(System::Object^ sender, System::ComponentModel::DoWorkEventArgs^ e) {
while(!backgroundWorker1->CancellationPending){
if(backgroundWorker1->CancellationPending){
e->Cancel=true;
return;
}
t+=period;
if(t<10*period){
this->chart1->ChartAreas["ChartArea1"]->AxisX->Minimum=0;
this->chart1->ChartAreas["ChartArea1"]->AxisX->Maximum=t+10*period;
}
else {
this->chart1->ChartAreas["ChartArea1"]->AxisX->Minimum=t-10*period;
this->chart1->ChartAreas["ChartArea1"]->AxisX->Maximum=t+10*period;
}
*myStream<<t<<";";
for (int n=0;n<8;n++){
adc_array[n]= this->serialPort1->ReadByte();
}
Array::Copy(adc_array,ADC,8);
for(int f=0; f<8; f++){
*myStream<<ADC[f]<<";";
}
*myStream<<endl;
backgroundWorker1->ReportProgress(t);
}
}
private: System::Void backgroundWorker1_ProgressChanged(System::Object^ sender, System::ComponentModel::ProgressChangedEventArgs^ e) {
chart1->Series["ADC1"]->Points->AddXY(t,ADC[0]);
chart1->Series["ADC2"]->Points->AddXY(t,ADC[1]);
chart1->Series["ADC3"]->Points->AddXY(t,ADC[2]);
chart1->Series["ADC4"]->Points->AddXY(t,ADC[3]);
chart1->Series["ADC5"]->Points->AddXY(t,ADC[4]);
chart1->Series["ADC6"]->Points->AddXY(t,ADC[5]);
chart1->Series["ADC7"]->Points->AddXY(t,ADC[6]);
chart1->Series["ADC8"]->Points->AddXY(t,ADC[7]);
}
the user is allowed to define intervals in seconds for data acquisition (in the code this interval is w after conversion to double). In this case, the program sends a pulse to the MCU requesting a new data transmission. So far, I have been testing this for 1 second intervals (note, during each interval the MCU sends 8 frames, each representing an ADC). However, I need to get this to run for 10ms intervals at some point. Will this be possible? Any idea on how to solve the few problems I mentioned at the beginning?
Thanks in advance
UPDATE
Just to give you an idea of what's happening:
I commented the charting part and ran the program for about 5 minutes, with a reading interval of 1s. So I expected to get around 5x60=300 values in the output file, but I only got 39 (i.e. starting from 1s till 39s). The program was still running, but the data were not getting stored anymore.
Testing was done in release mode and not debug mode. In debug mode, setting a break point under serialport->readbyte(), does not reproduce the problem. My guess is it's a timing issue between program and MCU.
You are making several standard mistakes. First off, do NOT unplug the cable when the port is opened. Many USB emulators don't know how to deal with that, the FTDI driver is particularly notorious about that. They just make the port disappear while it is in use, this invariably gives code that uses the port a severe heart attack. An uncatchable exception is common.
Secondly, you are accessing properties of a class that is not thread-safe in a worker thread. The Chart control was made to be used only in a UI thread, accessing the ChartAreas property in a worker is going to buy you a lot of misery. Getting an InvalidOperationException is pretty typical when you violate threading requirements, it is however not consistently implemented. Nastiness includes random AccessViolationExceptions, corrupted data and deadlock.
Third, you are setting completely unrealistic goals. Pursuing an update every 10 milliseconds is pointless, the human eye cannot perceive that. Anything past 50 milliseconds just turns into a blur. Something that is taken advantage of when you watch a movie in the cinema, it displays at 24 frames per second. The failure mode for that is unpleasant as well, you'll eventually reach a point where you are pummeling the UI thread (or the Chart control) with more updates than it can process. The side effect is that the UI stops painting itself, it is too busy trying to keep up with the deluge of invoke requests. And the amount of memory your program consumes keeps building, the update queue grows without bounds. That does eventually end with an OOM exception, it takes a while to consume 2 jiggabytes however. You will need to prevent this from happening, you need to throttle the rate at which you invoke. A simple thread-safe counter can take care of that.
Forth, you are accessing the data you gather in more than one thread without taking care of thread-safety. The ADC array content is being changed by the worker while the UI thread is reading it. Various amounts of misery from that, bad data at a minimum. A simply workaround is to pass a copy of the data to the ReportProgress method. In general, address these kind of threading problems by using pull instead of push. Get rid of the fire-hose problem by having the UI thread pace the requests instead of trying to have the UI thread keep up.
I have just gotten started with Arduino and barely have any idea about more of the advanced stuff. It seems pretty straightforward. Now I'm one who usually likes to integrate two devices together, so i was wondering if i could control a servo with the computer's keyboard or two hardware push buttons attached to the Arduino board.
In case it helps, I'm using an Arduino Uno board. Here is the example code i am using to sweep the servo for now
// Sweep
// by BARRAGAN <http://barraganstudio.com>
// This example code is in the public domain.
#include <Servo.h>
Servo myservo; // create servo object to control a servo
// a maximum of eight servo objects can be created
int pos = 0; // variable to store the servo position
void setup()
{
myservo.attach(11); // attaches the servo on pin 9 to the servo object
}
void loop()
{
for(pos = 0; pos < 45; pos += 1) // goes from 0 degrees to 180 degrees
{ // in steps of 1 degree
myservo.write(pos); // tell servo to go to position in variable 'pos'
delay(10); // waits 15ms for the servo to reach the position
}
for(pos = 45; pos>=1; pos-=1) // goes from 180 degrees to 0 degrees
{
myservo.write(pos); // tell servo to go to position in variable 'pos'
delay(10); // waits 15ms for the servo to reach the position
}
}
Now, let's say I wanted to change the servo's angle via pressing the
left/right arrow keys on my computer's keyboard. How would i go
about doing that?
Alternatively, what if i attached two push buttons to the Arduino,
and pressing one would move the servo either left or right depending
on the the button. Which ports would i plug the buttons into? Any
code samples or diagrams would greatly help!
To move a servo attached to an arduino attached to a computer you will need two components.
You will need software on your computer to accept keyboard commands and send commands to the arduino via the serial port. I would recommend a language like python or java to do that as a simple app can written quite easily.
Check this playground link for an example of using Java. And for an example in python check out this project.
There is a bug/feature built into the arduino that will give you grief as you go on here. The arduino is designed to auto reset when a serial connection is made to it via usb. This page has a detailed description of the issue and cites several ways to deal with it.
You will need to modify the sketch on the arduino to listen to the serial port and adjust the servo's position based on the commands received from your computer. Check out the python link above. It is an complete (hardware, pc software and arduino sketch) project designed to do something very similar to what you are trying to do.
I recommend you start with either component and try to get it going. As you run into problems, post your code and someone will be glad to help further.
As for the second question, adding buttons to the arduino is fairly simple. You will connect them to digital inputs. There are hundreds of examples on the web. Search for "add button to arduino" and see what you get. (lol... 1.3 million hits). Here again, try it and post specifics for more help.
For serial communication use putty
it is a cross platform Serial and ssh client
for the left and right arrow commands:
there are no ascii characters for arrow's: but there are utf-8;
putty or an other client sends utf-8 characters for the basic ascii characters are utf-8 and ascii exactly the same;
and the arduino reads only ascii characters;
the arduino reads
--> : 27, 91, 67
<-- : 27, 91, 68
so it is not that simple to read that.
you could use something like this
int pos = 0;
Serial.flush(); // flush all received data
while(Serial.avaialble()<3); // wait for the 3 ascii chars
if(Serial.read()==27){ // first char
if(Serial.read()==91){ //second char
switch (Serial.read()){
case 67: // Right arrow
myservo.write(++pos); // increment pos with 1 before write it
break;
case 68: // left arrow
myservo.write(--pos); // derement pos with 1 before write it
break;
case 65: // up arrow
myservo.write(++pos); // increment pos with 1 before write it
break;
case 66: // down arrow
myservo.write(--pos); // decrement pos with 1 before write it
break;
case default:
break;
}
}
}
but this is not a good solution
because of the arrow character is send in 3 bytes en when you flush it can flush the 27 so you read 91, 97, 27; and that is no valid so in doesn't work
you could write a algorithm to subtract the arrow command out of 5 ascii char's
or you can use 4 to move left and 6 to move right; which are ascii characters and in a numeric keypad are arrows drawn on those keys
I want to write MJPEG picture internet stream viewer. I think getting jpeg images using sockets it's not very hard problem. But i want to know how to make accurate streaming.
while (1)
{
get_image()
show_image()
sleep (SOME_TIME) // how to make it accurate?
}
Any suggestions would be great.
In order to make it accurate, there are two possibilities:
Using framerate from the streaming server. In this case, the client needs to keep the same framerate (calculate each time when you get frame, then show and sleep for a variable amount of time using feedback: if the calculated framerate is higher than on server -> sleep more; if lower -> sleep less; then, the framerate on the client side will drift around the original value from server). It can be received from server during the initialization of streaming connection (when you get picture size and other parameters) or it can be configured.
The most accurate approach, actually, is using of timestamps from server per each frame (which is either taken from file by demuxer or generated in image sensor driver in case of camera device). If MJPEG is packeted into RTP stream, these timestamps are already in RTP header. So, client's task is trivial: show picture using time calculating from time offset, current timestamp and time base.
Update
For the first solution:
time_to_sleep = time_to_sleep_base = 1/framerate;
number_of_frames = 0;
time = current_time();
while (1)
{
get_image();
show_image();
sleep (time_to_sleep);
/* update time to sleep */
number_of_frames++;
cur_time = current_time();
cur_framerate = number_of_frames/(cur_time - time);
if (cur_framerate > framerate)
time_to_sleep += alpha*time_to_sleep;
else
time_to_sleep -= alpha*time_to_sleep;
time = cur_time;
}
, where alpha is a constant parameter of reactivity of the feedback (0.1..0.5) to play with.
However, it's better to organize queue for input images to make the process of showing smoother. The size of queue can be parametrized and could be somewhere around 1 sec time of showing, i.e. numerically equal to framerate.