Confused about Passing user data to PortAudio Callbacks - c

This is my first post here and I'm fairly new to programming and especially with C. A couple weeks ago I started working through the Audio Programming Book(MIT press) and have been expand on some examples to try to understand things further.
I think my question lies with how I'm trying to pass data (retrieved from the user in an initialization function) to a PortAudio callback. I feel like what I've done isn't that different from the examples (both from the book and PortAudio's examples like paex_sine.c), but for some reason I can't my code to work and I've been banging my head against a wall trying to understand why. I've tried searching pretty extensively for solutions or example code to study, but I kind of don't know what I don't know, so that hasn't returned much.
How do I get user data into the callback?
Am I just not understanding how pointers and structs work and trying to force them to do things they don't want to?
Or, am I just overlooking something really obvious?
The following code either gives a really high pitched output, short high pitched blips, or no (audible) output:
#include <stdio.h>
#include <math.h>
#include "portaudio.h"
#define FRAME_BLOCK_LEN 64
#define SAMPLING_RATE 44100
#define TWO_PI (3.14159265f * 2.0f)
PaStream *audioStream;
double si = 0;
typedef struct
{
float frequency;
float phase;
}
paTestData;
int audio_callback (const void *inputBuffer, void *outputBuffer,
unsigned long framesPerBuffer,
const PaStreamCallbackTimeInfo* timeinfo,
PaStreamCallbackFlags statusFlags,
void *userData )
{
paTestData *data = (paTestData*)userData;
float *out = (float*)outputBuffer;
unsigned long i;
// data->frequency = 400;
for(i = 0; i < framesPerBuffer; i++){
si = TWO_PI * data->frequency / SAMPLING_RATE; // calculate sampling-incr
*out++ = sin(data->phase);
*out++ = sin(data->phase);
data->phase += si; // add sampling-incr to phase
}
return paContinue;
}
void init_stuff()
{
float frequency;
int i;
PaStreamParameters outputParameters;
paTestData data;
printf("type the modulator frequency in Hz: ");
scanf("%f", &data.frequency); // get modulator frequency
printf("you chose data.frequency %.2f\n",data.frequency);
data.phase = 0.0;
printf("initializing Portaudio. Please wait...\n");
Pa_Initialize(); // initialize Portaudio
outputParameters.device = Pa_GetDefaultOutputDevice(); /* default output device */
outputParameters.channelCount = 2; /* stereo output */
outputParameters.sampleFormat = paFloat32; /* 32 bit floating point output */
outputParameters.suggestedLatency = Pa_GetDeviceInfo( outputParameters.device )->defaultLowOutputLatency;
outputParameters.hostApiSpecificStreamInfo = NULL;
Pa_OpenStream( // open paStream object
&audioStream, // portaudio stream object
NULL, // input params
&outputParameters, // output params
SAMPLING_RATE, // SampleRate
FRAME_BLOCK_LEN, // frames per buffer
paNoFlag, // set no Flag
audio_callback, // callbak function address
&data ); // user data
Pa_StartStream(audioStream); // start the callback mechanism
printf("running... press space bar and enter to exit\n");
}
void terminate_stuff()
{
Pa_StopStream(audioStream); // stop callback mechanism
Pa_CloseStream(audioStream); // destroy audio stream object
Pa_Terminate(); // terminate portaudio
}
int main(void)
{
init_stuff();
while(getchar() != ' ') Pa_Sleep(100);
terminate_stuff();
return 0;
}
Uncommenting data->frequency = 400; at least plays a 400hz sine wave, but that ignores any user input done in init_stuff()
If I put a printf("%f\n",data->frequency); inside the callback, it prints 0.000000 or something like -146730090609497866240.000000.
It's pretty unpredictable, and this really makes me think it's pointer related.
My goal for this code is to eventually incorporate envelope generators to change the pitch and possibly incorporate wavetable oscillators so I'm not calculating sin(x) for every iteration.
I can get envelopes and wavetables to work while using a blocking API like portsf that's used in the book, but trying to adapt any of that code from earlier chapters to use PortAudio callbacks is turning my brain to mush.
Thanks so much!

The problem you're having with your callback data is that it goes out of scope and memory is deallocated as soon as init_stuff finishes execution.
You should allocate memory for your callback data using malloc or new and passing the pointer to it for the callback.
For example:
void init_stuff()
{
float frequency;
int i;
PaStreamParameters outputParameters;
paTestData *data = (paTestData *) malloc(sizeof(paTestData));
printf("type the modulator frequency in Hz: ");
scanf("%f", &(data->frequency)); // get modulator frequency
printf("you chose data.frequency %.2f\n",data->frequency);
data->phase = 0.0;
...
Pa_OpenStream( // open paStream object
&audioStream, // portaudio stream object
NULL, // input params
&outputParameters, // output params
SAMPLING_RATE, // SampleRate
FRAME_BLOCK_LEN, // frames per buffer
paNoFlag, // set no Flag
audio_callback, // callbak function address
data );
...

I wasn't able to get the original code working using malloc but based on both suggestions, I realized another workable solution. Because running init_stuff() caused my data to get deallocated, I'm for now just making all my assignments and calls to Pa_OpenStream() from main.
Works beautifully and I can now send whatever data I want to the callback. Thanks for the help!

Related

Can't run GPU_FFT in an infinite loop

I am currently trying to take FFT using GPU_FFT libraries provided with Pi. I've successfully taken FFT of the data. However, whenever I try this in a while(1) loop, somehow it takes FFT 1023 times and stops at 1024th. I've read about someone who faced with exact same issue on Raspberry Pi official forum, but couldn't figure out how to fix mine.
The code below is function to compute FFT
void computeFFT(float *Input,float *Output){
struct GPU_FFT_COMPLEX *base;
struct GPU_FFT *fft;
int i;
int ret = gpu_fft_prepare(mb,log2_N,GPU_FFT_FWD,1,&fft);
base = fft->in;
for(i=0;i<N;i++)
base[i].re = Input[i]; base[i].im =0.0;
gpu_fft_execute(fft);
base = fft->out;
for(i=0;i<N;i++)
Output[i]= base[i].re;
gpu_fft_release(fft);
}
and this is my while(1) loop
while (1){
if (mb != 0)
mb = mbox_open();
adc1Value = a2d(adc1Channel);
voltage1 = ((2.5/4096) * adc1Value);
QueueGet(Queue);
QueuePut(voltage1);
OutData = computeFFT(Queue,OutData);
printf("%f\n",OutData[0]); // Printing out signal value on 0Hz
}
mbox_close(mb);
free(OutData);
return 0;
I do believe problem is related to where I use mbox_open and close functions.
Things that I tried
Changing Queue into new array with random number generated with rand().
Changing location of mbox functions as suggested.
Disabling a2d function
Changing FFT length to 512, 2048, 4096.
Yet none of them was the solution.

Audio samples producer multiple threads OSX

This question is a follow-up to a former question (Audio producer threads with OSX AudioComponent consumer thread and callback in C), including a test example, which works and behaves as expected but does not quite answer the question. I have substantially rephrased the question, and re-coded the example, so that it only contains plain-C code. (I've found out that few Objective-C portions of code in the former example only caused confusion and distracted the reader from what's essential in the question.)
In order to take advantage of multiple processor cores as well as to make the CoreAudio pull-model render thread as lightweight as possible, the LPCM samples' producer routine clearly has to "sit" on a different thread, outside the real-lime-priority render thread/callback. It must feed the samples to a circular buffer (TPCircularBuffer in this example), from which the system would schedule data pull-out in quants of inNumberFrames.
The Grand Central Dispatch API offers a simple solution, which I've deduced upon some individual research (including trial-and-error coding). This solution is elegant, since it doesn't block anything nor conflict between push and pull models. Yet the GCD, which is supposed to take care of "sub-threading" does not by far meet the specific parallelization requirements for the work threads of the producer code, so I had to explicitely spawn a number of POSIX threads, depending on the number of logical cores available. Although results are already remarkable in terms of speeding-up the computation I still feel a bit unconfortable mixing the POSIX and GCD. In particular it goes for the variable wait_interval, and computing it properly, not by predicting how many PCM samples may the render thread require for the next cycle.
Here's the shortened and simplified (pseudo)code for my test program, in plain-C.
Controller declaration:
#include "TPCircularBuffer.h"
#include <AudioToolbox/AudioToolbox.h>
#include <AudioUnit/AudioUnit.h>
#include <dispatch/dispatch.h>
#include <sys/sysctl.h>
#include <pthread.h>
typedef struct {
TPCircularBuffer buffer;
AudioComponentInstance toneUnit;
Float64 sampleRate;
AudioStreamBasicDescription streamFormat;
Float32* f; //array of updated frequencies
Float32* a; //array of updated amps
Float32* prevf; //array of prev. frequencies
Float32* preva; //array of prev. amps
Float32* val;
int* arg;
int* previous_arg;
UInt32 frames;
int state;
Boolean midif; //wip
} MyAudioController;
MyAudioController gen;
dispatch_semaphore_t mSemaphore;
Boolean multithreading, NF;
typedef struct data{
int tid;
int cpuCount;
}data;
Controller management:
void setup (void){
// Initialize circular buffer
TPCircularBufferInit(&(self->buffer), kBufferLength);
// Create the semaphore
mSemaphore = dispatch_semaphore_create(0);
// Setup audio
createToneUnit(&gen);
}
void dealloc (void) {
// Release buffer resources
TPCircularBufferCleanup(&buffer);
// Clean up semaphore
dispatch_release(mSemaphore);
// dispose of audio
if(gen.toneUnit){
AudioOutputUnitStop(gen.toneUnit);
AudioUnitUninitialize(gen.toneUnit);
AudioComponentInstanceDispose(gen.toneUnit);
}
}
Dispatcher call (launching producer queue from the main thread):
void dproducer (Boolean on, Boolean multithreading, Boolean NF)
{
if (on == true)
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
if((multithreading)||(NF))
producerSum(on);
else
producer(on);
});
}
return;
}
Threadable producer routine:
void producerSum(Boolean on)
{
int rc;
int num = getCPUnum();
pthread_t threads[num];
data thread_args[num];
void* resulT;
static Float32 frames [FR_MAX];
Float32 wait_interval;
int bytesToCopy;
Float32 floatmax;
while(on){
wait_interval = FACT*(gen.frames)/(gen.sampleRate);
Float32 damp = 1./(Float32)(gen.frames);
bytesToCopy = gen.frames*sizeof(Float32);
memset(frames, 0, FR_MAX*sizeof(Float32));
availableBytes = 0;
fbuffW = (Float32**)calloc(num + 1, sizeof(Float32*));
for (int i=0; i<num; ++i)
{
fbuffW[i] = (Float32*)calloc(gen.frames, sizeof(Float32));
thread_args[i].tid = i;
thread_args[i].cpuCount = num;
rc = pthread_create(&threads[i], NULL, producerTN, (void *) &thread_args[i]);
}
for (int i=0; i<num; ++i) rc = pthread_join(threads[i], &resulT);
for(UInt32 samp = 0; samp < gen.frames; samp++)
for(int i = 0; i < num; i++)
frames[samp] += fbuffW[i][samp];
//code for managing producer state and GUI updates
{ ... }
float *head = TPCircularBufferHead(&(gen.buffer), &availableBytes);
memcpy(head,(const void*)frames,MIN(bytesToCopy, availableBytes));//copies frames to head
TPCircularBufferProduce(&(gen.buffer),MIN(bytesToCopy,availableBytes));
dispatch_semaphore_wait(mSemaphore, dispatch_time(DISPATCH_TIME_NOW, wait_interval * NSEC_PER_SEC));
if(gen.state == stopped){gen.state = idle; on = false;}
for(int i = 0; i <= num; i++)
free(fbuffW[i]);
free(fbuffW);
}
return;
}
A single producer thread may look somewhat like this:
void *producerT (void *TN)
{
Float32 samples[FR_MAX];
data threadData;
threadData = *((data *)TN);
int tid = threadData.tid;
int step = threadData.cpuCount;
int *ret = calloc(1,sizeof(int));
do_something(tid, step, &samples);
{ … }
return (void*)ret;
}
Here is the render callback (CoreAudio real-time consumer thread):
static OSStatus audioRenderCallback(void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData) {
MyAudioController *THIS = (MyAudioController *)inRefCon;
// An event happens in the render thread- signal whoever is waiting
if (THIS->state == active) dispatch_semaphore_signal(mSemaphore);
// Mono audio rendering: we only need one target buffer
const int channel = 0;
Float32* targetBuffer = (Float32 *)ioData->mBuffers[channel].mData;
memset(targetBuffer,0,inNumberFrames*sizeof(Float32));
// Pull samples from circular buffer
int32_t availableBytes;
Float32 *buffer = TPCircularBufferTail(&THIS->buffer, &availableBytes);
//copy circularBuffer content to target buffer
int bytesToCopy = ioData->mBuffers[channel].mDataByteSize;
memcpy(targetBuffer, buffer, MIN(bytesToCopy, availableBytes));
{ … };
TPCircularBufferConsume(&THIS->buffer, availableBytes);
THIS->frames = inNumberFrames;
return noErr;
}
Grand Central Dispatch already takes care of dispatching operations to multiple processor cores and threads. In typical real-time audio rendering or processing, one never needs to wait on a signal or semaphore, as the circular buffer consumption rate is very predictable, and drifts extremely slowly over time. The AVAudioSession API (if available) and Audio Unit API and callback allow you to set and determine the callback buffer size, and thus the maximum rate at which the circular buffer can change. Thus you can dispatch all render operations on a timer, render the exact number needed per timer period, and let the buffer size and state compensate for any jitter in thread dispatch time.
In extremely long running audio renders, you might want to measure the drift between timer operations and real-time audio consumption (sample rate), and tweak the number of samples rendered or the timer offset.

Issue with SPI (Serial Port Comm), stuck on ioctl()

I'm trying to access a SPI sensor using the SPIDEV driver but my code gets stuck on IOCTL.
I'm running embedded Linux on the SAM9X5EK (mounting AT91SAM9G25). The device is connected to SPI0. I enabled CONFIG_SPI_SPIDEV and CONFIG_SPI_ATMEL in menuconfig and added the proper code to the BSP file:
static struct spi_board_info spidev_board_info[] {
{
.modalias = "spidev",
.max_speed_hz = 1000000,
.bus_num = 0,
.chips_select = 0,
.mode = SPI_MODE_3,
},
...
};
spi_register_board_info(spidev_board_info, ARRAY_SIZE(spidev_board_info));
1MHz is the maximum accepted by the sensor, I tried 500kHz but I get an error during Linux boot (too slow apparently). .bus_num and .chips_select should correct (I also tried all other combinations). SPI_MODE_3 I checked the datasheet for it.
I get no error while booting and devices appear correctly as /dev/spidevX.X. I manage to open the file and obtain a valid file descriptor. I'm now trying to access the device with the following code (inspired by examples I found online).
#define MY_SPIDEV_DELAY_USECS 100
// #define MY_SPIDEV_SPEED_HZ 1000000
#define MY_SPIDEV_BITS_PER_WORD 8
int spidevReadRegister(int fd,
unsigned int num_out_bytes,
unsigned char *out_buffer,
unsigned int num_in_bytes,
unsigned char *in_buffer)
{
struct spi_ioc_transfer mesg[2] = { {0}, };
uint8_t num_tr = 0;
int ret;
// Write data
mesg[0].tx_buf = (unsigned long)out_buffer;
mesg[0].rx_buf = (unsigned long)NULL;
mesg[0].len = num_out_bytes;
// mesg[0].delay_usecs = MY_SPIDEV_DELAY_USECS,
// mesg[0].speed_hz = MY_SPIDEV_SPEED_HZ;
mesg[0].bits_per_word = MY_SPIDEV_BITS_PER_WORD;
mesg[0].cs_change = 0;
num_tr++;
// Read data
mesg[1].tx_buf = (unsigned long)NULL;
mesg[1].rx_buf = (unsigned long)in_buffer;
mesg[1].len = num_in_bytes;
// mesg[1].delay_usecs = MY_SPIDEV_DELAY_USECS,
// mesg[1].speed_hz = MY_SPIDEV_SPEED_HZ;
mesg[1].bits_per_word = MY_SPIDEV_BITS_PER_WORD;
mesg[1].cs_change = 1;
num_tr++;
// Do the actual transmission
if(num_tr > 0)
{
ret = ioctl(fd, SPI_IOC_MESSAGE(num_tr), mesg);
if(ret == -1)
{
printf("Error: %d\n", errno);
return -1;
}
}
return 0;
}
Then I'm using this function:
#define OPTICAL_SENSOR_ADDR "/dev/spidev0.0"
...
int fd;
fd = open(OPTICAL_SENSOR_ADDR, O_RDWR);
if (fd<=0) {
printf("Device not found\n");
exit(1);
}
uint8_t buffer1[1] = {0x3a};
uint8_t buffer2[1] = {0};
spidevReadRegister(fd, 1, buffer1, 1, buffer2);
When I run it, the code get stuck on IOCTL!
I did this way because, in order to read a register on the sensor, I need to send a byte with its address in it and then get the answer back without changing CS (however, when I tried using write() and read() functions, while learning, I got the same result, stuck on them).
I'm aware that specifying .speed_hz causes a ENOPROTOOPT error on Atmel (I checked spidev.c) so I commented that part.
Why does it get stuck? I though it can be as the device is created but it actually doesn't "feel" any hardware. As I wasn't sure if hardware SPI0 corresponded to bus_num 0 or 1, I tried both, but still no success (btw, which one is it?).
UPDATE: I managed to have the SPI working! Half of it.. MOSI is transmitting the right data, but CLK doesn't start... any idea?
When I'm working with SPI I always use an oscyloscope to see the output of the io's. If you have a 4 channel scope ypu can easily debug the issue, and find out if you're axcessing the right io's, using the right speed, etc. I usually compare the signal I get to the datasheet diagram.
I think there are several issues here. First of all SPI is bidirectional. So if yo want to send something over the bus you also get something. Therefor always you have to provide a valid buffer to rx_buf and tx_buf.
Second, all members of the struct spi_ioc_transfer have to be initialized with a valid value. Otherwise they just point to some memory address and the underlying process is accessing arbitrary data, thus leading to unknown behavior.
Third, why do you use a for loop with ioctl? You already tell ioctl you haven an array of spi_ioc_transfer structs. So all defined transaction will be performed with one ioctl call.
Fourth ioctl needs a pointer to your struct array. So ioctl should look like this:
ret = ioctl(fd, SPI_IOC_MESSAGE(num_tr), &mesg);
You see there is room for improvement in your code.
This is how I do it in a c++ library for the raspberry pi. The whole library will soon be on github. I'll update my answer when it is done.
void SPIBus::spiReadWrite(std::vector<std::vector<uint8_t> > &data, uint32_t speed,
uint16_t delay, uint8_t bitsPerWord, uint8_t cs_change)
{
struct spi_ioc_transfer transfer[data.size()];
int i = 0;
for (std::vector<uint8_t> &d : data)
{
//see <linux/spi/spidev.h> for details!
transfer[i].tx_buf = reinterpret_cast<__u64>(d.data());
transfer[i].rx_buf = reinterpret_cast<__u64>(d.data());
transfer[i].len = d.size(); //number of bytes in vector
transfer[i].speed_hz = speed;
transfer[i].delay_usecs = delay;
transfer[i].bits_per_word = bitsPerWord;
transfer[i].cs_change = cs_change;
i++
}
int status = ioctl(this->fileDescriptor, SPI_IOC_MESSAGE(data.size()), &transfer);
if (status < 0)
{
std::string errMessage(strerror(errno));
throw std::runtime_error("Failed to do full duplex read/write operation "
"on SPI Bus " + this->deviceNode + ". Error message: " +
errMessage);
}
}

Audio samplerate converter using libsndfile and libsamplerate. Not sure if using function src_simple correctly

I have been building a simple samplerate converter in c using libsndfile and libsamplerate. I just cant seem to get the src_simple function of libsamplerate to work, whatever I try. I have striped back my code to be as simple as possible and it now just outputs a silent audio file of identical sampling rate:
#include <stdio.h>
#include <sndfile.h>
#include <samplerate.h>
#define BUFFER_LEN 1024
#define MAX_CHANNELS 6
int main ()
{
static double datain [BUFFER_LEN];
static double dataout [BUFFER_LEN];
SNDFILE *infile, *outfile;
SF_INFO sfinfo, sfinfo2 ;
int readcount ;
const char *infilename = "C:/Users/Oli/Desktop/MARTYTHM.wav" ;
const char *outfilename = "C:/Users/Oli/Desktop/Done.wav" ;
SRC_DATA src_data;
infile = sf_open (infilename, SFM_READ, &sfinfo);
outfile = sf_open (outfilename, SFM_WRITE, &sfinfo);
src_data.data_in = datain
src_data.input_frames = BUFFER_LEN;
src_data.data_out = dataout;
src_data.output_frames = BUFFER_LEN;
src_data.src_ratio = 0.5;
src_simple (&src_data, SRC_SINC_BEST_QUALITY, 1);
while ((readcount = sf_read_double (infile, datain, BUFFER_LEN)))
{
src_simple (&src_data, SRC_SINC_BEST_QUALITY, 1);
sf_write_double (outfile, dataout, readcount) ;
};
sf_close (infile);
sf_close (outfile);
sf_open ("C:/Users/Oli/Desktop/Done.wav", SFM_READ, &sfinfo2);
printf("%d", sfinfo2.samplerate);
return 0;
}
It's really starting to stress me out. The program is a uni project and is due very soon, it is making me very anxious as whatever I try seems to result in failure. Can anyone please help me?
I'm not an expert on this particular library, but just from looking at the online documentation I see a few problems with your code:
src_simple apparently works with floats, yet your buffers are doubles - I think you need to change the buffers to float and use sf_read_float/sf_write_float for I/O.
src_simple is the "simple" interface and is intended to be applied to an entire waveform in one call, not in chunks as you are doing - see http://www.mega-nerd.com/SRC/faq.html#Q004 - you should first get the input file size, then allocate sufficient memory, read in the whole file, convert it in one go, then write the converted output data to your output file.
when changing sample rate you will get a different number of samples in the output file than in the output file (around half as many in for case), yet you're writing the same number of samples that you read (readcount). You should probably be using src_data.output_frames_gen as the number of frames to write, not readcount.

Arduino conversions in C

void digitalClockDisplay(){
// digital clock display of the time
Serial.print(hour());
printDigits(minute());
Serial.println();
}
void printDigits(int digits){
// utility function for digital clock display: prints preceding colon and leading 0
Serial.print(":");
if(digits < 10)
Serial.print('0');
Serial.print(digits);
}
//I tried something like this
//void time(){
//char* hr = (char*)hour();
//Serial.println(hr);
//}
//But when I print it it gives a whole bunch of jibberish
Here are the two functions I'm using what I'm trying to do is make a function like the digitalClockDisplay function but one that returns the hour:minute as a char* once I have that I want to be able to compare that to another char*
hour() seems to be returning a int, so
char* hr = (char*)hour();
Serial.println(hr);
casts a int to a pointer and then sends the bytes at that (meaningless) address to Serial.
You probably want something like:
char hr[8];
snprintf(hr,8,"%i:%02i",hour(),minute());
Serial.println(hr);

Resources