When porting from NAOqi to qi framework I achieved a partial success. I do however still have the following problem.
I do not know how to implement sound processing with ALSoundExtractor in qi framework.
In old Naoqi, there is an example:
http://doc.aldebaran.com/2-8/dev/cpp/examples/audio/soundprocessing/soundprocessing.html
where a class is created:
class ALSoundProcessing : public ALSoundExtractor
then a function overriding a virtual function is declared, one that is used for sound processing:
void process(...)
What I don't now is:
How to create a class in qi framework that inherits from the old style class ALSoundExtractor?
How to declare a function that is overriding the virtual function - technically the base class function process() expects variables in old AL:: convention.
Alternatively, is there any other way to read the audio channels?
I never worked with ALExtractor nor ALSoundExtractor, but here is what I know.
How to create a class in qi framework that inherits from the old style class ALSoundExtractor?
in the old Naoqi, an "ALExtractor"
could run either from within the main process (using autoload.ini) or from another one (known as remote mode). With the qi framework, only the remote mode is supported.
could inherit from ALExtractor or ALAudioExtractor to get some code factored out. Those classes have not been ported to the qi framework. So if you don't want to keep using libnaoqi, you should find a way to do without them.
Good news: inheriting from them never was really needed. You'll find yourself in a similar position as in the following question where an extractor is implemented in python (and thus cannot inherit from a C++ class, nor be loaded in the main process from autoload.ini).
NAO robot remote audio problems
How to declare a function that is overriding the virtual function - technically the base class function process() expects variables in old AL:: convention.
Whenever you use the "old Naoqi" you're actually using a compatibility layer on top of the qi framework.
So whenever you use the "old Naoqi", you're already using the qi framework.
libqi's qi::AnyValue is extensible at runtime, libnaoqi extends it to let it know how to handle an ALValue: how to convert it into primitive types (floating point number, list of ints, string, buffer, etc.).
So whenever an old ALSoundExtractor receives an AL::ALvalue, it is actually a qi::AnyValue which has been converted into an ALValue just before calling the process() method.
If you don't link with libnaoqi, you won't be able to use the value as an ALValue, but you can use it as a qi::AnyValue or even use it as a primitive type.
The original prototype is (cfr doxygen http://doc.aldebaran.com/2-8/ref/libalaudio/classAL_1_1ALSoundExtractor.html) is
void ALSoundExtractor::process (const int &nbOfChannels, const int &nbrOfSamplesByChannel, const AL_SOUND_FORMAT *buffer, const ALValue ×tamp);
Since timestamp is probably a list of two ints, I would try something like this
void TmpSoundExtractor::process (const int &nbOfChannels, const int &nbrOfSamplesByChannel, qi::AnyValue buffer, const std::vector<int> ×tamp);
I'm not sure how to handle the buffer variable, but let first get the rest working.
To use this API, you must write a Qi Service that advertises this method:
void processRemote(
int nbOfChannels,
int nbrOfSamplesByChannel,
const qi::AnyValue& timestamp,
const qi::AnyValue& buffer)
{
std::pair<char*, size_t> charBuffer = value.unwrap().asRaw();
const signed short* data = (const signed short*)charBuffer.first;
// process the data like in the example.
}
Note that with the Qi framework:
AL::ALValue is replaced by qi::AnyValue.
Getting the binary data (aka "raw") is slightly different.
AL_SOUND_FORMAT is replaced by signed short*.
ALSoundExtractor is not available, so we needed to do the conversion to const AL_SOUND_FORMAT* by ourselves.
Say your service is registered as "MySoundExtractor", you will have to tell ALAudioDevice to start the sound extraction and send the data to your service as follows:
auto audio = session->service("ALAudioDevice").value();
int nNbrChannelFlag = 0; // ALL_Channels: 0, AL::LEFTCHANNEL: 1, AL::RIGHTCHANNEL: 2; AL::FRONTCHANNEL: 3 or AL::REARCHANNEL: 4.
int nDeinterleave = 0;
int nSampleRate = 48000;
audio->setClientPreferences("MySoundExtractor", nSampleRate, nNbrChannelFlag, nDeinterleave);
audio->subscribe("MySoundExtractor");
Note that I did not test this code, so let me know what may be wrong.
The following is what has eventually worked for me and concludes the topic.
// **************** service.h ****************
typedef signed short AL_SOUND_FORMAT; // copy from alaudio/alsoundextractor.h
class SoundProcessing
{
public:
SoundProcessing(qi::SessionPtr session);
void init(void); // a replacement for a function automatically called in NAOqi 2.1.4
virtual ~SoundProcessing(void);
void processRemote(const int& nbOfChannels, const int& nbrOfSamplesByChannel, const qi::AnyValue& timestamp, const qi::AnyValue& buffer);
private:
qi::SessionPtr _session;
qi::AnyObject audio;
};
// **************** service.cpp ****************
SoundProcessing::SoundProcessing(qi::SessionPtr session) : _session(session)
{
_session->waitForService("ALAudioDevice");
audio = _session->service("ALAudioDevice");
} // constructor
QI_REGISTER_MT_OBJECT(SoundProcessing, init, processRemote);
SoundProcessing::~SoundProcessing(void)
{
audio.call<qi::AnyValue>("unsubscribe", "SoundProcessing");
} // destructor
void SoundProcessing::init(void)
{
audio.call<qi::AnyValue>("setClientPreferences",
"SoundProcessing",
_FREQ48K, // 48000 Hz requested
0,
1
);
audio.call<qi::AnyValue>("subscribe", "SoundProcessing");
} // SoundProcessing::init
void SoundProcessing::processRemote(const int& nbOfChannels,const int& nbrOfSamplesByChannel, const qi::AnyValue& timestamp, const qi::AnyValue& qibuffer)
{
std::pair<char*, size_t> charBuffer = qibuffer.unwrap().asRaw();
AL_SOUND_FORMAT *buffer = (AL_SOUND_FORMAT *)charBuffer.first;
(...)
} // SoundProcessing::process
// **************** main.cpp ****************
int main(int argc, char* argv[])
{
qi::ApplicationSession app(argc, argv);
app.start();
qi::SessionPtr session = app.session();
session->registerService("SoundProcessing", qi::AnyObject(boost::make_shared<SoundProcessing>(session)));
qi::AnyObject sp = session->service("SoundProcessing");
sp.call<qi::AnyValue>("init");
app.run();
return 0;
}
The following is what I did. The code compiles, but I won't have a chance to test it on a live robot for about one week or so.
typedef signed short AL_SOUND_FORMAT; // copy from alaudio/alsoundextractor.h
void process(const int& nbOfChannels, const int& nbrOfSamplesByChannel, const AL_SOUND_FORMAT *buffer, const qi::AnyValue& timeStamp); // I do not use the timeStamp variable in my code, so AnyValue would work?
qi::AnyObject audioDevice = _session->service("ALAudioDevice"); // same variable name as in the original ALSoundExtractor module, just as a convenience
audioDevice.call<qi::AnyValue>("setClientPreferences", audioDevice.call<qi::AnyValue>("getName"), 48000, 0, 1);
audioDevice.call<qi::AnyValue>("subscribe", audioDevice.call<qi::AnyValue>("getName")); // this is the key call
audioDevice.call<qi::AnyValue>("startDetection"); // is it still necessary?
My question is - do I do it right now? If I cannot override the virtual function process(), does subscribing of my module guarantee a callback to my process(...)?
I am trying to wrap a library for porting purpose. The library exposes a function say -
fooLib(int , char , function pointer A);
signature of the function pointer A is
void handler(DataFormat);
where DataFormat is a struct
I don't want my wrapper to expose this library's callback function. I want to create a different function that should be used by the consumers of my wrapper, say
int handlerNew(NewDataFormat);
where NewDataFormat is my struct
The question now is how can I link these two functions? Whenever the library calls handler I want it to call my callback handlerNew after filling up the NewDataFormat struct from the DataFormat.
As long as you don't need thread safety, this is not hard. You just have to provide a private (static) handler with the library's interface that transforms the library data struct into your wrapped version, then calls your callback with that as an argument. Your interface will look like:
// wrapped_foo_lib.h
typedef struct { ... } NewDataFormat;
typedef void (*WRAPPED_CALLBACK)(NewDataFormat);
void wrappedFooLibCall(int x, char c, WRAPPED_CALLBACK cb);
Your implementation, which the client never gets to see is:
// wrapped_foo_lib.c
// This static var makes this module _not_ thread safe.
static WRAPPED_CALLBACK wrapped_callback;
static void private_handler(DataFormat data) {
NewDataFormat new_data = ...; // extract new_data from data
wrapped_callback(new_data);
}
void wrappedFooLibCall(int x, char c, WRAPPED_CALLBACK cb) {
wrapped_callback = cb;
foo_lib(x, c, private_handler);
}
The non-thread safety is why every API callback should include a void * that you get to define, which is passed on to the callback. I.e. your furnished library should be defined as
fooLib(int, char, void (*)(DataFormat, void *env));
void handler(DataFormat, void *env);
Now when you call fooLib, you furnish any struct at all as env, and it's passed back to you. This way you can dispense with the static variable in the wrapper:
// wrapped_foo_lib.c
typedef struct { WRAPPED_CALLBACK wrapped_callback; } ENV;
static void private_handler(DataFormat data, void *void_env) {
ENV *env = (ENV*)void_env;
NewDataFormat new_data = ...; // extract new_data from data
env->wrapped_callback(new_data);
}
void wrappedFooLibCall(int x, char c, WRAPPED_CALLBACK cb) {
ENV env[1] = {{ cb }};
foo_lib(x, c, env);
}
This is thread safe because ENV is stack allocated. A nice example of this done well is the libpng.
Feel free to update the C90 to more modern syntax.
So first I load in a DLL I need
local ffi = require("ffi")
local theDLL = ffi.load("thisDLL")
in the ffi cdef I have this struct
ffi.cdef [[
typedef struct {
/*
* begin_proj callback
*/
bool (__cdecl *begin_proj)(char *proj);
/*
* save_proj_state
*/
bool (__cdecl *save_proj_state)(unsigned char **buffer, int *len);
} StructCallbacks;
I also have this function in the cdef
__declspec(dllexport) int __cdecl start_session(StructCallbacks *cb);
Now I would like to call this function
print(theDLL.start_session(myCallbacks))
the question is how can I pass the structs the function needs (how do I make myCallbacks a struct of callbacks to Lua functions)?
Just create the struct and assign the fields to Lua functions, as you would with any other value.
local callbacks = ffi.new("StructCallbacks")
callbacks.begin_proj = function(proj) return false end
callbacks.save_proj_state = function(buffer, len) return true end
See the FFI callback docs for more in-depth information on callbacks.
How do I pass a delegate to an external C function taking a function pointer, in D?
Let me cross post what I said on the newsgroup:
How do I pass a delegate to an external C function taking a
function pointer?
You can't do it directly in general, unless you can modify the C
function, then you can hack around it, but a delegate and a
regular function pointer are pretty different animals.
But perhaps you can magic hack it. Observe:
// a C function that needs a plain function
extern(C) void test(void function() f) {
// pretend this is implemented in C
f();
}
// just create a random delegate
void delegate() foo(int a) {
return { import std.stdio; writeln(a); };
}
// what we want to work
void main() {
auto dg = foo(10);
dg(); // works
//test(dg); // won't work
test(bindDelegate(dg)); // we want this
}
// transform delegate into pointer..
import std.traits;
auto bindDelegate(T, string file = __FILE__, size_t line = __LINE__)(T t) if(isDelegate!T) {
static T dg;
dg = t;
extern(C)
static ReturnType!T func(ParameterTypeTuple!T args) {
return dg(args);
}
return &func;
}
What bindDelegate does is create a special static variable and
function for that specific call. It is as if we wrote a separate
function and global to hold it.
The __FILE__, __LINE__ things are a filthy hack to make it
instantiate a separate variable+function pair for different
lines so the global variable holding the delegate won't be so
easily overwritten.
Suppose there is a library function (can not modify) that accept a callback (function pointer) as its argument which will be called at some point in the future. My question: is there a way to store extra data along with the function pointer, so that when the callback is called, the extra data can be retrieved. The program is in c.
For example:
// callback's type, no argument
typedef void (*callback_t)();
// the library function
void regist_callback(callback_t cb);
// store data with the function pointer
callback_t store_data(callback_t cb, int data);
// retrieve data within the callback
int retrieve_data();
void my_callback() {
int a;
a = retrieve_data();
// do something with a ...
}
int my_func(...) {
// some variables that i want to pass to my_callback
int a;
// ... regist_callback may be called multiple times
regist_callback(store_data(my_callback, a));
// ...
}
The problem is because callback_t accept no argument. My idea is to generate a small piece of asm code each time to fill into regist_callback, when it is called, it can find the real callback and its data and store it on the stack (or some unused register), then jump to the real callback, and inside the callback, the data can be found.
pseudocode:
typedef struct {
// some asm code knows the following is the real callback
char trampoline_code[X];
callback_t real_callback;
int data;
} func_ptr_t;
callback_t store_data(callback_t cb, int data) {
// ... malloc a func_ptr_t
func_ptr_t * fpt = malloc(...);
// fill the trampoline_code, different machine and
// different calling conversion are different
// ...
fpt->real_callback = cb;
fpt->data = data;
return (callback_t)fpt;
}
int retrieve_data() {
// ... some asm code to retrive data on stack (or some register)
// and return
}
Is it reasonable? Is there any previous work done for such problem?
Unfortunately you're likely to be prohibited from executing your trampoline in more and more systems as time goes on, as executing data is a pretty common way of exploiting security vulnerabilities.
I'd start by reporting the bug to the author of the library. Everybody should know better than to offer a callback interface with no private data parameter.
Having such a limitation would make me think twice about how whether or not the library is reentrant. I would suggest ensuring you can only have one call outstanding at a time, and store the callback parameter in a global variable.
If you believe that the library is fit for use, then you could extend this by writing n different callback trampolines, each referring to their own global data, and wrap that up in some management API.