Nanopb without callbacks - c

I'm using Nanopb to try and send protobuf messages from a VxWorks based National Instruments Compact RIO (9025). My cross compilation works great, and I can even send a complete message with data types that don't require extra encoding. What's getting me is the callbacks. My code is cross compiled and called from LabVIEW and the callback based structure of Nanopb seems to break (error out, crash, target reboots, whatever) on the target machine. If I run it without any callbacks it works great.
Here is the code in question:
bool encode_string(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
{
char *str = "Woo hoo!";
if (!pb_encode_tag_for_field(stream, field))
return false;
return pb_encode_string(stream, (uint8_t*)str, strlen(str));
}
extern "C" uint16_t getPacket(uint8_t* packet)
{
uint8_t buffer[256];
uint16_t packetSize;
ExampleMsg msg = {};
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
msg.name.funcs.encode = &encode_string;
msg.value = 17;
msg.number = 18;
pb_encode(&stream, ExampleMsg_fields, &msg);
packetSize = stream.bytes_written;
memcpy(packet, buffer, 256);
return packetSize;
}
And here's the proto file:
syntax = "proto2"
message ExampleMsg {
required int32 value = 1;
required int32 number = 2;
required string name = 3;
}
I have tried making the callback an extern "C" as well and it didn't change anything. I've also tried adding a nanopb options file with a max length and either didn't understand it correctly or it didn't work either.
If I remove the string from the proto message and remove the callback, it works great. It seems like the callback structure is not going to work in this LabVIEW -> C library environment. Is there another way I can encode the message without the callback structure? Or somehow embed the callback into the getPacket() function?
Updated code:
extern "C" uint16_t getPacket(uint8_t* packet)
{
uint8_t buffer[256];
for (unsigned int i = 0; i < 256; ++i)
buffer[i] = 0;
uint16_t packetSize;
ExampleMsg msg = {};
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
msg.name.funcs.encode = &encode_string;
msg.value = 17;
msg.number = 18;
char name[] = "Woo hoo!";
strncpy(msg.name, name, strlen(name));
pb_encode(&stream, ExampleMsg_fields, &msg);
packetSize = stream.bytes_written;
memcpy(packet, buffer, sizeof(buffer));
return packetSize;
}
Updated proto file:
syntax = "proto2"
import "nanopb.proto";
message ExampleMsg {
required int32 value = 1;
required int32 number = 2;
required string name = 3 [(nanopb).max_size = 40];
}

You can avoid callbacks by giving a maximum size for the string field using the option (nanopb).max_size = 123 in the .proto file. Then nanopb can generate a simple char array in the structure (relevant part of documentation).
Regarding why callbacks don't work: just a guess, but try adding extern "C" also to the callback function. I assume you are using C++ there, so perhaps on that platform the C and C++ calling conventions differ and that causes the crash.
Does the VxWorks serial console give any more information about the crash? I don't remember if it does that for functions called from LabView, so running some test code directly from the VxWorks shell may be worth a try also.

Perhaps the first hurdle is how the code handles strings.
LabVIEW's native string representation is not null-terminated like C, but you can configure LabVIEW to use a different representation or update your code to handle LabVIEW's native format.
LabVIEW stores a string in a special format in which the first four bytes of the array of characters form a 32-bit signed integer that stores how many characters appear in the string. Thus, a string with n characters requires n + 4 bytes to store in memory.
LabVIEW Help: Using Arrays and Strings in the Call Library Function Node
http://zone.ni.com/reference/en-XX/help/371361L-01/lvexcodeconcepts/array_and_string_options/

Related

FFMPEG remux sample without writing to file

Let's consider this very nice and easy to use remux sample by horgh.
I'd like to achieve the same task: convert an RTSP H264 encoded stream to a fragmented MP4 stream.
This code does exactly this task.
However I don't want to write the mp4 onto disk at all, but I need to get a byte buffer or array in C with the contents that would normally written to disk.
How is that achievable?
This sample uses vs_open_output to define the output format and this function needs an output url.
If I would get rid of outputting the contents to disk, how shall I modify this code?
Or there might be better alternatives as well, those are also welcomed.
Update:
As szatmary recommended, I have checked his example link.
However as I stated in the question I need the output as buffer instead of a file.
This example demonstrates nicely how can I read my custom source and give it to ffmpeg.
What I need is how can open the input as standard (with avformat_open_input) then do my custom modification with the packets and then instead writing to file, write to a buffer.
What have I tried?
Based on szatmary's example I created some buffers and initialization:
uint8_t *buffer;
buffer = (uint8_t *)av_malloc(4096);
format_ctx = avformat_alloc_context();
format_ctx->pb = avio_alloc_context(
buffer, 4096, // internal buffer and its size
1, // write flag (1=true, 0=false)
opaque, // user data, will be passed to our callback functions
0, // no read
&IOWriteFunc,
&IOSeekFunc
);
format_ctx->flags |= AVFMT_FLAG_CUSTOM_IO;
AVOutputFormat * const output_format = av_guess_format("mp4", NULL, NULL);
format_ctx->oformat = output_format;
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
Then of course I have created 'IOWriteFunc' and 'IOSeekFunc':
static int IOWriteFunc(void *opaque, uint8_t *buf, int buf_size) {
printf("Bytes read: %d\n", buf_size);
int len = buf_size;
return (int)len;
}
static int64_t IOSeekFunc (void *opaque, int64_t offset, int whence) {
switch(whence){
case SEEK_SET:
return 1;
break;
case SEEK_CUR:
return 1;
break;
case SEEK_END:
return 1;
break;
case AVSEEK_SIZE:
return 4096;
break;
default:
return -1;
}
return 1;
}
Then I need to write the header to the output buffer, and the expected behaviour here is to print "Bytes read: x":
AVDictionary * opts = NULL;
av_dict_set(&opts, "movflags", "frag_keyframe+empty_moov", 0);
av_dict_set_int(&opts, "flush_packets", 1, 0);
avformat_write_header(output->format_ctx, &opts)
In the last line during execution, it always runs into segfault, here is the backtrace:
#0 0x00007ffff7a6ee30 in () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
#1 0x00007ffff7a98189 in avformat_init_output () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
#2 0x00007ffff7a98ca5 in avformat_write_header () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
...
The hard thing for me with the example is that it uses avformat_open_input.
However there is no such thing for the output (no avformat_open_ouput).
Update2:
I have found another example for reading: doc/examples/avio_reading.c.
There are mentions of a similar example for writing (avio_writing.c), but ffmpeg does not have this available (at least in my google search).
Is this task really this hard to solve? standard rtsp input to custom avio?
Fortunately ffmpeg.org is down. Great.
It was a silly mistake:
In the initialization part I called this:
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
However before this I already put the avio buffers into format_ctx:
format_ctx->pb = ...
Also, this line is unnecessary:
format_ctx = avformat_alloc_context();
Correct order:
AVOutputFormat * const output_format = av_guess_format("mp4", NULL, NULL);
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
format_ctx->pb = avio_alloc_context(
buffer, 4096, // internal buffer and its size
1, // write flag (1=true, 0=false)
opaque, // user data, will be passed to our callback functions
0, // no read
&IOWriteFunc,
&IOSeekFunc
);
format_ctx->flags |= AVFMT_FLAG_CUSTOM_IO;
format_ctx->oformat = output_format; //might be unncessary too
Segfault is gone now.
You need to write a AVIOContext implementation.

Convert array<Byte> to Struct in managed CPP

This has to be one of the most common things to do when working with IP Sockets. You get the data as a array and then you probably need to convert it to a struct.
The code looks something like this:
array<Byte> ^ buffer = gcnew array<Byte>(2048);
while( nDataLen < 16 )
{
nDataLen += m_Socket->Receive( buffer, 16-nDataLen, SocketFlags::None );
m_Socket->Poll( 100000, SelectMode::SelectRead );
}
Now I need to take the 16 Bytes I just received and convert them to the following struct;
[StructLayout(LayoutKind::Explicit, Size=16, Pack=1,CharSet=CharSet::Ansi)]
public ref struct ResponseHeader
{
public:
[FieldOffset(0)]
UInt32 m_StartMessage; // STRP
[FieldOffset(4)]
Int32 m_MessageLength; // Total message length
[FieldOffset(8),MarshalAs(UnmanagedType::ByValTStr,SizeConst=6)]
String ^ m_Status; // Message Status
[FieldOffset(14),MarshalAs(UnmanagedType::ByValTStr,SizeConst=2)]
String ^ m_Reason; // Message Reason
};
I would like to see a cast from the array directly to the struct but I am not sure that is doable in managed code, simple in native cpp.
If there isn't a way to cast this, then what is the best way to proceed?

ASN1_TIME_print functionality without BIO?

As described in this question: Openssl C++ get expiry date, there is the possibility to write an ASN1 time into a BIO buffer and then read it back into a custom buffer buf:
BIO *bio;
int write = 0;
bio = BIO_new(BIO_s_mem());
if (bio) {
if (ASN1_TIME_print(bio, tm))
write = BIO_read(bio, buf, len-1);
BIO_free(bio);
}
buf[write]='\0';
return write;
How could this be achieved without using BIO at all? The ASN1_TIME_print function is only present when OPENSSL_NO_BIO is not defined. Is there a way to write the time directly into a given buffer?
You can try the sample code below. It doesn't use BIO, but should give you the same output as the OP's example. If you don't trust the ASN1_TIME string, you'll want to add some error checking for:
notBefore->data is > 10 chars
each char value is between '0' and '9'
values for year, month, day, hour, minute, second
type
You should test for the type (i.e. UTC), if you expect multiple types.
You should also test whether or not the date/time is GMT and add that to the string if you want the output to match exactly as if using BIOs. see:
openssl/crypto/asn1/t_x509.c - ASN1_UTCTIME_print or ASN1_GENERALIZEDTIME_print
ASN1_TIME* notBefore = NULL;
int len = 32;
char buf[len];
struct tm tm_time;
notBefore = X509_get_notBefore(x509_cert);
// Format ASN1_TIME with type UTC into a tm struct
if(notBefore->type == V_ASN1_UTCTIME){
strptime((const char*)notBefore->data, "%y%m%d%H%M%SZ" , &tm_time);
strftime(buf, sizeof(char) * len, "%h %d %H:%M:%S %Y", &tm_time);
}
// Format ASN1_TIME with type "Generalized" into a tm struct
if(notBefore->type == V_ASN1_GENERALIZEDTIME){
// I didn't look this format up, but it shouldn't be too difficult
}
I think this should be possible, at least in terms of writing the time directly into a given buffer -- but you'll still need to use BIOs.
Ideally, BIO_new_mem_buf would suit, given that it creates an in-memory BIO using a given buffer as the source. Unfortunately, that function treats the given buffer as read-only, which is not what we want. However, we can create our own function (let's call it BIO_new_mem_buf2), based on the BIO_new_mem_buf source code:
BIO *BIO_new_mem_buf2(void *buf, int len)
{
BIO *ret;
BUF_MEM *b;
size_t sz;
if (!buf) {
BIOerr(BIO_F_BIO_NEW_MEM_BUF, BIO_R_NULL_PARAMETER);
return NULL;
}
sz = (size_t)len;
if (!(ret = BIO_new(BIO_s_mem())))
return NULL;
b = (BUF_MEM *)ret->ptr;
b->data = buf;
b->length = sz;
b->max = sz;
return ret;
}
This is just like BIO_new_mem_buf, except that a) the len argument must indicate the size of the given buffer, and b) the BIO is not marked "readonly".
With the above, you should now be able to call:
ASN1_TIME_print(bio, tm)
and have the time appear in your given buffer.
Note that I have not tested the above code, so YMMV. Hope this helps!

Splitting a comma-delimited string of integers

My background is not in C (it's in Real Studio - similar to VB) and I'm really struggling to split a comma-delimited string since I'm not used to low-level string handling.
I'm sending strings to an Arduino over serial. These strings are commands in a certain format. For instance:
#20,2000,5!
#10,423,0!
'#' is the header indicating a new command and '!' is the terminating footer marking the end of a command. The first integer after '#' is the command id and the remaining integers are data (the number of integers passed as data may be anywhere from 0 - 10 integers).
I've written a sketch that gets the command (stripped of the '#' and '!') and calls a function called handleCommand() when there is a command to handle. The problem is, I really don't know how to split this command up to handle it!
Here's the sketch code:
String command; // a string to hold the incoming command
boolean commandReceived = false; // whether the command has been received in full
void setup() {
// put your setup code here, to run once:
Serial.begin(9600);
}
void loop() {
// main loop
handleCommand();
}
void serialEvent(){
while (Serial.available()) {
// all we do is construct the incoming command to be handled in the main loop
// get the incoming byte from the serial stream
char incomingByte = (char)Serial.read();
if (incomingByte == '!')
{
// marks the end of a command
commandReceived = true;
return;
}
else if (incomingByte == '#')
{
// marks the start of a new command
command = "";
commandReceived = false;
return;
}
else
{
command += incomingByte;
return;
}
}
}
void handleCommand() {
if (!commandReceived) return; // no command to handle
// variables to hold the command id and the command data
int id;
int data[9];
// NOT SURE WHAT TO DO HERE!!
// flag that we've handled the command
commandReceived = false;
}
Say my PC sends the Arduino the string "#20,2000,5!". My sketch ends up with a String variable (called command) that contains "20,2000,5" and the commandRecieved boolean variable is set to True so the handleCommand() function is called.
What I would like to do in the (currently useless) handleCommand() function is assign 20 to a variable called id and 2000 and 5 to an array of integers called data, i.e: data[0] = 2000, data[1] = 5, etc.
I've read about strtok() and atoi() but frankly I just can't get my head around them and the concept of pointers. I'm sure my Arduino sketch could be optimised too.
Since you're using the Arduino core String type, strtok and other string.h functions aren't appropriate. Note that you can change your code to use standard C null-terminated strings instead, but using Arduino String will let you do this without using pointers.
The String type gives you indexOf and substring.
Assuming a String with the # and ! stripped off, finding your command and arguments would look something like this:
// given: String command
int data[MAX_ARGS];
int numArgs = 0;
int beginIdx = 0;
int idx = command.indexOf(",");
String arg;
char charBuffer[16];
while (idx != -1)
{
arg = command.substring(beginIdx, idx);
arg.toCharArray(charBuffer, 16);
// add error handling for atoi:
data[numArgs++] = atoi(charBuffer);
beginIdx = idx + 1;
idx = command.indexOf(",", beginIdx);
}
data[numArgs++] = command.substring(beginIdx);
This will give you your entire command in the data array, including the command number at data[0], while you've specified that only the args should be in data. But the necessary changes are minor.
seems to work, could be buggy:
#include<stdio.h>
#include <string.h>
int main(){
char string[]="20,2000,5";
int a,b,c;
sscanf(string,"%i,%i,%i",&a,&b,&c);
printf("%i %i %i\n",a,b,c);
a=b=c=0;
a=atoi(strtok(string,","));
b=atoi(strtok(0,","));
c=atoi(strtok(0,","));
printf("%i %i %i\n",a,b,c);
return 0;
}

fwrite() and file corruption

I'm trying to write a wchar array to a file in C, however there is some sort of corruption and unrelevant data like variables and paths like this
c.:.\.p.r.o.g.r.a.m. .f.i.l.e.s.\.m.i.c.r.o.s.o.f.t. .v.i.s.u.a.l. .s.t.u.d.i.o. 1.0...0.\.v.c.\.i.n.c.l.u.d.e.\.x.s.t.r.i.n.g..l.i.s.t...i.n.s.e.r.t
are written on to the file along with the correct data (example) I have confirmed that the buffer is null-terminated and contains proper data.
Heres my code:
myfile = fopen("logs.txt","ab+");
fseek(myfile,0,SEEK_END);
long int size = ftell(myfile);
fseek(myfile,0,SEEK_SET);
if (size == 0)
{
wchar_t bom_mark = 0xFFFE;
size_t written = fwrite(&bom_mark,sizeof(wchar_t),1,myfile);
}
// in another func
while (true)
{
[..]
unsigned char Temp[512];
iBytesRcvd = recv(sclient_socket,(char*)&Temp,iSize,NULL);
if(iBytesRcvd > 0 )
{
WCHAR* unicode_recv = (WCHAR*)&Temp;
fwrite(unicode_recv,sizeof(WCHAR),wcslen(unicode_recv),myfile);
fflush(myfile);
}
[..]
}
What could be causing this?
recv() will not null-terminate &Temp, so wcslen() runs over the bytes actually written by recv(). You will get correct results if you just use iBytesReceived as byte count for fwrite() instead of using wcslen() and hoping the data received is correctly null-terminated (wide-NULL-terminated, that is):
fwrite(unicode_recv, 1, iBytesReceived, myfile);

Resources