POSIX AIO callback called with the same (wrong) sigval repeatedly - c

I send 100 requests using aio_write. I set the sigval (callback argument) to some address using request.aio_sigevent.sigev_value.sival_ptr = some_address;. Each request has a different address. The callback is supposed to write some data to the address.
I expect the callback to be called 100 times with the 100 different arguments I gave it. Instead, the callback is called 100 times with the same argument every time: the argument for the 100th request. I've tried with both SIGEV_THREAD and SIGEV_SIGNAL with the same result both times.
The code snippet where I send the requests:
for (int i = 0; i < num_requests; i++) {
struct aiocb request = build_request(/* snip */, &array[i]);
aio_write(&request);
}
(where build_request simply constructs a struct aiocb and writes to the aio fields, as well as the aio_sigevent fields: sigev_notify, sigev_signo or sigev_notify_function, and sigev_value.sival_ptr.)
Why is this happening?

From the aio_write manual:
The control block must not be changed while the write operation is in progress.
In your case the variable goes out of scope as soon as each loop iteration ends. So it's Undefined Behaviour both in terms of pure C and in terms of the API being called. One solution is to create a static or dynamic array of aiocb.

Related

Synchronize with sigev_notify_function()

I would like to read (asynchronously) BLOCK_SIZE bytes of one file, and the BLOCK_SIZE bytes of the second file, printing what has been read to the buffer as soon as the respective buffer has been filled. Let me illustrate what I mean:
// in main()
int infile_fd = open(infile_name, O_RDONLY); // add error checking
int maskfile_fd = open(maskfile_name, O_RDONLY); // add error checking
char* buffer_infile = malloc(BLOCK_SIZE); // add error checking
char* buffer_maskfile = malloc(BLOCK_SIZE); // add error checking
struct aiocb cb_infile;
struct aiocb cb_maskfile;
// set AIO control blocks
memset(&cb_infile, 0, sizeof(struct aiocb));
cb_infile.aio_fildes = infile_fd;
cb_infile.aio_buf = buffer_infile;
cb_infile.aio_nbytes = BLOCK_SIZE;
cb_infile.aio_sigevent.sigev_notify = SIGEV_THREAD;
cb_infile.aio_sigevent.sigev_notify_function = print_buffer;
cb_infile.aio_sigevent.sigev_value.sival_ptr = buffer_infile;
memset(&cb_maskfile, 0, sizeof(struct aiocb));
cb_maskfile.aio_fildes = maskfile_fd;
cb_maskfile.aio_buf = buffer_maskfile;
cb_maskfile.aio_nbytes = BLOCK_SIZE;
cb_maskfile.aio_sigevent.sigev_notify = SIGEV_THREAD;
cb_maskfile.aio_sigevent.sigev_notify_function = print_buffer;
cb_maskfile.aio_sigevent.sigev_value.sival_ptr = buffer_maskfile;
and the print_buffer() function is defined as follows:
void print_buffer(union sigval sv)
{
printf("%s\n", __func__);
printf("buffer address: %p\n", sv.sival_ptr);
printf("buffer: %.128s\n", (char*)sv.sival_ptr);
}
By the end of the program I do the usual clean up, i.e.
// clean up
close(infile_fd); // add error checking
close(maskfile_fd); // add error checking
free(buffer_infile);
printf("buffer_inline freed\n");
free(buffer_maskfile);
printf("buffer_maskfile freed\n");
The problem is, every once in a while buffer_inline gets freed before print_buffer manages to print its contents to the console. In a usual case I would employ some kind of pthread_join() but as far as I know this is impossible since POSIX does not specify that sigev_notify_function must be implemented using threads, and besides, how would I get the TID of such thread to call pthread_join() on?
Don't do it this way, if you can avoid it. If you can, just let process termination take care of it all.
Otherwise, the answer indicated in Andrew Henle's comment above is right on. You need to be sure that no more sigev_notify_functions will improperly reference the buffers.
The easiest way to do this is simply to countdown the number of expected notifications before freeing the buffers.
Note: your SIGEV_THREAD function is executed in a separate thread, though not necessarily a new thread each time. (POSIX.1-2017 System Interfaces ยง2.4.2) Importantly, you are not meant to manage this thread's lifecycle: it is detached by default, with PTHREAD_CREATE_JOINABLE explicitly noted as undefined behavior.
As an aside, I'd suggest never using SIGEV_THREAD in robust code. Per spec, the signal mask of the sigev_notify_function thread is implementation-defined. Yikes. For me, that makes it per se unreliable. In my view, SIGEV_SIGNAL and a dedicated signal-handling thread are much safer.

How to get buffer into CompletionROUTINE as apart of WSARecvFrom call?

I'm working on a UDP server, and am trying to use overlapped IO. I've been trying to use MSDN examples and documentation for research but haven't found use of the lpCompletionRoutine argument of the function.
I notice you pass a PWSAOVERLAPPED to WSARecvFrom, and it contains a LPVOID Pointer member. Would I create my own user-data structure that contains a reference to the buffer and pass it as a pointer inside this Pointer member of PWSAOVERLAPPED?
I thought it was a bit redundant though that the received bytes were available in two places:
WSARecvFrom's lpNumberOfBytesRecvd argument, and lpCompletionRoutine's cbTransferred parameter.
Example of my current completion routine:
void CALLBACK CompletionROUTINE(
DWORD dwError,
DWORD cbTransferred,
LPWSAOVERLAPPED lpOverlapped,
DWORD dwFlags
) {
UNREFERENCED_PARAMETER(dwError);
UNREFERENCED_PARAMETER(lpOverlapped);
UNREFERENCED_PARAMETER(dwFlags);
/* Best way to get the bytes read here? */
Printf(L"Recieved %d bytes\n", cbTransferred);
}
and my call to WSARecvFrom:
iResult = WSARecvFrom(
listenSocket,
&wsaBuffer,
1,
&dwBytesRecieved,
&dwFlags,
(PSOCKADDR)&sender,
&senderAddrSize,
&wsaOverlapped,
CompletionROUTINE
);
From the WSAOVERLAPPED structure documentation:
hEvent Type: HANDLE
If an overlapped I/O operation is issued without an I/O completion routine (the operation's lpCompletionRoutine parameter is
set to null), then this parameter should either contain a valid handle
to a WSAEVENT object or be null. If the lpCompletionRoutine parameter
of the call is non-null then applications are free to use this
parameter as necessary.
So since I am providing a lpCompletionRoutine parameter, I can use WSAEvent as a pointer to my user defined data.
Thank you to the commenter that lead me to this finding.

pthread POSIX C library detachstate

I was asked from where do we know that when passing NULL as a second argument in pthread_create() function the thread is made joinable.
I mean, I know that man pages state so, but a justification in code was demanded.
I know that when NULL is passed in, default attributes are used:
const struct pthread_attr *iattr = (struct pthread_attr *) attr;
if (iattr == NULL)
/* Is this the best idea? On NUMA machines this could mean accessing far-away memory. */
iattr = &default_attr;
I know that it should be somewhere in the code of pthread library, but I don't know where exactly.
I know that the definition of default_attr is in pthread_create.c:
static const struct pthread_attr default_attr = { /* Just some value > 0 which gets rounded to the nearest page size. */ .guardsize = 1, };
http://sourceware.org/git/?p=glibc.git;a=blob;f=nptl/pthread_create.c;h=4fe0755079e5491ad360c3b4f26c182543a0bd6e;hb=HEAD#l457
but I do not know where is exactly stated in the code that this result in a joinable thread.
Thanks in advance.
First off, from the code you pasted you can see that default_attr contains zeroes in almost all fields (there's no such thing as a half-initialized variable in C: if you only initialize some fields, the others are set to 0).
Second, pthread_create contains this code:
/* Initialize the field for the ID of the thread which is waiting
for us. This is a self-reference in case the thread is created
detached. */
pd->joinid = iattr->flags & ATTR_FLAG_DETACHSTATE ? pd : NULL;
This line checks whether iattr->flags has the ATTR_FLAG_DETACHSTATE bit set, which (for default_attr) it doesn't because default_attr.flags is 0. Thus it sets pd->joinid to NULL and not to pd as for detached threads.
(Note that this answer only applies to GNU glibc and not to POSIX pthreads in general.)

Caching packets captured from pcap

This is a follow-up question to this:
Rebuilding a packet to inject via pcap
What I want to accomplish:
functionA: Capture packets with pcap. Modify source/destination addresses. Recalculate checksums. Inject with pcap.
functionB: Create two threads. Thread 1 sends a magic packet to wake sleeping client. Thread 2 captures packets with pcap and caches the packets into an array of u_char *'s, since pcap puts the packet data serially into "u_char * packet". When both threads terminate, I then change the headers then inject each of the cached packets.
What I need help with:
functionA: I can do everything but calculate checksums. I tried to verify the original checksum by calculating it myself with a function but they never match. However, this issue is not as important because I don't need it to demo my final project. I understand that if IP checksums are incorrect, the receiving computer will discard the packet. But when I demo, so long as my client computer can be shown to have received this incorrect packet, I have proven my overall concept and will not fail. :)
functionB: I guess this is the more important problem. I don't know of an easy way to cache my captured packets. What I'm working on right now is as follows:
functionB creates a pointer to an array that stores u_char * called cachedPackets. So cachedPackets basically points to an array that stores "strings".
It'll be something like this? u_char ** cachedPackets[100], enough array elements for 100 packets.
After this, I start two threads. Thread1 to wake my sleeping client. Thread2 to open another pcap session so no data is lost while client is waking. Thread1 is easy, I've already tested my send magic packet function independently. Thread2 is where I'm screwing up.
Thread2 eventually calls int pcap_loop(pcap_t *p, int cut, pcap_handler callback, u_char *user).
callback is the function that will be run after each packet is captured. It is where I will be caching the packet into the array.
callback takes parameters ( u_char* user,
const struct pcap_pkthdr* packet_header,
const u_char* packet_data )
user is the same string in the 4th argument of pcap_loop.
So I was thinking, I could sneakily give my callback function the pointer to the array of string by type casting it.
pcap_loop(asdf, asdf, callback, (u_char *)cachedPackets);
Since I don't know how big the incoming packets will be, I'll dynamically allocate enough space in the callback function. I will also keep track of my position in the array with a static int.
this is what the callback looks like:
void cacheCall(u_char * user, const struct pcap_pkthdr * header, const u_char * packet)
static int cacheindex = 0;
u_char ** cachethis = (u_char **)user;
//u_char * cachething = *cachethis;
(*cachethis)[cacheindex] = (u_char *) malloc(header->len); <--- 497
int i = 0;
for(i = 0; i < header->len; i++)
{
(*cachethis)[cacheindex][i] = packet[i]; <-------------------503
}
//memcpy(cachething[cacheindex], packet, header->len);
cacheindex++;
but when I compile, i get
497: warning: assignment makes integer from pointer without a cast
503: error: subscripted value is neither array nor pointer
That was pretty longwinded, hopefully my knowledge of what I'm doing isn't completely misinformed. Any help would be awesome! :)
u_char ** cachethis;
cachethis is a pointer-to-pointer-to-u_char.
So:
*cachethis
is a pointer-to-u_char, and:
(*cachethis)[i]
is a plain u_char.
So line 497 tries to store a pointer into an u_char, and line 503 tries to subscript a u_char, both of which are invalid.
Looks like what you want is simply:
cachethis[i]
and
cachethis[i][j]

Linux - ioctl with FIONREAD always 0

I'm trying to get to know how many bytes there are readable at my TCP socket. I am calling ioctl with the Flag "FIONREAD" which should actually give me this value.
When I call the function I get as return val 0 ( so no Error ) but also my integer argument gets the value 0. That would be no problem but when I call the recv() method I actually read some Bytes out of the socket. What am I doing wrong?
// here some Code:
char recBuffer[BUFFERLENGTH] = {0};
int bytesAv = 0;
int bytesRead = 0;
int flags = 0;
if ( ioctl (m_Socket,FIONREAD,&bytesAv) < 0 )
{
// Error
}
if ( bytesAv < 1 )
{
// No Data Available
}
bytesRead = recv(m_Socket,recBuffer,BUFFERLENGTH,flags);
When I call the recv function i acutally read some valid Data ( which I expected )
It's happening very quickly, that's why you don't see anything. What you're doing:
ioctl: Is there data for me ? No, nothing yet
recv: Block until there is data for me. Some (short) time later: Here is your data
So if you really want to see FIONREAD, just wait for it.
/* Try FIONREAD until we get *something* or ioctl fails. */
while (!bytesAv && ioctl (m_Socket,FIONREAD,&bytesAv) >= 0)
sleep(1);
The real answer here is to use select(2) like cnicutar said. Toby, what you aren't understanding is that you have a race condition. First you look at the socket and ask how many bytes are there. Then, while your code is processing the "no data here" block, bytes are being received by the hardware & OS asynchronous to your application. So, by the time that the recv() function is called, the answer of "no bytes are available" is no longer true...
if ( ioctl (m_Socket,FIONREAD,&bytesAv) < 0 )
{ // Error
}
// BYTES MIGHT BE RECEIVED BY HARDWARE/OS HERE!
if ( bytesAv < 1 ) // AND HERE!
{
// No Data Available
// BUT BYTES MIGHT BE RECEIVED BY HARDWARE/OS HERE!
}
// AND MORE BYTES MIGHT BE RECEIVED BY HARDWARE/OS HERE!
bytesRead = recv(m_Socket,recBuffer,BUFFERLENGTH,flags);
// AND NOW bytesRead IS NOT EQUAL TO 0!
Sure, a small sleep probably fixed your program two years ago, but it also taught you terrible coding practice and you lost out on an opportunity to learn how to use sockets correctly by using select().
Further, as Karoly Horvath said, you can tell recv to not read more bytes than you can store in the buffer that the user passed in. Then your function interface becomes "This fn will return as many bytes as are available on the socket, but not more than [buffer size you passed in]".
This means that this function doesn't need to worry about clearing the buffer any more. The caller can call your function as many times as necessary to clear all of the bytes out of it (or you can provide a separate fn that discards the data wholesale and not tie up that functionality in any specific data gather function). Your function is more flexible by not doing too many things. You can then create a wrapper function that is smart to your data transfer needs of a particular application, and that fn calls the get_data fn and the clear_socket fn as needed for that specific app. Now you are building a library you can carry around from project to project, and maybe job to job if you're so lucky as to have an employer that lets you take code with you.
Use select() then ioctl(FIONREAD) then recv()
You're doing nothing wrong, if you are using blocking I/O recv() will block untill the data is available.

Resources