Receiving data function stalls when requesting large chunk of data - c

I'm creating a mini web server in C.
The following function is supposed to read in data from the client computer.
The objective is to read the second piece of data after the first space. Each piece of incoming data is space separated.
For example, if the incoming data is:
GET /123/456
then I'd expect /123/456.
If the incoming data is:
GET /123/456 789
then I'd still expect /123/456.
This relevant fragment is from an external function that sets up a 10 KB buffer and calls the problematic function:
//nsock is a valid socket handle from an accept() call.
printf("CLIENT CONNECTION!\n");
char req[10000];long reqsz=10000;
getreq(req,&reqsz,nsock);
printf("Received %d bytes\n",reqsz);
printf("Data: %s\n",req);
"CLIENT CONNECTION!" appears on the screen, but "Received" does not appear if bufsize inside the function is a high value. If I set it to a low value like 16, or 100, then everything is displayed correctly. Why do large numbers like 5000 not work?
This is the problematic function:
//getreq params in: req=external buffer for data
// reqsz=size of external buffer. I set 10000
// nsock=valid socket pointer from accept()
//
//getreq params out: reqsz=actual size of data returned
// req=actual data
//
void getreq(char* req,unsigned long *reqsz,long nsock){
//bufsize=how many bytes to read at once. High values like 5000 cause a stall.
//buffer=buffer of data from recv call
const unsigned long ibs=*reqsz,bufsize=5000;
char buffer[ibs],*rp=req;
//spacect=# of spaces in data read
//szct=iterator variable
//mysz=total length of returned data
//bufct=buffer counter to prevent segfault
//recvsz=data size returned from recv or
// forced -2 if buffer hits capacity
// or 2nd space in returned data is found
unsigned long spacect=0,szct=0,mysz=0,bufct=0;
long recvsz=1;char *p=buffer;
//
//Expected data: GET /whatever HTTP/x.x but we
// want /whatever
//
//loop until 2nd space is found or
//ibs bytes of data have been processed
while (recvsz > 0 && bufct < ibs){
recvsz=recv(nsock, p, bufsize, 0);
if (recvsz < 1){break;}
for (szct=1;szct<=recvsz;szct++){
if (*p==' '){spacect++;if (spacect > 2){spacect=2;recvsz=-2;break;}}
if (spacect==1 && *p != ' '){mysz++;if (mysz <= *reqsz){*rp++=*p;}}
p++;bufct++;if (bufct > ibs){recvsz=-2;break;}
}
}
// Process rest of data to try to avoid client errors
while (recvsz == -2){
recvsz=recv(nsock, buffer, bufsize, 0);
}
*reqsz=mysz;
}

Related

Managing UART buffer Thingy91

I have a Thingy91 Nordic device, and I am integrating it with a sensor that works at 1200 baud rate. It basically sends strings that i need to store in a buffer and parse for usage. However I am facing a peculiar issue wherein I am getting the correct string the 1st time I recieve data, but after that I recieve garbage values.
Below is the part of the code:
uint8_t message_buf[100];
void uart_cb(struct device *x) {
uart_irq_update(x);
if (uart_irq_rx_ready(x)) {
uart_fifo_read(x, message_buf, sizeof(message_buf));
printk("%s", message_buf);
}
}
void main(){
struct device *uart = device_get_binding(UART_PORT);
......
uart_irq_callback_set(uart, uart_cb);
......
}
We think that it might be a problem in managing the message_buf while getting the new string, and wanted to know the correct procedure of managing the buffer. Also,what could be the root cause that I get correct data on the first call and get garbage values later on.
Regards,
Adeel.
Looking at the API document:
static int uart_fifo_read(structdevice *dev, u8_t *rx_data, const int size)
Read data from FIFO.
This function is expected to be called from UART interrupt handler (ISR), if uart_irq_rx_ready() returns true. Result of calling this function not from an ISR is undefined (hardware-dependent). It’s unspecified whether “RX ready” condition as returned by uart_irq_rx_ready() is level- or edge- triggered. That means that once uart_irq_rx_ready() is detected, uart_fifo_read() must be called until it reads all available data in the FIFO (i.e. until it returns less data than was requested).
Returns: Number of bytes read.
Parameters:
dev: UART device structure.
rx_data: Data container.
size: Container size.
It clearly specifies that the API must be called until the return value is lesser than requested number of bytes (size).
Your code:
if (uart_irq_rx_ready(x)) {
uart_fifo_read(x, message_buf, sizeof(message_buf));
printk("%s", message_buf);
}
It calls uart_fifo_read() once and then exits the callback uart_cb, which deviates from what is recommended.
You should read from FIFO until it is empty:
int readBytes = 0;
if (uart_irq_rx_ready(x))
{
do{
readBytes += uart_fifo_read(x, &message_buf[readBytes], (sizeof(message_buf)-readBytes));
while(readBytes < sizeof(message_buf));
printk("Received %d bytes: ", readBytes);
// printk("%s", message_buf); // Certainly the code will crash if it isn't a string that fits in message_buf!
}

How can I use a dynamic array for UDP receipt in D?

I would like to create a simple UDP server that can receive messages of varying length. However, it seems as if D's Socket.receiveFrom() expects a static length buffer array. When the following code runs:
void main() {
UdpSocket server_s;
Address client_addr;
ubyte[] in_buf;
ptrdiff_t bytesin;
server_s = new UdpSocket();
server_s.bind(new InternetAddress(InternetAddress.ADDR_ANY, PORT_NUM));
bytesin = server_s.receiveFrom(in_buf, client_addr);
if (bytesin == 0 || bytesin == Socket.ERROR) {
writeln("Error receiving, bytesin: ", bytesin);
return;
}
// Do stuff
}
receiveFrom() immediately falls through with bytesin == 0. Why is this? Can I even use dynamic arrays for receiving over UDP?
receive and receiveFrom don't do the allocation themselves. You can pass an array of some fixed size big enough to hold any packet you expect, then slice it based on how many bytes you received.
If you preallocated 64 KB, that ought to fit everything you could conceivably get. I tend to just use 4 KB buffers though.
ubyte[4096] in_buf;
bytesin = server_s.receiveFrom(in_buf, client_addr);
// check for error first then
auto message_received = in_buf[0 .. bytesin];
// process it
// keep looping, reusing the buffer, to get more stuff

Getting raw data using libusb

I'm doing reverse engineering about a ultrasound probe on the Linux side. I want to capture raw data from an ultrasound probe. I'm programming with C and using the libusb API.
There are two BULK IN endpoints in the device (2 and 6). The device is sending 2048 bytes data, but it is sending data as 512 bytes with four block.
This picture is data flow on the Windows side, and I want to copy that to the Linux side. You see four data blocks with endpoint 02 and after that four data blocks with endpoint 06.
But there is a problem about timing. The first data block of endpoint 02's and first data block of endpoint 06's are close to each other acoording to time. But in data flow they are not in sequence.
I see that the computer is reading the first data blocks of endpoint 02 and 06. After that, the computer is reading the other three data blocks of endpoint 02 and endpoint 06. But in USB Analyzer, the data flow is being viewed according to the endpoint number. The sequence is different according to time.
On the Linux side, I write code like this:
int index = 0;
imageBuffer2 = (unsigned char *) malloc(2048);
imageBuffer6 = (unsigned char *) malloc(2048);
while (1) {
libusb_bulk_transfer(devh, BULK_EP_2, imageBuffer2, 2048, &actual2, 0);
libusb_bulk_transfer(devh, BULK_EP_6, imageBuffer6, 2048, &actual6, 0);
//Delay
for(index = 0; index <= 10000000; index ++)
{
}
}
So that result is in picture as below
In other words, in my code all reading data is being read in sequence according to time and endpoint number. My result is different from the data flow on the Windows side.
In brief, I have two BULK IN endpoints, and they are starting read data close according to time. How is it possible?
It's not clear to me whether you're using a different method for getting the data on Windows or not, I'm going to assume that you are.
I'm not an expert on libusb by any means, but my guess would be that you are overwriting you data with each call, since you're using the same buffer each time. Try giving your buffer a fixed value before using the transfer method, and then evaluate the result.
If it is the case, I believe something along the lines of the following would also work in C:
imageBuffer2 = (unsigned char *) malloc(2048);
char *imageBuffer2P = imageBuffer2;
imageBuffer6 = (unsigned char *) malloc(2048);
char *imageBuffer6P = imageBuffer6;
int dataRead2 = 0;
int dataRead6 = 0;
while(dataRead2 < 2048 || dataRead6 < 2048)
{
int actual2 = 0;
int actual6 = 0;
libusb_bulk_transfer(devh, BULK_EP_2, imageBuffer2P, 2048-dataRead2, &actual2, 200);
libusb_bulk_transfer(devh, BULK_EP_6, imageBuffer6P, 2048-dataRead6, &actual6, 200);
dataRead2 += actual2;
dataRead6 += actual6;
imageBuffer2P += actual2;
imageBuffer6P += actual6;
usleep(1);
}

Displaying 2D arrays of pixel values in a console application

I'm continuously sending 2D arrays of pixel values (uint32) from LabVIEW to a C-program through TCP/IP with the resolution 160x120. The purpose of the C-program is to display the received pixel values as 2D arrays in the console application. I'm sending the pixels as stream of bytes, and using the recv function in Ws2_32.lib to receive the bytes in the C-program. Then I'm converting the bytes to uint32 values and displaying them in the console application using a 2D arrays, so every 2D array will represent an image.
I have en issue with the frame rate though. I'm able to send 30 frames per second in LabVIEW, but when I open the TCP/IP connection with the C-program, the frame rate goes down to 1 frame per second. It must be an issue with the C-program, since I managed to send the desired frames per second with the same LabVIEW program to a corresponding C# program.
The C-code:
#define DEFAULT_BUFLEN 256
#define IMAGEX 120
#define IMAGEY 160
WSADATA wsa;
SOCKET s , new_socket;
struct sockaddr_in server , client;
int c;
int iResult;
char recvbuf[DEFAULT_BUFLEN];
int recvbuflen = DEFAULT_BUFLEN;
typedef unsigned int uint32_t;
unsigned int x=0,y=0,i,n;
uint32_t image[IMAGEX][IMAGEY];
size_t len;
uint32_t* p;
p = (uint32_t*)recvbuf;
do
{
iResult = recv(new_socket, recvbuf, recvbuflen, 0);
len = iResult/sizeof(uint32_t);
for(i=0; i < len; i++)
{
image[x][y] = p[i];
x++;
if (x >= IMAGEX)
{
x=0;
y++;
}
if (y >= IMAGEY)
{
y = 0;
x = 0;
//print image
for (n=0; n< IMAGEX*IMAGEY; n++)
{
printf("%d",image[n%IMAGEX][n/IMAGEY]);
if (n % IMAGEX)
{
printf(" ");
}
else
{
printf("\n");
}
}
}
}
} while ( iResult > 0 );
try reducing the prints .. Since you are reading and printing in the same thread, the data in the TCP connection will fill up and it will then back pressure the other end (LABView) and the LABView will stop sending data until it gets the green signal from the other end (you C program)
To start with you can debug by replacing this
for (n=0; n< IMAGEX*IMAGEY; n++)
{
printf("%d",image[n%IMAGEX][n/IMAGEY]);
if (n % IMAGEX)
{
printf(" ");
}
else
{
printf("\n");
}
}
with
printf("One frame recv\n");
and see if it makes any difference. I am assuming your tcp connection has ample bandwidth
Very hard to diagnose without further information. I can give a few suggestions, however.
First of all, your recv call is using a small buffer, so you are spending a lot of time calling it. Why not read a whole frame at a time? Also, you read in the data and then copy it to the image array. Wouldn't it be simpler to just use the image array itself? Combining those two suggestions would have recv reading a full frame directly into the image array, saving a lot of time.
Another source of the problem could be the console. With the sample code you provided, you are attempting to write 30*120*160=57,600 integer values per second to the terminal. If the average value, with delimiter, takes up 8 characters, that's 4 million characters per second. It's entirely possible that the display just can't go that fast, in which case things would back up and slow down all the way to the server writing to the socket.
There are several ways to handle this, but it's too much to go into here.

How to receive int array from client side (written in C) to server (written in python)

I just want to send an array adc_array=[w, x, y, z] from client to server. Below is the client side code whereas my server is in python which accepts json only. I get no error when i compile the code however get 2 warnings :
1- warning: pointer targets in passing argument 2 of 'UDPWrite' differ in signedness.
2- warning: no newline at end of file.
But at the server side, i am not able to receive the whole array, instead i just get the first character of the array i.e. [ .
I am new to C programming. I would really appreciate any help.
// Main function
void FlyportTask()
{
// Flyport connects to default network
WFConnect(WF_DEFAULT);
while(WFGetStat() != CONNECTED);
vTaskDelay(25);
UARTWrite(1,"Flyport Wi-fi connected...hello world!\r\n");
BOOL UdpSocketOpenRequest=TRUE;
BYTE UdpSocket=0;
// openinging UDP socket
if (UdpSocketOpenRequest) //open socket
{
UdpSocketOpenRequest=FALSE;
if (UdpSocket!=0) //if this is not equals to zero
{
UDPClientClose(UdpSocket);
}
UARTWrite(1,"OpenSocket\r\n");
UdpSocket= UDPClientOpen("10.0.0.106", "8000"); //Client socket opening
}
while(1)
{
//defining pointer
int *array_pointer;
int adc_array[4];
int j;
char buf[10]; //buffer to print
// I have made a separate function to get adc values which returns the pointer to the array.
array_pointer = get_adcval();
UARTWrite (1, "ADC Array\r\n");
for (j = 0; j < 4; j++)
{
adc_array[j] = *(array_pointer + j);
sprintf (buf, "%d", adc_array[j]);
UARTWrite (1, buf);
UARTWrite (1, "\n");
}
//if UDP socket is open, send the data
if ((UdpSocket!=0))
{
// defining pointer of serial_out
char *s_out;
int size;
// creating a JSON array from adc_array with 4 elements
cJSON * int_array = cJSON_CreateIntArray(adc_array,4);
// Serializing the array
s_out = cJSON_Print(int_array);
//Writing to the serial output/monitor
UARTWrite(1, "\r\narray to be sent\r\n");
UARTWrite(1, s_out);
UARTWrite(1,"\r\n");
// Assume adc_array=[1021, 1022, 1023, 1024]
// I get output [1021, 1022, 1023, 1024]
//compose message
size = strlen(s_out);
UDPWrite (UdpSocket, s_out, size);
// at the server side, i just receive only first character i.e. [
/*to free the memory */
free(s_out);
}
//
// remember to add delay vTaskDelay(50) 50ms
//remember to close the socket
}
}
You didn't allocated memory for s_out. even if it is printing correct result on UART but still it can be overwritten by any of the UARTWrite functions or strlen() function in the next lines. If it is overwritten then the "size" variable will get the number of bytes starting from the first byte to first null character in the memory (this is how strlen() functions). hence the "size" value can be totally random. it can be 0 or 1 or 1000. if the size is not correct then you will receive only "size" number of bytes. In your case it is possible that size is one. try printing size before UDPWrite. fix this problem by adding a malloc call before serializing the array.
If it doesn't work either then check your receiver side. is your receiver working fine if you send some dummy data from a tested python client (or any other tested or reliable client)? if no then there is some problem with your receiver.
Print out what strlen(s_out) returns, also print out the return value of UDPWrite ( I assume that like any write function this will be returning the size of the data which is written to the socket).
By reading the function names I presume you are using UDP transmission which is unreliable.

Resources