Unable to receive large array using sockets in c - c

Question

I am sending integer array of size from one raspberry pi to another using POSIX sockets in C. I am writing an integer array of size 131072 from one pi, the return value from the write command shows that all 131072 values have been written. ret = write(socket, &array, sizeof(array)) Using the same method for receive ret = read(socket, &array, sizeof(array)) shows that all the sent values are not been read rather the number of values read correctly is also not constant but varies between 10000 to 20000.
I tried to use the read function inside a loop where I read integer in each loop iteration
for(int i =0; i<131072; i++){
ret = read(socket, &value, sizeof(value));
data[i] = value;}
I was able to receive all the values with no error or losses.

Answer 1

The underlying protocol (probably TCP and IP) splits the data into packets, which might arrive as individual data blocks on the receiving application. Theoretically, read() might receive every single byte individually (i.e. return 1 each time when called 2000 times). Your application needs to be able to work with that.

Answer 2

You might need to use code similar to this:
multi-rw.h
#ifndef JLSS_ID_MULTI_RW_H
#define JLSS_ID_MULTI_RW_H
#include <sys/types.h>
extern ssize_t multi_read(int fd, char *buffer, size_t nbytes);
extern ssize_t multi_write(int fd, const char *buffer, size_t nbytes);
#endif
multi-rw.c
It's a little unfortunate that the block of test code has to precede the active functions, but that's necessary (and test code is necessary — at least, I find it extremely helpful and reassuring). I suppose it could go into a separate header file (multi-rw-test.h or thereabouts) which would be conditionally included; it would be better for presentation on SO, but otherwise is just another file to worry about.
#include "multi-rw.h"
#include <unistd.h>
#ifdef TEST
#ifndef MAX_WRITE_SIZE
#define MAX_WRITE_SIZE 64
#endif
#ifndef MAX_READ_SIZE
#define MAX_READ_SIZE 64
#endif
static inline size_t min_size(size_t x, size_t y) { return (x < y) ? x : y; }
static inline ssize_t pseudo_read(int fd, char *buffer, size_t nbytes)
{
return read(fd, buffer, min_size(MAX_READ_SIZE, nbytes));
}
static inline ssize_t pseudo_write(int fd, const char *buffer, size_t nbytes)
{
return write(fd, buffer, min_size(MAX_READ_SIZE, nbytes));
}
#undef read
#undef write
#define read(fd, buffer, nbytes) pseudo_read(fd, buffer, nbytes)
#define write(fd, buffer, nbytes) pseudo_write(fd, buffer, nbytes)
#endif
ssize_t multi_read(int fd, char *buffer, size_t nbytes)
{
ssize_t nb = 0;
size_t nleft = nbytes;
ssize_t tbytes = 0;
while (nleft > 0 && (nb = read(fd, buffer, nleft)) > 0)
{
tbytes += nb;
buffer += nb;
nleft -= nb;
}
if (tbytes == 0)
tbytes = nb;
return tbytes;
}
ssize_t multi_write(int fd, const char *buffer, size_t nbytes)
{
ssize_t nb = 0;
size_t nleft = nbytes;
ssize_t tbytes = 0;
while (nleft > 0 && (nb = write(fd, buffer, nleft)) > 0)
{
tbytes += nb;
buffer += nb;
nleft -= nb;
}
if (tbytes == 0)
tbytes = nb;
return tbytes;
}
#ifdef TEST
#include "stderr.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv)
{
if (argc != 0)
err_setarg0(argv[0]);
char buffer[4096];
ssize_t ibytes;
while ((ibytes = multi_read(0, buffer, sizeof(buffer))) > 0)
{
ssize_t obytes;
if ((obytes = multi_write(1, buffer, ibytes)) != ibytes)
err_syserr("failed to write %lld bytes - only wrote %lld bytes\n",
(long long)ibytes, (long long)obytes);
}
if (ibytes < 0)
err_syserr("I/O error reading standard input: ");
return 0;
}
#endif
The test harness allows you to test the code reading from standard input and writing to standard output. You can configure the amount of data read via (for example) compilation command line options -DMAX_WRITE_SIZE=132 and -DMAX_READ_SIZE=103. You need to test it on files smaller than (a) 4096 bytes, and (b) smaller than the maximum read and write sizes, and also on files bigger than 4096 bytes. If you are motivated enough, you can upgrade the pseudo_read() and pseudo_write() functions to generate errors quasi-randomly, to see how the code handles such errors.