I have some pipe communication code - received bytes are no match for sent bytes.
There is a loop where 'CallNamedPipe' is called to send messages to server.
Now only 1st message is received intact, all the rest are received partially filled with 0xCD byte.
It seems, that when I free the memory after sending - it is still being read by server thread.
MSDN says that CallNamedPipe() is a complete message sequence: open pipe, sent bytes and close the pipe.
So, this seems strange to me. I must mention, that this code is built by VC++ 6.0 - a very old compiler. Code runs on Windows 7, maybe I need to use compatibility mode? Both client and server executables run on the same physical system, not remotely. Client uses CreateProcess() on startup to start the server. The messages are sent much later on, so racing conditions should not matter, I hope.
Thanks for any advice.
============ Client side (pseudocode): ============
for (iPiece=0; iPiece < nPieces; ++iPiece)
{
buffer = malloc (2048);
// copy some data bytes into buffer (1..2048 bytes)
// log 1st 32 DWORDS from message about to be sent
if (! CallNamedPipe (name, buffer, nBytes, ..., 1000))
{
// diagnostics: call to GetLastError(), etc.
}
free (buffer);
}
============ Server side (pseudocode): ============
DWORD __stdcall ServerThreadProc (PVOID p)
{
UINT cbMaxMsg = 0x10000; // 64K for a pipe message
PVOID buffer = malloc (cbMaxMsg);
HANDLE hPipe;
BOOL fAbort = 0;
hPipe = CreateNamedPipe (name, PIPE_ACCESS_DUPLEX,
PIPE_WAIT | PIPE_TYPE_MESSAGE | PIPE_READMODE_MESSAGE,
PIPE_UNLIMITED_INSTANCES, cbMaxMsg, cbMaxMsg, 1000, NULL);
while (fAbort == 0) // one of pipe messages sets fAbort=1, so thread can return.
{
if (ConnectNamedPipe (hPipe, NULL))
{
DWORD bytesLoaded = 0;
ReadFile (hPipe, buffer, cbMaxMsg, &bytesLoaded, NULL);
if (bytesLoaded)
{
// log 1st 32 DWORDS from received message
// process the pipe message (switch/case)
// data may be written back to client after processing
FlushFileBuffers (hPipe);
}
DisconnectNamedPipe (hPipe);
}
else
{
// diagnostics, GLE(), etc.
}
}
free (buffer);
CloseHandle (hPipe);
return 0;
}
Using PIPE_TYPE_MESSAGE makes the pipe a message pipe. It will give a message like behavior to the server. If you try and read more than the standard message size, it will give you complete messages.
0xCD is a standard fill, I think it is for heap, but not sure.
So far, we are reading 64k of data, and writing 2k of data. It looks like CallNamedPipe doesn't return until the data is accepted (as there is a timeout - set to 1000ms). The behavior of these systems, is that once kernel has the data buffer then the client code is not allowed to change the memory.
I would say the most likely case, is that the buffer is not being filled correctly, and that the amount of data in the server is consistent with messages in the pipe.
You have not provided enough data to verify this.
Related
I am creating a server/client TCP in C.
The idea is for the server to send a relatively large amount of information. However, the buffer in the client has a size of only 512 (I don't want to increase this size), and obviously, the information sent by the server is larger than this. Let's imagine 812 bytes.
What I want to do is, in the client, read 512 bytes, print them on the client's console, and then read the remaining bytes, and print them as well.
Here's what should happen:
1) Create server, and block in the read() system call (waiting for the client to write something);
2) Create the client, and write something in the socket, and then blocks on read(), waiting for the server to respond;
3) The server's read() call returns, and now server has to send that large amount of data, using the following code (after creating a new process):
dup2(new_socketfd, STDOUT_FILENO); // Redirect to socket
execlp("/application", "application", NULL); // Application that prints the information to send to the client
Let's imagine "application" printed 812 bytes of data to the socket.
4) Now the client has to read 812 bytes, with a buffer size of 512. That's my problem.
How can I approach this problem? I was wondering if I could make a loop, and read until there's nothing to read, 512 by 512 bytes. But as soon as there's nothing to read, client will block on read().
Any ideas?
recv will block when there is no data in the stream. Any data extracted from the stream, the length is returned from recv.
You can write a simple function to extract the full data just by using an offset variable and checking the return value.
A simple function like this will do.
ssize_t readfull(int descriptor,char* buffer, ssize_t sizetoread){
ssize_t offset = 0;
while (offset <sizetoread) {
ssize_t read = recv(descriptor,buffer+offset,sizetoread-offset,0);
if(read < 1){
return offset;
}
offset+=read;
}
return offset;
}
Also servers typically send some kind of EOF when the data is finished. Either the server might first send the length of the message to be read which is a constant size either four or eight bytes, then it sends the data so you know ahead of time how much to read. Or, in the case of HTTP for example, there is the content-length field as well as the '\r\n' delimeters.
Realistically there is no way to know how much data the server has available to send you, it's impractical. The server has to tell you how much data there is through some kind of indicator.
Since you're writing the server yourself, you can first send a four byte message which can be an int value of how much data the client should read.
So your server can look like this:
int sizetosend = arbitrarysize;
send(descriptor,(char*)&sizetosend,sizeof(int),0);
send(descriptor,buffer,sizetosend,0);
Then on your client side, read four bytes then the buffer.
int sizetoread = 0;
ssize_t read = recv(descriptor,(char*)&sizetoread,sizeof(int),0);
if(read < 4)
return;
//Now just follow the code I posted above
Today I have encountered some weird looking code that at first glance it's not apparent to me what it does.
send(file_desc,"Input \'y\' to continue.\t",0x18,0);
read(file_desc,buffer,100);
iVar1 = strcmp("y",(char *)buffer);
if (iVar1 == 0) {
// some more code
}
It seems that a text string is being written into the file descriptor. Immediately then after that it reads from that file descriptor into a buffer. And it compares if the text written into the buffer is a "y".
My understanding (please correct me if I am wrong), is that it writes some data which is a text string into the file descriptor, and then the file descriptor acts as a temporary storage location for anything you write to it. And after that it reads that data from the file descriptor into the buffer. It actually is the same file descriptor. It seems as a primitive way of using a file descriptor to copy data from the text string into the buffer. Why not just use a strcpy() instead?
What would be the use case of writing to a file descriptor and then immediately read from it? It seems like a convoluted way to copy data using file descriptors. Or maybe I don't understand this code well enough, what this sequence of a send() and a read() does?
And assuming that this code is instead using the file descriptor to copy the text string "Input \'y\' to continue.\t" into the buffer, why are they comparing it with the string "y"? It should probably be false every single time.
I am assuming that any data written into a file descriptor stays in that file descriptor until it is read from. So here it seems that send() is being used to write the string into, and read() is used to read it back out.
In man send it says:
The only difference between send() and write(2) is the presence of flags. With a zero
flags argument, send() is equivalent to write(2).
why would they use send() instead of write()? This code is just so mind boggling.
Edit: here's the full function where this code is originally from:
void send_read(int file_desc)
{
int are_equal;
undefined2 buffer [8];
char local_28 [32];
/* 0x6e == 110 == 'n' */
buffer[0] = 0x6e;
send(file_desc,"Input \'y\' to continue.\t",0x18,0);
read(file_desc,buffer,100);
are_equal = strcmp("y",(char *)buffer);
if (are_equal == 0) {
FUN_00400a86(file_desc,local_28);
}
else {
close(file_desc);
}
return;
}
The send() and recv() functions are for use with sockets (send: send a message on a socket — recv: receive a message from a connected socket). See also the POSIX description of Sockets in general.
Socket file descriptors are bi-directional — you can read and write on them. You can't read what you wrote, unlike with pipe file descriptors. With pipes, the process writing to the write end of a pipe can read what it wrote from the read end of the pipe — if another process didn't read it first. When a process writes on a socket, that information goes to the peer process and cannot be read by the writer.
send(2) is a system call that can only be used with sockets. A socket is a descriptor that allows you to use it to send data or receive from a remote point (a remote socket) that can be on a different computer or in the same as you are. But it works like a phone line, what you send is received by your parnter and what he/she sends is received by you. read(2) system call can be used by sockets, while send(2) cannot be used by files, so your sample code is mixing calls related to files with calls related to sockets (that's not uncommon, as read(2) and write(2) can both be used with sockets)
The code you post above is erroneous, as it blindly compares the received buffer with strcmp function, assuming that it received a null terminated string. This can be the case, but it also cannot.
Even if the sender (in the other side of the connection) agreed on sending a full message, nul terminated string. The receiver must first get the amount of data received (this is the return value of the read(2) call, which can be:
-! indicating some error on reception. The connection can be reset by the other side, or the other side can have rebooted while you send the data.
0 indicating no more data or end of data (the other side closed the connection) This can happen if the other side has a timeout and you take too much to respond. It closes the connection without sending anything. You just receive nothing.
n some data, less than the buffer size, but including the full packet sent by the peer (and the agreed nul byte it sent with it). This is the only case in which you can safely strcmp the data.
n some data, less than the buffer size, and less than the data transmitted. This can happen due to some data fragmentation of the data in several packets. Then you have to do another read until you have all the data send by your peer. Packet fragmentation is something natural in TCP, for example.
n some data, less than the buffer size, and more than the data transmitted. The sender did another transmit, after the one you receive, and both packets got into the kernel buffer. You have to investigate this case, as you have one full packet, and must save the rest of the received data in the buffer, for later processing, or you'll lose data you have received.
n some data, the full buffer filled, and no space to store the full transmitted data remained. You have filled the buffer and no \0 char came... the packet is larger than the buffer, you run out of buffer space and have to decide what to do (allocate other buffer to receive the rest, discard the data, or whatever you decide to do) This will not happen to you because you expect a packet of 1 or 2 characters, and you have a buffer of 100, but who knows...
At least, and as a minimum safe net, you can do this:
send(file_desc,"Input \'y\' to continue.\t",0x18,0);
int n = read(file_desc,buffer,sizeof buffer - 1); /* one cell reserved for '\0' */
switch (n) {
case -1: /* error */
do_error();
break;
case 0: /* disconnect */
do_disconnect();
break;
default: /* some data */
buffer[n] = '\0'; /* append the null */
break;
}
if (n > 0) {
iVar1 = strcmp("y",(char *)buffer);
if (iVar1 == 0) {
// some more code
}
}
Note:
As you didn't post a complete and verifiable example, I couldn't post a complete and verifiable response.
My apologies for that.
I want write a namedpipe client in windows OS, which can send data at any time even the client is receiving data.Example of MSDN only shows that sending data after receive something.And the serial operation is not what I want. Because the data I transfer between client and server is not so big,that is to say, IO operation should not be a time-consuming process, I didn't use OVERLAP in client.
The code that I modify in MSDN example of client is as below:
the main thread keep reading data, and the child-thread keep sending data to server. However, the server blocked when reading data when debugging.
std::thread t([&] {
cbToWrite = (lstrlen(lpvMessage) + 1) * sizeof(TCHAR);
_tprintf(TEXT("Sending %d byte message: \"%s\"\n"), cbToWrite, lpvMessage);
fSuccess = WriteFile(
hPipe, // pipe handle
lpvMessage, // message
cbToWrite, // message length
&cbWritten, // bytes written
NULL); // not overlapped
if (!fSuccess)
{
_tprintf(TEXT("WriteFile to pipe failed. GLE=%d\n"), GetLastError());
return -1;
}
printf("\nMessage sent to server, receiving reply as follows:\n");
});
while (1) // main thread always reading
{
do
{
// Read from the pipe.
fSuccess = ReadFile(
hPipe, // pipe handle
chBuf, // buffer to receive reply
BUFSIZE * sizeof(TCHAR), // size of buffer
&cbRead, // number of bytes read
NULL); // not overlapped
if (!fSuccess && GetLastError() != ERROR_MORE_DATA)
break;
_tprintf(TEXT("\"%s\"\n"), chBuf);
} while (!fSuccess); // repeat loop if ERROR_MORE_DATA
if (!fSuccess)
{
_tprintf(TEXT("ReadFile from pipe failed. GLE=%d\n"), GetLastError());
return -1;
}
}
t.join();
I expect someone can correct that code to get it work, or can you tell the standard practice or suggestions?
Thanks so much!
from CreateFile documentation about FILE_FLAG_OVERLAPPED
If this flag is specified, the file can be used for simultaneous
read and write operations.
If this flag is not specified, then I/O operations are serialized
I/O operations are serialized mean that the new I/O request will be wait until previous not complete. so even use multiple threads here not help, if you not use FILE_FLAG_OVERLAPPED. for example you can from one thread begin read operation and wait until data not exist. if you call write on this file from another thread - write will be wait in I/O subsystem code, until your read not complete. even if you say query file name (via GetFileInformationByHandleEx with FileNameInfo) this request will be serialized and wait until your read not completed.
so only option for simultaneous I/O operations (not only read write but all) use FILE_FLAG_OVERLAPPED when you create file.
I'm building a multi-client<->server messaging application over TCP.
I created a non blocking server using epoll to multiplex linux file descriptors.
When a fd receives data, I read() /or/ recv() into buf.
I know that I need to either specify a data length* at the start of the transmission, or use a delimiter** at the end of the transmission to segregate the messages.
*using a data length:
char *buffer_ptr = buffer;
do {
switch (recvd_bytes = recv(new_socket, buffer_ptr, rem_bytes, 0)) {
case -1: return SOCKET_ERR;
case 0: return CLOSE_SOCKET;
default: break;
}
buffer_ptr += recvd_bytes;
rem_bytes -= recvd_bytes;
} while (rem_bytes != 0);
**using a delimiter:
void get_all_buf(int sock, std::string & inStr)
{
int n = 1, total = 0, found = 0;
char c;
char temp[1024*1024];
// Keep reading up to a '\n'
while (!found) {
n = recv(sock, &temp[total], sizeof(temp) - total - 1, 0);
if (n == -1) {
/* Error, check 'errno' for more details */
break;
}
total += n;
temp[total] = '\0';
found = (strchr(temp, '\n') != 0);
}
inStr = temp;
}
My question: Is it OK to loop over recv() until one of those conditions is met? What if a client sends a bogus message length or no delimiter or there is packet loss? Wont I be stuck looping recv() in my program forever?
Is it OK to loop over recv() until one of those conditions is met?
Probably not, at least not for production-quality code. As you suggested, the problem with looping until you get the full message is that it leaves your thread at the mercy of the client -- if a client decides to only send part of the message and then wait for a long time (or even forever) without sending the last part, then your thread will be blocked (or looping) indefinitely and unable to serve any other purpose -- usually not what you want.
What if a client sends a bogus message length
Then you're in trouble (although if you've chosen a maximum-message-size you can detect obviously bogus message-lengths that are larger than that size, and defend yourself by e.g. forcibly closing the connection)
or there is packet loss?
If there is a reasonably small amount of packet loss, the TCP layer will automatically retransmit the data, so your program won't notice the difference (other than the message officially "arriving" a bit later than it otherwise would have). If there is really bad packet loss (e.g. someone pulled the Ethernet cable out of the wall for 5 minutes), then the rest of the message might be delayed for several minutes or more (until connectivity recovers, or the TCP layer gives up and closes the TCP connection), trapping your thread in the loop.
So what is the industrial-grade, evil-client-and-awful-network-proof solution to this dilemma, so that your server can remain responsive to other clients even when a particular client is not behaving itself?
The answer is this: don't depend on receiving the entire message all at once. Instead, you need to set up a simple state-machine for each client, such that you can recv() as many (or as few) bytes from that client's TCP socket as it cares to send to you at any particular time, and save those bytes to a local (per-client) buffer that is associated with that client, and then go back to your normal event loop even though you haven't received the entire message yet. Keep careful track of how many valid received-bytes-of-data you currently have on-hand from each client, and after each recv() call has returned, check to see if the associated per-client incoming-data-buffer contains an entire message yet, or not -- if it does, parse the message, act on it, then remove it from the buffer. Lather, rinse, and repeat.
I have a problem about a server (call servera) that receive a file from another server end send it to the client. The problem is that client receive 0 as file size and so zero bytes of the file:
/* receive file size from serverB */
recv(s,&bytes,sizeof(bytes),0);
/* send file size to client */
send(file_descriptor,&bytes,sizeof(bytes),0);
bytes = ntohs(bytes);
/* receive (from serverb) and send immediately (to client)*/
while (total != bytes) {
nread = read(s,&c,sizeof(char));
if(nread == 1){
send(file_descriptor,&c,sizeof(c),0);
total += nread;
}
}
What's wrong?
Everything could be wrong.
You must check I/O calls for errors before relying on side-effects, otherwise you will get unpredictable results.
In your case, perhaps the first recv() fails, leaving bytes uninitialized at 0.
Also, the loop reading a single byte at a time is very inefficient, and still fails to check that it manages to send that byte (send() can fail in which case you need to re-try).