I would like to create a IOCP application to received TCP data, Once received, I will have to processing the data and write out the bytes partially. Understand that WSASend with IOCP can do the job. But I am worrying whether WSASend to the queue and does GetQueuedCompletionStatus synchronizely?. For Example:-
void processing_from_other_thread(...) {
...
DWORD state = still_doing1;
WSASend( ..,..,.., &state, .. );
DWORD state = still_doing2;
WSASend( ..,..,.., &state, .. );
DWORD state = still_doing3;
WSASend( ..,..,.., &state, .. );
DWORD state = Done;
PostCompletionQueue(....);
}
From the context above, will GetQueuedCompletionStatus getting them orderly?
GetQueuedCompletionStatus();
return still_doing1
GetQueuedCompletionStatus();
return still_doing2
GetQueuedCompletionStatus();
return still_doing3
GetQueuedCompletionStatus();
return Done
Continue
I just want to make sure the future design is correctly, I am afraid they are not orderly, For example, Return still_doing2 completed before still_doing1. The data sent might affected to client side.
Under IOCP I/O model, the application is responsible for organizing incoming and outgoing data. In other words, the process posts different I/O related messages to IOCP, but there is no guaranteed IOCP sends and/or receives data in that exact order.
This is a key part of asynchronous I/O.
Related
I wrote a short TCP client application to test a SW broker using blocking sockets. It works well and I am happy with it. Now I need to modify it and I have a problem reading not solecited messages (messages coming from the server not initiated by the client) because recv() blocks the program flow until there is no data to read.
Good solution for me may be to find a way to check if there are available data to read before calling recv() or that recv() may exit if there are no data to read within a certain period.
So I tried with this function:
static bool isDataAvailable(int socket)
{
fd_set sready;
struct timeval nowait;
FD_ZERO(&sready);
FD_SET((unsigned int) socket, &sready);
memset((char *) &nowait, 0, sizeof(nowait));
bool res = select(socket + 1, &sready, NULL, NULL, &nowait);
if(FD_ISSET(socket, &sready))
res = true;
else
res = false;
return res;
}
//------------------------------------------------------------------------
but the result is that after the call all my application works as I selected to work using non blocking sockets. I normally use it on Linux and it works. I found some info around the web about select() and seems to me it would work also under Windows. But... something is going wrong.
Any help and suggestion will be appreciated.
Thanks,
Massimo
When I open and read file in OVERLAPPED manner on Win32 api, I then have several ways to complete IO request including waiting for file handle (or event in overlapped structure) using
WaitForSingleObject
GetOverlappedResult with bWait=TRUE
Both functions seems to have same effect: thread stopped until handle or event is signaled, and that means data is placed in buffer provided to ReadFile.
So, what is the difference? Why do I need GetOverlappedResult?
i full agree with Remus Rusanu answer . also instead create own IOCP and thread pool, which will be listen on this IOCP, you can use or BindIoCompletionCallback or CreateThreadpoolIo (begin from vista) - in this case system yourself create IOCP and thread pool wich will be listen on this IOCP and when some operation completed - call your callback. this is very simplify code vs own iocp/thread pool (own iocp/thread pool really i think have sense implement only when you have very big count I/O (say socket io on server side) and need special optimization for perfomance)
however
So, what is the difference? Why do I need GetOverlappedResult
how you can see GetOverlappedResult[Ex] not only wait for result, but
return to you NumberOfBytesTransferred if operation is completed.
if operation is completed with error NTSTATUS - convert it to win32
error and set last error
if operation still pending and you want wait - it select wait on
hFile or hEvent
so GetOverlappedResult[Ex] do much more than simply call WaitForSingleObject
however not very hard implement this API yourself. for example
BOOL
WINAPI
MyGetOverlappedResult(
_In_ HANDLE hFile,
_In_ LPOVERLAPPED lpOverlapped,
_Out_ LPDWORD lpNumberOfBytesTransferred,
_In_ BOOL bWait
)
{
if ((NTSTATUS)lpOverlapped->Internal == STATUS_PENDING)
{
if (!bWait)
{
SetLastError(ERROR_IO_INCOMPLETE);
return FALSE;
}
if (lpOverlapped->hEvent)
{
hFile = lpOverlapped->hEvent;
}
if (WaitForSingleObject(hFile, INFINITE) != WAIT_OBJECT_0)
{
return FALSE;
}
}
else
{
MemoryBarrier();
}
*lpNumberOfBytesTransferred = (ULONG)lpOverlapped->InternalHigh;
NTSTATUS status = (NTSTATUS)lpOverlapped->Internal;
if (status)
{
RtlNtStatusToDosError(status);
}
return NT_SUCCESS(status);
}
so what better : use GetOverlappedResult[Ex] or implement it functional yourself ?
You could use either, but truly that's not the 'right' way of doing it. you should attach the handle to an IO completion port and then wait on the completion port. This way you have one pool of threads servicing many IO events, as you can attach multiple handles to a completion port. I recommend reading Designing Applications for High Performance.
Say that I have two programs, a client and a server, and that I am running the client and the server on the same computer (so the speed is extremely fast), and say that the client socket's receive buffer is empty, and that the server will not send any data to the client except if the client told the server to do so.
Now in the client, I call WSASend() and then after it I call WSARecv():
WSASend(...); // tell the server to send me some data
WSARecv(...);
So in the code above, WSASend() is telling the server to send some data to the client (for example: the string "hello").
Now after some time, two completion packets will be placed in the completion port:
The first completion packet is for WSASend() (telling me that the data
has been placed in the client socket's send buffer).
The second completion packet is for WSARecv() (telling me that the data
has been placed in the buffer that I passed to WSARecv() when I
called it).
Now my question is: is it possible that the completion packet for WSARecv() be placed in the completion port before the completion packet for WSASend() (so when I call GetQueuedCompletionStatus() I will get the completion packet for WSARecv() first)?
you must never assume any order of completion packets you got. you must have independent from this knowledge - which is operation complete.
you must define some structure inherited from OVERLAPPED and it this structure place all data related to operation. including tag which describe type of operation. so when you extract pointer to OVERLAPPED from IOCP you cast it to this structure and will be know - are this for recv or send, connect or disconnect.. for example
class IO_IRP : public OVERLAPPED
{
//...
DWORD m_opCode;// `recv`, `send`, `dsct`, `cnct`
IO_IRP(DWORD opCode,...) : m_opCode(opCode) {}
VOID IOCompletionRoutine(DWORD dwErrorCode, DWORD dwNumberOfBytesTransfered)
{
// switch (m_opCode)
m_pObj->IOCompletionRoutine(m_packet, m_opCode, dwErrorCode, dwNumberOfBytesTransfered, Pointer);
delete this;
}
static VOID CALLBACK _IOCompletionRoutine(DWORD dwErrorCode, DWORD dwNumberOfBytesTransfered, LPOVERLAPPED lpOverlapped)
{
static_cast<IO_IRP*>(lpOverlapped)->IOCompletionRoutine(dwErrorCode, dwNumberOfBytesTransfered);
}
};
// recv
if (IO_IRP* Irp = new IO_IRP('recv', ..))
{
WSARecv(..., Irp);
...
}
I am working on newtwork event based socket application.
When client has sent some data and there is something to be read on the socket, FD_READ network event is generated.
Now according to my understanding, when server wants to write over the socket, there must be an event generated i.e. FD_WRITE. But how this message will be generated?
When there is something available to be read, FD_READ is automatically generated but what about FD_WRITE when server wants to write something?
Anyone who can help me with this confusion please?
Following is the code snippet:
WSAEVENT hEvent = WSACreateEvent();
WSANETWORKEVENTS events;
WSAEventSelect(newSocketIdentifier, hEvent, FD_READ | FD_WRITE);
while(1)
{ //while(1) starts
waitRet = WSAWaitForMultipleEvents(1, &hEvent, FALSE, WSA_INFINITE, FALSE);
//WSAResetEvent(hEvent);
if(WSAEnumNetworkEvents(newSocketIdentifier,hEvent,&events) == SOCKET_ERROR)
{
//Failure
}
else
{ //else event occurred starts
if(events.lNetworkEvents & FD_READ)
{
//recvfrom()
}
if(events.lNetworkEvents & FD_WRITE)
{
//sendto()
}
}
}
FD_WRITE means you can write to the socket right now. If the send buffers fill up (you're sending data faster than it can be sent on the network), eventually you won't be able to write anymore until you wait a bit.
Once you make a write that fails due to the buffers being full, this message will be sent to you to let you know you can retry that send.
It's also sent when you first open up the socket to let you know it's there and you can start writing.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms741576(v=vs.85).aspx
The FD_WRITE network event is handled slightly differently. An
FD_WRITE network event is recorded when a socket is first connected
with a call to the connect, ConnectEx, WSAConnect, WSAConnectByList,
or WSAConnectByName function or when a socket is accepted with accept,
AcceptEx, or WSAAccept function and then after a send fails with
WSAEWOULDBLOCK and buffer space becomes available. Therefore, an
application can assume that sends are possible starting from the first
FD_WRITE network event setting and lasting until a send returns
WSAEWOULDBLOCK. After such a failure the application will find out
that sends are again possible when an FD_WRITE network event is
recorded and the associated event object is set.
So, ideally you're probably keeping a flag as to whether it's OK to write, right now. It starts off as true, but eventually, you get a WSAEWOULDBLOCK when calling sendto, and you set it to false. Once you receive FD_WRITE, you set the flag back to true and resume sending packets.
For an application in C, i need to response more than one clients.
I setup the connection with a code like,
bind(...);
listen(...);
while(1){
accept(...);//accept a client
recv(...);//receive something
send(...);//send something to client
bzero(buf);//clear buffer
}
This works great when i have only one client. Other clients also can connect to the server but although they command something, server does not response back to clients who connected after the first client. How can i solve this problem?
Write a server using asynchronous, nonblocking connections.
Instead of a single set of data about a client, you need to create a struct. Each instance of the struct holds the data for each client.
The code looks vaguely like:
socket(...)
fcntl(...) to mark O_NONBLOCK
bind(...)
listen(...)
create poll entry for server socket.
while(1) {
poll(...)
if( fds[server_slot].revents & POLLIN ) {
accept(...)
fcntl(...) mark O_NONBLOCK
create poll and data array entries.
}
if( fds[i].revents & POLLIN ) {
recv(...) into data[i]
if connection i closed then clean up.
}
if( fds[i].revents & POLLOUT ) {
send(...) pending info for data[i]
}
}
If any of your calls return the error EAGAIN instead of success then don't panic. You just try again later. Be prepared for EAGAIN even if poll claims the socket is ready: it's good practice and more robust.
i need to response more than one clients.
Use Threading.
Basically you want your main thread to only do the accept part, and then handle the rest to another thread of execution (which can be either a thread or a process).
Whenever your main thread returns from "accept", give the socket descriptor to another thread, and call accept again (this can be done with fork, with pthread_create, or by maintaining a thread pool and using synchronization, for instance condition variables, to indicate that a new client has been accepted).
While the main thread will handle possible new clients incoming, the other threads will deal with the recv/send.