I have 2 programs written in C, one program writes to the pcap file and the second program reads from it at the same time.For writing ,I am using the following code
while(j < 100000)
{
pcount = pcap_dispatch(p,2000,&pcap_dump,(u_char *)pd);
j = j+pcount;
printf("Got %d packets , total packets %d\n",pcount,j);
}
And for decoding the packets, I am using the following code
while( (returnValue=pcap_next_ex(pcap,&header,&data)) >= 0)
{
printf("Packet # %d ",++packetCount);
printf("return value %d\n",returnValue);
}
When i run the program separately i.e when I stop writing to the pcap file, It decodes the packets perfectly. But when I run both the programs at the same time, the decoder does not decode all the packets. If i get 100 packets, the decoder will show only 50-60 decoded packets.
Any help will be appreciated.
In my opinion, reader's file is not getting updated as soon as the writer writes in the pcap file. This might be due to the reason that reader's file pointer is not refreshed i.e its reading the non updated version of file.
Hope it will help you.
This is what pipes are for. I suggest something like
pcap_writer -w - | tee permanent-file.pcap | pcap_reader -r -
where pcap_writer and pcap_reader are your programs. This way you create something that can be combined in a different manner, if wanted.
Related
I've got several USB to 422 adapters in my test system. I've used FTProg to give each adapter a specific name: Sensor1, Sensor2, etc. They will all be plugged in at power on. I don't want to hard code each adapter to a specific ttyUSBx. I want the drivers to figure out which tty it needs to use. I'm developing in C for a linux system. My first thought was to something like this in my startup code.
system("dmesg | find_usb.py");
The python script would find the devices since each one has a unique Product Description. Then using the usb tree to associate each device with its ttyUSBx. The script would then create /tmp/USBDevs which would just be a simple device:tty pairing that would be easy for the C code to search.
I've been told...DoN't UsE sYsTeM...use posix_spawn(). But I'm having problems getting the output of dmesg piped to my python script. This isn't working
char *my_args[] = {"dmesg", "|", "find_usb.py", NULL};
pid_t pid;
int status;
status = posix_spawn(&pid, "/bin/dmesg", NULL, NULL, my_args, NULL);
if(status == 0){
if(waitpid(pid, &status, 0) != -1);{
printf("posix_spawn exited: %i", status);
}
I've been trying to figure out how to do this with posix_spawn_file_actions(), but I'm not allowed to hit the peak of the 'Ballmer Curve' at work.
Thanks in advance
Instead of using /dev/ttyUSB* devices, write udev rules to generate named symlinks to the devices. For a brief how-to, see here. Basically, you'll have an udev rule for each device, ending with say SYMLINK+=Sensor-name, and in your program, use /dev/Sensor-name for each sensor. (I do recommend using Sensor- prefix, noting the initial Capital letter, as all device names are currently lowercase. This avoids any clashes with existing devices.)
These symlinks will then only exist when the matching device is plugged in, and will point to the correct device (/dev/ttyUSB* in this case). When the device is removed, udev automagically deletes the symlink also. Just make sure your udev rule identifies the device precisely (not just vendor:device, but serial number also). I'd expect the rule to look something like
SUBSYSTEM=="tty", ATTRS{idVendor}=="VVVV", ATTRS{idProduct}=="PPPP", ATTRS{serial}=="SSSSSSSS", SYMLINK+="Sensor-name"
where VVVV is the USB Vendor ID (four hexadecimal digits), PPPP is the USB Product ID (four hexadecimal digits), and SSSSSSSS is the serial number string. You can see these values using e.g. udevadm info -a -n /dev/ttyUSB* when the device is plugged in.
If you still insist on parsing dmesg output, using your own script is a good idea.
You could use FILE *handle = popen("dmesg | find_usb.py", "r"); and read from handle like it was a file. When complete, close the handle using int exitstatus = pclose(handle);. See man popen and man pclose for the details, and man 2 wait for the WIFEXITED(), WEXITSTATUS(), WIFSIGNALED(), WTERMSIG() macros you'll need to use to examine exitstatus (although in your case, I suppose you can just ignore any errors).
If you do want to use posix_spawn() (or roughly equivalently, fork() and execvp()), you'd need to set up at least one pipe (to read the output of the spawned command) – two if you spawn/fork+exec both dmesg and your Python script –, and that gets a bit more complicated. See man pipe for details on that. Personally, I would rewrite the Python script so that it executes dmesg itself internally, and only outputs the device name(s). With posix_spawn(), you'd init a posix_file_actions_t, with three actions: _adddup2() to duplicate the write end of the pipe to STDOUT_FILENO, and two _addclose()s to close both ends of the pipe. However, I myself prefer to use fork() and exec() instead, somewhat similar to the example by Glärbo in this answer.
So the other day I got a relatively simple assignment, which was to build some client and some server code that in which the server received a message and returned its current system time. It wasn't hard to do, i delivered and got an easy mark.
I started thinking a bit more on it and I decided to set out and try to send the contents of specific file server --> client (server sends contents to client). While i was building the code I tested every so often locally and it worked as intended, the real problem arose when I uploaded the server code to my server (hah) running Ubuntu. Started the server, everything OK, started the client, asked for "index.html" and BAM! half the file wasn't received.
The server prints it (I made it so it printed the contents of file as it sent so I could troubleshoot with more ease).
I have been searching for a bit now and every time I find something that looks useful it ends being in a different programming language and cant find any equivalents in C.
Using sleeps in both client and server code seems to solve this, but I figure it's not good practice.
The code is a mess so I'll include what I figure is relevant, I'll also include a link tot he full code. I really meant to improve it but so got demotivated while trying to fix this that I just made it worse.
Client side
printf("Please specify the filename: ");
fgets(msg,1000,stdin); // get message from std input
if(strcmp(msg,"\n")==0) {
printf("Wrong file name or format\n");
printf("Please specify the filename: ");
fgets(msg,1000,stdin); // get message from std input
}
while(strcmp(msg,"!stop\n")) {
msg[strlen(msg)-1]='\0';
write(sockfd,msg,strlen(msg));
FILE *fp = NULL;
char filecontent[1000];
bzero(filecontent,sizeof(filecontent));
while( (n = read(sockfd,filecontent,1000)) && strcmp(filecontent,"Over and out!")!=0 ) {
if(strcmp(filecontent,"No such file")!=0 && fp == NULL) {
fp = fopen(msg,"w");
}
printf("%s",filecontent);
if(fp !=NULL)
fprintf(fp, "%s",filecontent);
bzero(filecontent,sizeof(filecontent));
}
if(fp != NULL)
fclose(fp);
printf("\nPlease specify the filename: ");
fgets(msg,1000,stdin); // get message from std input
if(strcmp(msg,"\n")==0) {
printf("Wrong file name or format\n");
printf("Please specify the filename: ");
fgets(msg,1000,stdin); // get message from std input
}
}
Server side
char date[50];
time_t ticks;
struct tm *tinfo;
time(&ticks);
tinfo=localtime(&ticks);
strcpy(date,asctime(tinfo));
printf("DATA: %s\n",date);
write(newsocketfd,date,sizeof(date));
while( (n = read(newsocketfd,msg,1000)) && strcmp(msg,"!stop\n")!=0) {
//printf("MSG: %s\n",msg);
if(n<0)
error("ERROR READING");
/////////READING FILE/////////////
char *filename = malloc(sizeof(msg)+1);
strcpy(filename,msg);
printf("'server filename:%s'\n",filename);
FILE *fp = fopen( filename,"r");
if(fp == NULL) {
printf("No such file found\n");
write(newsocketfd,"No such file",sizeof("No such file"));
}
while( fp!=NULL && fgets(msg,1000,fp)!=NULL){
write(newsocketfd,msg,sizeof(msg));
msg[strlen(msg)-1]='\0';
printf("server: '%s'\n",msg);
bzero(msg,sizeof(msg));
}
bzero(msg,sizeof(msg));
bzero(filename,strlen(filename));
n = write(newsocketfd,"Over and out!",sizeof("Over and out!"));
printf("Over\n");
}
sorry for any headaches. Full code here.
Examples:
I think this pretty much shows the problem
My thinking was, the server reads the file, line by line, and sends its, line by line, to the client, when it's done the server sends "over" and the client stops reading from there, it seems however that the client never receives all the information or the "over" signal. Worth adding that this works perfectly fine if I run both codes on my local machine.
Welcome to the world of network programming! Network protocols are layered for a reason. When you send something on a TCP socket, and immediately close the socket, the delivery is unreliable: it may be correctly delivered to the peer, or may vanish because of race conditions.
The only reliable way is to only close the socket when the peer sends an acknowledgement that it could receive everything that was sent. Standard protocol use control messages for that, and you really should contemplate that, but if you do not need your server to be warned for client failures, you could simply have the client to close the connection when it has received "Over and out!". BTW, you should be aware that as TCP is a stream protocol, nothing can guarantee that the message will not be splitted in more than one read, or concatenated to other bytes. So you should keep the end of the previous read (size of the signal string minus one byte), concatenate next read to that and search the string anywhere in the buffer.
Another common way is to use a graceful shutdown: the sender uses shutdown(socket.SHUT_WR) to signal that the communication is over without closing the socket and waits (with a read) for the peer to close the socket when everything has been correctly delivered.
I need to save UDP packets to a file and would like to use the pcap format to reuse the various tools available (wireshark, tcpdump, ...).
There are some information in this thread but I can't find how to write the global file header 'struct pcap_file_header'.
pcap_t* pd = pcap_open_dead(DLT_RAW, 65535);
pcap_dumper_t* pdumper = pcap_dump_open(pd, filename);
struct pcap_file_header file_hdr;
file_hdr.magic_number = 0xa1b2c3d4;
file_hdr.version_major = 2;
file_hdr.version_minor = 4;
file_hdr.thiszone = 0;
file_hdr.sigfigs = 0;
file_hdr.snaplen = 65535;
file_hdr.linktype = 1;
// How do I write file_hdr to m_pdumper?
while( (len = recvmsg(sd, &msg_hdr, 0)) > 0 )
pcap_dump((u_char*)m_pdumper, &m_pcap_pkthdr, (const u_char*)&data);
How should I write the global file header?
If there is no specific pcap function available, how can I retrieve the file descriptor to insert the header using write()?
You shouldn't need to write that header, pcap_open_dead should do it for you. You only need to fill out and write that header yourself if you want to write the file directly instead of using pcap_dump and friends. There's an example here of a trivial program write out a pcap file with those functions.
original answer, concerning writing the file directly:
I can't remember exactly how this works, but I wrote a patch to redir a while ago that would write out pcap files, you may be able to use it as an example.
You can find it attached to this debian bug. (bug link fixed.)
Some of it is for faking the ethernet and IP headers, and may not be applicable as you're using pcap_dump_open and pcap_dump where as the patch linked above writes out the pcap file without using any libraries, but I'll leave this here anyway in case it helps.
If you are interested in UDP and TCP only, you should use DLT_EN10MB instead of DLT_RAW ( cf pcap_open_dead to simulate full UDP packets capture ).
It is much better when editing in WireShak.
I am working with libpcap to check http info. libpcap can not reassemble tcp segment.
there are many corner cases to deal manually. I also read wireshark source code. It's too big.
Does have any open source code can reassemble tcp and disect http data in c?
Have hacked code of driftnet, tcpflow, pcap etc. earlier.
tcpflow can re-assemble dumps from ie tcpdump. A "typical" chain of work could be:
$ tcpdump -nnieth0 -w dump.raw
# dump dum dump
$ mkdir tmp && cd tmp
tmp/$ tcpflow -r ../dump.raw
# This joins the transfers into separate files
# Now one can investigate each transfere from those separate files
# Next join them to one:
tmp/$ cat * > ../dump.flow
tmp/$ cd ..
# Extract some data
$ foremost -i dump.flow
Believe you can find some useful lines in the source-code for these.
Else:
A HTTP parsing library: HTTP Parser
Easiest way to do this is download wireshark software open pcap file or live capture it. after that right click any packet and go to "follow tcp stream"...you will see your http data in opened window.
If you want to build a program from scratch.-
A pcap file structure for tcp transaction is something like this:
[pcap_file_header]
for each packet
[pcap_packet] --this contains packet len info
[ip_header]----usually of size 20 or more
[tcp_header]--usually of size 20 or more
[packet] ---len stored in pcap packet header
Now to read the info, first get your pcap file in stream pointer.
Read packet file header(google for various type of struct)
start a loop
for each packet
read pcap_phdr from file or stream
add offset of ip_hdr length and tcp hdr length
for example pointer=pointer+20(for ip)+20(for tcp)
the pointer should be pointing to your data
so just give read pcap_phdr.caplen amount of byte and print it character by character.
The smallest TCP/IP stack I know of, which is open source is uIP but it is a bit odd, as it is designed for extremely small systems (microcontrollers)
Another small Open Source TCP/IP stack, which is a bit more traditional is lwIP
I am building a server client model in C. The clients connects to the server and they start exchanging data. However, the user can end the client at any time in the program, but the server is not notified about it. The server keeps sending that data even after the client is closed.
I was in the impression that send function will return -1 if the server is unable to send the data, but my server program just stuck at send
if((byteSent = send(new_fd, fileContents, strlen(fileContents), 0)) == -1){ //
the program just halts at the above line.
How do I overcome this problem?
//Code
exitT = 0;
//execution_count = 1;
for(i=0;i<execution_count;i++)
{
sleep(time_delay);
//getting the current time on the server machine
time_t t;
time(&t);
char *time=ctime(&t);
printf("The Execution time at server = %s\n",time);
system(exec_command);
/*Open the file, get file size, read the contents and close the file*/
// Open the file
fp = fopen(fileName,"r");
// Get File Size
fseek(fp,0,SEEK_END);
dataLength = ftell(fp);
rewind(fp);
fileContents = (char*)malloc(dataLength+1);
// Read File
fread(fileContents,1,dataLength,fp);
fileContents[dataLength] = '\0';
// Close file
fclose(fp);
printf("sockfd = %d \n",new_fd);
// send file length to client
rc=send(new_fd, &dataLength, sizeof(dataLength), 0) ;
printf("length of client data = %d \n",rc);
printf("sockfd = %d \n",new_fd);
// send time to client
rc=send(new_fd, time, strlen(time), 0) ;
printf("length of client time = %d \n",rc);
usleep(20000);
// Send file contents to Client
while(dataLength>0){
printf("sockfd = %d \n",new_fd);
if((byteSent = send(new_fd, fileContents, strlen(fileContents), 0)) == -1){
printf("bytes sent = %d \n",byteSent);
exitT = 1;
break;
}
dataLength-=byteSent;
}
//Delete the log file
sprintf(deleteCommand,"rm %s",fileName);
system(deleteCommand);
if(exitT == 1)
break;
}
bzero(fileName,sizeof(fileName));
bzero(exec_command,sizeof(exec_command));
bzero(deleteCommand,sizeof(deleteCommand));
//decClientNum();
kill(parent_id,SIGALRM);
close(new_fd); // parent doesn't need this
printf("STATUS = CLOSED\n");
exit(0);
}
Thanks
I assume you are coding for a Linux or Posix system.
When a syscall like send fails it returns -1 and sets the errno; you very probably should use errno to find out why it failed.
You could use strace to find out which syscalls are done by your sever, or some other one. Of course, use also the gdb debugger.
You very probably need to multiplex inputs or outputs. The system calls doing that are poll, select (and related ppoll and pselect). Read e.g. the select_tut(2) man page.
You may want to use (or at least to study the source code of) existing event oriented libraries like libevent, libev etc.. (Both Gtk and Qt frameworks provide also their own, which might be used even outside of GUI applications).
I strongly suggest reading about advanced unix programming and unix network programing (and perhaps also about advanced linux programming).
maybe you're using a tcp protocol and the server is waiting for an ACK. Try using udp if you want your connection to be asynchronous.
From the man page: No indication of failure to deliver is implicit in a send(). Locally detected errors are indicated by a return value of -1.
Proably something like this might help: http://stefan.buettcher.org/cs/conn_closed.html
I think I am pretty late in the party, but I think this answer might help someone.
If space is not available at the sending socket to hold the message to be transmitted, and the socket file descriptor does not have O_NONBLOCK set, send() shall block until space is available.
When send() function gets stuck, there might be a situation like, TCP window size has become 0. It happens when the other end of the connection is not consuming received data.
There might be a scenario like this, the receiving end process is running by GDB and segfault occurred.
The TCP connection remains established.
Data is being send continuously.
The receiver end is not consuming it.
Consequently the receiver TCP window size will keep decreasing and you can send data till it is greater than zero. Once it becomes 0, send() function will get stuck forever.
As the situation mentioned in the question is not a scenario of closed connection. When a process writes something on a closed TCP connection, it receives a signal SIGPIPE. Default handler of SIGPIPE terminates the process. So, in a closed connection scenario if you are not using your own SIGPIPE handler then process should be terminated by default handler whenever something is written on the socket.