I have gone through the C document at avro
and I see that I can get only avro output to file. How do I get the serialized output to a buffer so that I can send over a tcp socket. Any help is much appreciated.
There is an avro_writer_memory() exactly for this case, it takes buffer pointer and length as parameters and gives you avro_writer_t that can be used in regular write functions. You can find its usage in tests, like this or this. The minimum example is going to be something like this (outputting encoded value to stderr, so better redirect that to some file and examine it after program run):
#include <avro.h>
#include <stdio.h>
#include <unistd.h>
static const char json_schema[] = "{ \"type\": \"string\" }";
int main(void)
{
char buf[1024];
avro_writer_t writer;
avro_schema_t schema;
avro_value_iface_t* iface;
avro_value_t val;
size_t len;
if (avro_schema_from_json_literal(json_schema, &schema) != 0) {
printf("failed to initialize schema\n");
goto out;
}
if ((writer = avro_writer_memory(buf, sizeof(buf))) == NULL) {
printf("failed to initialize writer\n");
goto out_schema;
}
if ((iface = avro_generic_class_from_schema(schema)) == NULL) {
printf("failed to get class from schema\n");
goto out_writer;
}
if (avro_generic_value_new(iface, &val) != 0) {
printf("failed to create new value\n");
goto out_iface;
}
if (avro_value_reset(&val) != 0) {
printf("failed to reset value\n");
goto out_val;
}
if (avro_value_set_string(&val, "some string wrapped by avro") != 0) {
printf("failed to set value string\n");
goto out_val;
}
if (avro_value_write(writer, &val) != 0) {
printf("failed to write value into the buffer\n");
goto out_val;
}
len = avro_writer_tell(writer);
printf("got %lu bytes\n", (unsigned long)len);
if (write(STDERR_FILENO, buf, len) != len) {
printf("failed to write to stderr, oops\n");
goto out_val;
}
out_val:
avro_value_decref(&val);
out_iface:
avro_value_iface_decref(iface);
out_writer:
avro_writer_free(writer);
out_schema:
avro_schema_decref(schema);
out:
return 0;
}
Also, there is an avro_writer_memory_set_dest() that allows to set new buffer to use by the existing writer.
I'm working on a Server in C using a Multi-Process / Multi-Threaded architecture. At the start I've a main process that forks 10 different processes. Each process creates a pool of threads. Each process is, also, the controller of its pool (creates new thread / extends the pool when needed).
The processes also listen on the socket (whose access is controlled by the main process with an unnamed semaphore in a shared memory area) and pass the socket file descriptor to one of the thread in their pool when a connection is acquired. The chosen thread (which is awakened by a pthread_cond_signal) reads from the socket with a recv using the option MSG_DONTWAIT.
ssize_t readn, writen;
size_t nleft;
char *buff;
buff = malloc(sizeof(char) * BUFF_SIZE);
if (buff == NULL) {
perror("malloc");
return EXIT_FAILURE;
}
char *ptr = buff;
/*if (fcntl(connsd, F_SETFL, O_NONBLOCK) == -1) { // set to non-blocking
perror("fcntl");
exit(EXIT_FAILURE);
}*/
errno = 0;
nleft = BUFF_SIZE;
while(nleft > 0) {
if ((readn = recv(connsd, ptr, nleft, MSG_DONTWAIT)) < 0) {
if (errno == EAGAIN || errno == EWOULDBLOCK) {
*ptr = '\0';
break;
}
else if (errno == EINTR)
readn = 0;
else {
perror("read");
return EXIT_FAILURE;
}
}
else if (readn == 0)
break;
if (buff[strlen(buff)-1] == '\0') {
break;
}
nleft -= readn;
ptr += readn;
}
The problem is that sometimes, when I try to connect to the server using Chrome (or Firefox) 3 threads seems to receive the HTTP Request but each one of them simply closes the connection because of this portion of code.
if (buff[strlen(buff)-1] != '\0') {
printf("%s\n",buff);
fflush(stdout);
buff[strlen(buff)-1] = '\0';
}
errno = 0;
if (strlen(buff) < 1) {
perror("No string");
if (shutdown(connsd,SHUT_RDWR) < 0) {
perror("shutdown");
return EXIT_FAILURE;
}
if (close(connsd) < 0) {
perror("close");
return EXIT_FAILURE;
}
return EXIT_FAILURE;
}
Other times we have 3 threads with different behaviours: one of them receives and reads from the socket the first HTTP Request (GET / HTTP/1.1). The second one is empty (request received (because it is awakened (?)) no string read). The third one receives and reads another HTTP Request (GET /favicon.ico HTTP/1.1).
Where is the problem that cause these behaviours? I can add other portions of code if needed.
Thank you very much for your time.
I am doing a university project, we have to use the Unix system call.
Right now, I'm really struggling to understand if, in my project, there really is a mistake. This is because, while in terminal it compiles and it starts and finishes without error, on xcode I get several errors.
In particular, I get errors when using semaphores.
I'll try to explain what errors, I receive, but since I'm not native English speakers forgive me in advance if I make some mistakes.
First, the program creates a number of child processes with a fork (). It does depending on how many clientei.txt located (i = iterator).
Immediately I block parent with a semaphore, I run the child up to a certain point, then I block it with a semaphore and I restart the parent.
At this point, the parent should read a message sent by his son, call a function to print the content inside a log.txt and restart the son.
Then the child does other things (including erase the message) and it block.
The parent restart, and everything is repeated for subsequent children.
While in terminal synchronization is perfect (everything happens at the right time without error) this both Linux and Mac, about XCode I had several errors:
semop: Resource temporarily unavailable (if I created more than 5 txt)
semop: File too large (if I created more than 2)
with 2 instead gave me two errors:
semop 1: Interrupted system call (this stops after running both processes)
semop 3: Identifier removed (with this in restarting the second process)
is not so much time that I do C then I do not know what to do. I would like first of all to know if I have to worry (so there is an error), or I have to be quiet because it is a bug in xcode.
If there was a mistake I kindly ask you not to ask me to change the code a lot.
This is mainly because they are close to expiring and I can not afford to do it all again.
I also ask you, if you can, to be as clear as possible. I understand enough English, but not as a mother-tongue, I can not always follow the responses here on StackOverflow.
The code is here:
https://www.dropbox.com/s/2utsb6r5d7kzzqj/xcode%2Bterminal.zip?dl=0
this zip contain a small part of the project that has this problem.
the terminal version works. there is a makefile in this version to simplify the compilation.
xcode version does not work. It contains the Debug folder. Indeed xcode, txt files, it does not read from the root folder where the codes are contained in the folder where it creates the compiled. There is a readme in each case with the procedure in detail.
I tried to minimize, I commented all in English.
I removed the code that was not needed, but I added the file with all the include and functions that use.
here the code:
main.c
key_t key, key_2;
int semid, semid_2;
union semun arg;
union semun arg_2;
struct sembuf sb_2 = {0, -1, 0};
char* nome_file;
nome_file = (char*) malloc(sizeof(char*));
int numero_clienti;
//semaphore for all the child
struct sembuf sb[numero_clienti];
int i_c;
for (i_c = 0; i_c < numero_clienti; i_c++) {
sb[i_c].sem_num = i_c;
sb[i_c].sem_op = -1;
sb[i_c].sem_flg = 0;
}
//cretion of first SEMAPHORE
{
//key creation
if ((key = ftok("cliente0.txt", 'J')) == -1)
{
perror("ftok");
exit(EXIT_FAILURE);
}
//creation of the semaphore
if ((semid = semget(key, numero_clienti, 0666 | IPC_CREAT | IPC_EXCL)) == -1)
{
perror("semget");
exit(EXIT_FAILURE);
}
//set value of all child semaphore
for (i_c = 0; i_c < numero_clienti; i_c++) {
arg.val = 0;
if (semctl(semid, i_c, SETVAL, arg) == -1)
{
perror("semctl");
exit(EXIT_FAILURE);
}
}
}
//cretion of second SEMAPHORE
{
//key creation
if ((key_2 = ftok("cliente1.txt", 'J')) == -1)
{
perror("ftok");
exit(EXIT_FAILURE);
}
//creation of the semaphore
if ((semid_2 = semget(key_2, 1, 0666 | IPC_CREAT | IPC_EXCL)) == -1)
{
perror("semget");
exit(EXIT_FAILURE);
}
//set value of parent semaphore
arg_2.val = 0;
if (semctl(semid_2, 0, SETVAL, arg_2) == -1)
{
perror("semctl");
exit(EXIT_FAILURE);
}
}
while(fd > 0 && pid > 0){
j++;
close(fd);
pid = fork();
if(pid != 0)
{
i++;
sprintf(nome_file, "./cliente%d.txt", i);
fd = open(nome_file, O_RDONLY);
}
switch(pid)
{
//error case
case -1:
{
perror("Error during fork.");
exit(EXIT_FAILURE);
break;
}
//child case
case 0:
{
puts("Child: I'm a child");
messaggio(numero_clienti, j);
puts("Child: I have to do something");
//Start parent
sb_2.sem_op = 1;
if (semop(semid_2, &sb_2, 1) == -1)
{
perror("semop");
exit(1);
}
//, stop itself
sb[j].sem_op = -1;
if (semop(semid, &sb[j], 1) == -1)
{
perror("semop");
exit(1);
}
printf("Child: I have to do something else %d\n", getpid());
_exit(EXIT_SUCCESS);
break;
}
//parent case
default:
{
puts("Parent: I'm a parent");
//Stop itself
sb_2.sem_op = -1;
if (semop(semid_2, &sb_2, 1) == -1)
{
perror("semop padre");
exit(1);
}
puts("Parent: now I can send the message, my child is blocked");
//restart child
sb[j].sem_op = 1;
if (semop(semid, &sb[j], 1) == -1)
{
perror("semop");
exit(1);
}
//stop itself
sb_2.sem_op = -1;
if (semop(semid_2, &sb_2, 1) == -1)
{
perror("semop");
exit(1);
}
puts("Parent: end of while");
break;
}
}
}
puts("Parent: I can restart all my child");
for (i_c = 0; i_c < numero_clienti; i_c++) {
sb[i_c].sem_op = 1;
if (semop(semid, &sb[i_c], 1) == -1)
{
perror("semop");
exit(1);
}
}
puts("I wait the end of my child...");
while (wait(NULL) != -1);
puts("All child end");
//remove semaphore I create
if (semctl(semid, 0, IPC_RMID, arg) == -1)
{
perror("semctl");
exit(1);
}
if (semctl(semid_2, 0, IPC_RMID, arg_2) == -1)
{
perror("semctl");
exit(1);
}
puts("FINE");
return 0;
}
cliente.c
#include "cliente.h"
/**
inside this function child do some thing.
1. at this point it give control to parent after it create a message
2. at this point it remove the message
*/
void messaggio(int numero_clienti, int num_j){
key_t key, key_2;
int semid, semid_2;
struct sembuf sb[numero_clienti];
int i_c;
for (i_c = 0; i_c < numero_clienti; i_c++) {
sb[i_c].sem_num = i_c;
sb[i_c].sem_op = -1;
sb[i_c].sem_flg = 0;
}
struct sembuf sb_2 = {0, -1, 0};
if ((key = ftok("cliente0.txt", 'J')) == -1) {
perror("ftok");
exit(1);
}
if ((semid = semget(key, 1, 0)) == -1) {
perror("semget");
exit(1);
}
if ((key_2 = ftok("cliente1.txt", 'J')) == -1) {
perror("ftok");
exit(1);
}
if ((semid_2 = semget(key_2, 1, 0)) == -1) {
perror("semget");
exit(1);
}
//creation of a message
//1. Restart parent
sb_2.sem_op = 1;
if (semop(semid_2, &sb_2, 1) == -1)
{
perror("semop");
exit(1);
}
puts("cambio sem");
//stop itself
sb[num_j].sem_op = -1;
if (semop(semid, &sb[num_j], 1) == -1)
{
perror("semop");
exit(1);
}
//here it can move again
puts("remove message");
puts("Figlio: sono tornato attivo, mio padre aspetta");
}
1st you do
nome_file = (char*) malloc(sizeof(char*));
which allocates 4 or 8 bytes (depending on the platform you compile on: 32 or 64bit).
Then you do
sprintf(nome_file, "./cliente%d.txt", i);
The latter writes to invalid memory, as "./cliente%d.txt" is 14+1 characters long plus the potenial number of digits from i if i>9 or and addtional sign if i<0.
To fix this allocate what is needed:
nome_file = malloc(13 + 10 + 1 + 1); /* 13 for the filename,
10 for the digits,
1 for a potential sign,
1 the C-"strings" 0-terminator. */
This is a really ugly bug, which is expected to be the main issue in your code.
Also in the sources (you linked) in function read_line() you allocate memory, which you do not properly initialise, but later depend on its content.
main.c:20
char* myb2 = (char*) malloc(sizeof(char*));
malloc() does not initialise the memory it allocates, so either do:
char * myb2 = calloc(1, sizeof(char*));
of add and addtional call to
memset(mb2, 0, sizeof(char*));
after the call to malloc().
This bug is nasty either.
Also^2 you should build using gcc's options -std=c99 -D_XOPEN_SOURCE.
That is because:
You are using C constructs available from C99 on only. Typically VLAs, so tell the compiler to treat the code as being C99 code by explcitly stating -std=c99
To #define _XOPEN_SOURCE is issued by gcc, for some header you include in your project.
Also^3 you seem to be not necessarily count the correct number of client(file)s, at least not if your files a distributed as per the archive you linked:
main.c:82
system("ls cliente* | wc -l");
Change this to be:
system("ls cliente*.txt | wc -l");
If the bug described above should return more files then there actually are the following code fails as well from a certain value of i on:
main.c:176
fd = open(nome_file, O_RDONLY);
The result of the above operation is NOT tested. The possible invalid fd is used and the infamous undefined behaviour is taking over. Everything can happen.
As a final note: It's mostly never a bug in the tools we are using.
I currently have a rudimentary implementation of Bash written in C. However, I'm getting issues when I try to redirect the standard output twice. Here is the relevant code:
Reading in each command:
for ( ; ; ) {
printf ("(%d)$ ", nCmd); // Prompt for command
fflush (stdout);
if ((line = getLine (stdin)) == NULL) // Read line
break; // Break on end of file
list = lex (line);
free (line);
if (list == NULL) {
continue;
} else if (getenv ("DUMP_LIST")) { // Dump token list only if
dumpList (list); // environment variable set
printf ("\n");
}
cmd = parse (list); // Parsed command?
freeList (list);
if (cmd == NULL) {
continue;
} else if (getenv ("DUMP_TREE")) { // Dump command tree only if
dumpTree (cmd, 0); // environment variable set
printf ("\n");
}
process (cmd); // Execute command
freeCMD (cmd); // Free associated storage
nCmd++; // Adjust prompt
}
The part of the shell we're my code is messing up:
if (cmdList->type==SIMPLE)
{
pid_t fork_result;
fork_result = fork();
if (fork_result < 0) {
fprintf(stderr, "Fork failure");
exit(EXIT_FAILURE);
}
if (fork_result == 0) {
if (cmdList->fromType==RED_IN)
{
int fe = open(cmdList->fromFile, O_RDONLY, 0);
dup2(fe, 0);
close(fe);
}
if ((cmdList->toType==RED_OUT) || (cmdList->fromType==RED_APP))
{
int fd = open(cmdList->toFile, O_CREAT | O_WRONLY, 0666);
dup2(fd, 1);
close(fd);
}
execvp(cmdList->argv[0],cmdList->argv);
exit(EXIT_FAILURE);
}
else {
int status;
wait(&status);
}
}
This last snippet of code works exactly how I intend it to when I'm reading in just one simple command. However, the issue arises when I use the for loop to try to redirect stout twice. For example, I try to run:
cat Tests/star.wars > +Bash.tmp
cat +Bash.tmp
cat Tests/stk.txt > +Bash.tmp
cat +Bash.tmp
The first command writes, say, "ABC" to Bash.tmp. However, when I run the second command, I expect it to return "DE". However, I'm getting "DEC" as the output. What is wrong?
O_WRONLY is "write-only" permissions. O_TRUNC is what truncates the file on open.
– Etan Reisner
I'm coding an IRC client and I would like implement a "/server" command to switch the connection of my client to an other server.
Before initialize the new connection I want to close the sockect's fd but the close() call fail. Anybody could say me why ?
Here is my code :
/* Main execution loop */
FD_ZERO(&irc->rdfs);
FD_SET(STDIN_FILENO, &irc->rdfs);
FD_SET(irc->socket_fd, &irc->rdfs);
if ((select(irc->socket_fd + 1, &irc->rdfs, NULL, NULL, NULL)) == -1)
{
if ((close(irc->socket_fd)) == -1)
exit(usage(CLOSE_ERROR));
exit(usage(SELECT_ERROR));
}
if (FD_ISSET(STDIN_FILENO, &irc->rdfs))
{
fgets(irc->buffer, SIZE - 1, stdin);
{
p = strstr(irc->buffer, RET);
if (p != NULL)
*p = 0;
else
irc->buffer[SIZE - 1] = 0;
}
write_on_server(irc, irc->buffer); /* The function where I call switch_server() in */
}
else if (FD_ISSET(irc->socket_fd, &irc->rdfs))
{
if ((read_on_server(irc)) == 0)
exit(usage(SERVER_DISCONNECT));
puts(irc->buffer);
}
And here is where I'm trying to close my socket's fd :
void switch_server(t_irc *irc)
{
if ((close(irc->socket_fd)) == -1) /* This is the close which fail */
exit(EXIT_FAILURE);
}
void write_on_server(t_irc *irc, const char * buffer)
{
if (!(strncmp("/server", buffer, strlen("/server"))))
switch_server(irc);
else
if ((send(irc->socket_fd, buffer, strlen(buffer), 0)) < 0)
{
if ((close(irc->socket_fd)) == -1)
exit(usage(CLOSE_ERROR));
exit(usage(CLIENT_SEND_ERROR));
}
}
Thanks a lot.
If you want to know why a syscall like close() failed, use perror() to print an error message to stderr, or strerror(errno) to convert the error code to a string and output it some other way.
Almost certainly the socket FD is invalid. You need to call perror() on that, and on the select() failure.