I have a function that basically does this:
int mmkdir(const char *path, mode_t mode)
{
struct stat st;
if (stat(path, &st) < 0)
{
if (errno != ENOENT)
return -1;
if (mkdir(path, S_IWUSR | S_IRUSR | S_IXUSR | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH) < 0 && errno != EEXIST)
return -1;
}
else if (!S_ISDIR(st.st_mode))
{
errno = ENOTDIR;
return -1;
}
return 0;
}
int fun(int id, int id2)
{
time_t ts;
struct tm timeinfo;
char buff[1024], buff1[20], buff2[20], buff3[20];
ts = time(NULL);
localtime_r(&ts, &timeinfo);
strftime(buff1, sizeof(buff1), "%Y%m%d", &timeinfo);
strftime(buff2, sizeof(buff2), "%H", &timeinfo);
snprintf(buff, sizeof(buff), "%s/%s", dir, buff1);
if (mmkdir(buff, ORDER_FILE_DEFAULT_PERMS) < 0)
return -1;
len = strlen(buff);
snprintf(buff + len, sizeof(buff) - len, "/%s", buff2);
if (mmkdir(buff, ORDER_FILE_DEFAULT_PERMS) < 0)
return -1;
len += strlen(buff2) + 1;
strftime(buff3, sizeof(buff2), "%M%S", &timeinfo);
snprintf(buff + len, sizeof(buff) - len - 1, "/%010d-%04d-%s%s.%s",
id, id2, buff2, buff3, ext);
return open(buff, O_WRONLY | O_APPEND | O_CREAT, S_IWUSR | S_IRUSR | S_IRGRP | S_IROTH);
}
Plese don't check for errors, it's a working code. It basically calls stat(2), mkdir(2) (sometimes) and open(2).
The problem is that when to I/O load on the server is pretty high, this piece of code sometimes takes even 7s (!!) to complete.
These files that this functions create are located in a folder in /, which is mounted:
/dev/md0 on / type ext4 (rw,errors=remount-ro)
and can be up to 1000 files in a single folder.
What may be the problem? Why this may take so long? Is there any misconfiguration?
I don't ask only for improvements in the code, but in the server configuration also, if possible.
As per request in the comments, the output of cat /proc/mdstat is:
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sdd1[1] sdc1[0]
488282000 blocks super 1.2 [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
15624120 blocks super 1.2 [2/2] [UU]
md2 : active raid1 sda3[0] sdb3[1]
1958900 blocks super 1.2 [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
470798200 blocks super 1.2 [2/2] [UU]
that means that the disks are ok
If this function is called substantially more than once per hour, and the folders are not deleted, then your mmkdir calls are redundant. I would implement some kind of caching scheme where you remember the last hour in which you created a folder, and skip creating a new folder if you already created it. This will remove two stat calls, which could make a big difference on an overloaded system.
There's likely to be no further improvement you can make to the code alone, since the open call would be the only remaining system call, and it cannot be elided without changing the meaning of the function.
Related
I have the following function used to test the file system (yaffs2). It is called by a FreeRTOS task with a stack size of 65535. In debug mode, everything appears to work fine, but when I enable -O1, I get a hard fault immediately after printing "Testing filesystem...".
I tried changing the test size to a much lower value, thinking maybe there is a memory issue (note that the FreeRTOS heap is 7MB wide), but nothing changed. Any ideas what might be causing this?
#include <stdio.h>
#include "FreeRTOS.h"
#include "task.h"
#include "yaffs_trace.h"
#include "yaffsfs.h"
int fs_test(void)
{
int fd;
int test_size = 1000000;
const char path[] = "nand/testfile.dat";
int pattern = 0x55;
uint8_t *test;
uint8_t *test_read;
uint32_t start_time, stop_time;
DEBUG_PRINT("Testing filesystem...\n");
fd = yaffs_open(path, O_CREAT | O_EXCL | O_WRONLY, S_IREAD | S_IWRITE);
if(fd < 0) {
DEBUG_PRINT("cannot create test file\n");
return -1;
}
test = pvPortMalloc(test_size);
test_read = pvPortMalloc(test_size);
if(test == NULL || test_read == NULL) {
DEBUG_PRINT("not enough memory\n");
vPortFree(test);
vPortFree(test_read);
yaffs_close(fd);
yaffs_rm(path);
return -2;
}
memset(test, pattern, test_size);
start_time = xTaskGetTickCount();
yaffs_write(fd, test, test_size);
yaffs_close(fd);
stop_time = xTaskGetTickCount();
DEBUG_PRINTF("Average write speed = %d kB/s\n", test_size / (stop_time-start_time));
fd = yaffs_open(path, O_RDWR, S_IREAD | S_IWRITE);
if(fd < 0) {
DEBUG_PRINT("cannot open test file\n");
vPortFree(test);
vPortFree(test_read);
yaffs_close(fd);
yaffs_rm(path);
return -1;
}
start_time = xTaskGetTickCount();
yaffs_read(fd, test_read, test_size);
yaffs_close(fd);
stop_time = xTaskGetTickCount();
DEBUG_PRINTF("Average read speed = %d kB/s\n", test_size / (stop_time-start_time));
if (!memcmp(test, test_read, test_size))
DEBUG_PRINT("file integrity test successful\n");
else
DEBUG_PRINT("file integrity test failed!\n");
yaffs_rm(path);
vPortFree(test);
vPortFree(test_read);
return 0;
}
I want to create N files in a for loop under Linux as O.S.;
This is my code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main() {
FILE *ptr;
char name[25];
for (int i = 0; i < 10; i++) {
snprintf(name, sizeof(name), "File_Nr%d.txt", i);
ptr = fopen(name, "w");
if( ptr == NULL ){
perror("Error creating file!");
}
fclose(ptr);
}
return 0;
}
It works and creates File_Nr0.txt to File_Nr9.txt.
Question: is this code "safe"?
Perhaps since you want to only create the files, you can directly use the open() system call, which has more options and more readable way of expressing options IMO.
int fd = open(name, O_WRONLY|O_CREAT|O_TRUNC,
S_IWUSR | S_IRUSR | S_IWGRP | S_IRGRP | S_IROTH);
or
int fd = creat(name, 0644) // consider it to be an alias for above.
if you want to fail when the file exists already,
int fd = open(name, O_WRONLY|O_CREAT|O_TRUNC|O_EXCL, 0644);
in fopen() you can achieve this by using the 'x' in mode param.
ptr = fopen(name, "wx");
And as #xception mentioned, you should return a non-zero error code when something goes wrong.
if (NULL == ptr) {
perror("...");
return -1; // or better the errno which was set by open() call..
}
And in your code, you attempt to close the ptr even in failure case, that is gonna give you a crash. So, you need to handle that.
I am trying to recreate the basic functionality of fopen() using the I/O system calls. I am somehow suppose to "set the default mode for open() calls with O_CREAT" however am unsure how to go about this. It's not perfect but this is what I've got so far.
MYFILE * myfopen(const char * path, const char * mode){
MYFILE *fp = (MYFILE *)malloc(sizeof(MYFILE)); //EDITED
int fd;
switch(mode[0]){
case 'r':
fd=open(path, O_RDONLY | O_CREAT, 0440);
break;
case 'w':
fd=open(path, O_WRONLY | O_CREAT | O_TRUNC, 0220);
break;
default:
fd=open(path, O_RDWR | O_CREAT | O_TRUNC, 0660);
}
if(fd < 0){
return NULL;
}
fp->fileD=fd;
fp->offset=0;
return fp;
}
fileD will be the file descriptor returned by the open call. I think the rest is self-explanatory.
It compiles but I'm getting a "segmentation fault" error when I try to run it. This function also fails to open a new file and associate a file descriptor to it.
I think the segmentation error might be somewhere in here:
int myfputc(int c, MYFILE * fp){
if(write(fp->fileD, &c, 1) == -1){
return 1;
}
++fp->offset; //how to gain access to the current offset of the stream?
return 0;
}
Where I'm trying to recreate fputc.
Here is the MYFILE struct:
typedef struct {
int fileD; // descriptor
int bufferSz; // buffer size
int bufferCh; // # of bytes in stream
int offset; //current offset position
int errorF; // error flag
int EOFF; // EOF flag
} MYFILE;
The file permissions should probably be 0644, or maybe 0666 (or maybe 0640/0660 to deny others access while allowing your group access), regardless of whether you're creating the file for reading or writing. You should not normally include execute permission (and you don't). I'd be willing to support fewer permissions for group (no write for group seems good to me). You can even make a file readonly for every other process while the current process has write permission with 0444 or tighter permissions. But the standard will use 0666 and let the umask remove permissions.
You might note that your code leaks if you fail to open the file. You should free(fp); before the return on the error path.
Note that you have not set all the fields in your structure, and neither has malloc(), so you have random junk in those fields. Curiously, there isn't a buffer in sight, even though there's a buffer size.
This code works for me. It cleans up, marginally, your myfopen() function, but otherwise, it runs without crashing. I think your problem is in other code.
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
typedef struct
{
int fileD; // descriptor
int bufferSz; // buffer size
int bufferCh; // # of bytes in stream
int offset; // current offset position
int errorF; // error flag
int EOFF; // EOF flag
} MYFILE;
static
MYFILE *myfopen(const char *path, const char *mode)
{
MYFILE *fp = (MYFILE *)malloc(sizeof(*fp));
int fd;
switch (mode[0])
{
case 'r':
fd = open(path, O_RDONLY | O_CREAT, 0640);
break;
case 'w':
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
break;
default:
fd = open(path, O_RDWR | O_CREAT | O_TRUNC, 0640);
break;
}
if (fd < 0)
{
free(fp);
return NULL;
}
fp->fileD = fd;
fp->offset = 0;
fp->bufferSz = 0;
fp->bufferCh = 0;
fp->errorF = 0;
fp->EOFF = 0;
return fp;
}
static
int myfputc(int c, MYFILE *fp)
{
if (write(fp->fileD, &c, 1) == -1)
{
return 1;
}
++fp->offset; // how to gain access to the current offset of the stream?
return 0;
}
int main(void)
{
MYFILE *fp = myfopen("./test.txt", "w");
if (fp != 0)
{
const char *src = "The text!\n";
while (*src != '\0')
myfputc(*src++, fp);
}
return 0;
}
Result:
$ ls -l test.txt
-rw-r----- 1 jleffler staff 10 Feb 15 19:11 test.txt
$ cat test.txt
The text!
$
I have the following function (that dumps a process memory region). If I write to stdout write(STDOUT_FILENO, buf, rd); it outputs the buffer correctly, the problem rises when I want to write the buffer to a file, the file gets written but with the same date over and over:
void dump_region(int fd, off64_t start, off64_t end)
{
char buf[4096];
int fdo;
fdo = open("memdump_log", O_WRONLY | O_CREAT, 0644);
if (fdo == -1) {
fprintf(stderr, "open failed: %m\n");
close(fd);
exit(1);
}
lseek64(fd, start, SEEK_SET);
while(start < end) {
int rd;
rd = read(fd, buf, 4096);
write(fdo, buf, rd);
//write(STDOUT_FILENO, buf, rd);
start += 4096;
}
close(fdo);
}
The function is accessed from main() like this:
if(maps && mem != -1) {
char buf[BUFSIZ + 1];
while(fgets(buf, BUFSIZ, maps)) {
off64_t start, end;
sscanf(buf, "%llx-%llx", &start, &end);
dump_region(mem, start, end);
}
}
Any idea where am I wrong?
Modify
fdo = open("memdump_log", O_WRONLY | O_CREAT, 0644);
into
fdo = open("memdump_log", O_WRONLY | O_CREAT | O_APPEND, 0644);
You need to seek to the end of your output file, or passing the O_APPEND to open
You keep reopening the output file on every call to dump_region. When opening a file it will always start writing at the start. Either keep the file open all the time, seek to the end, or try the O_APPEND flag.
i've trying to perform appending to an existing file from resource file using the C programming in Linux. However, my code doesn't work for that, can any1 do tell me what's wrong with the code and also how O_APPEND is working? thanks :)
char ifile[150];
char tfile[150];
char cc;
system("clear");
printf("Please enter your resource file name : ");
gets(ifile);
printf("Please enter your destination file name : ");
gets(tfile);
int in, out;
in = open(ifile, O_RDONLY);
int size = lseek(in,0L,SEEK_END);
out = open(tfile, O_WRONLY |O_APPEND);
char block[size];
int pdf;
while(read(in,&block,size) == size)
pdf = write(out,&block,size);
close(in);close(out);
if(pdf != -1)
printf("Successfully copy!");
else
perror("Failed to append! Error : ");
printf("Press enter to exit...");
do
{
cc = getchar();
} while(cc != '\n');
The problem here is that you deplace the reading cursor at the end of the file in order to know its size, but you don't rewind to the start of the file to be able to read. So read() reads EOF, and returns 0.
int size = lseek(in, 0L, SEEK_END);
out = open(tfile, O_WRONLY | O_APPEND);
should be
int size = lseek(in, 0L, SEEK_END);
lseek(in, 0L, SEEK_SET);
out = open(tfile, O_WRONLY | O_APPEND);
In addition, when you read and write, you should use block and not &block, since block is already a pointer (or an address).
Oh, and also... When you open the file out for writing... It will fail if the file does not exist already.
Here how to create it with rights set to 644:
out = open(tfile, O_WRONLY | O_APPEND | O_CREAT, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
(This won't have any effect if the file already exists)