Hi I'm working with OS161 and I'm trying to extend my inode structures so that when I write to disk, I write the first chunk of file data into the actual inode structure, because currently it's set up such that a large chunk of the structure is wasted.
struct sfs_inode {
u_int32_t sfi_size; /* Size of this file (bytes) */
u_int16_t sfi_type; /* One of SFS_TYPE_* above */
u_int16_t sfi_linkcount; /* Number of hard links to this file */
u_int32_t sfi_direct[SFS_NDIRECT]; /* Direct blocks */
u_int32_t sfi_indirect; /* Indirect block */
u_int32_t sfi_waste[128-3-SFS_NDIRECT]; /* unused space */
};
I'd like to replace the sfi_waste above with a char sfi_inlinedata[INLINE_SIZE]; so that
my disk io will always write/read the first INLINE_SIZE of data to the sfs_inode struct.
The following is the source I'm working from for the relevant io functions. I know I need to change the way that the offset maps to actual addresses in the io functions here, but I'm having trouble coming up with a concrete solution.
https://github.com/rbui/projectJailBait/blob/master/os161-1.11/kern/fs/sfs/sfs_vnode.c
Any help would be much appreciated!
Related
I'm currently working on a project involving the exploration of the linux kernel, particularly the XFS filesystem , right now I'm trying to read where a file resides on disk and i stumbled upon the xfs_ifork struct which looks like:
typedef struct xfs_ifork {
int if_bytes; /* bytes in if_u1 */
int if_real_bytes; /* bytes allocated in if_u1 */
struct xfs_btree_block *if_broot; /* file's incore btree root */
short if_broot_bytes; /* bytes allocated for root */
unsigned char if_flags; /* per-fork flags */
union {
xfs_bmbt_rec_host_t *if_extents;/* linear map file exts */
xfs_ext_irec_t *if_ext_irec; /* irec map file exts */
char *if_data; /* inline file data */
} if_u1;
union {
xfs_bmbt_rec_host_t if_inline_ext[XFS_INLINE_EXTS];
/* very small file extents */
char if_inline_data[XFS_INLINE_DATA];
/* very small file data */
xfs_dev_t if_rdev; /* dev number if special */
uuid_t if_uuid; /* mount point value */
} if_u2;
} xfs_ifork_t;
my lead on the u1 union, which encases different scenarios of file data fork, but i cant seem to have figured out the exact way to read them , the xfs_bmbt_rec_host_t struct refers to the disk address , written inside 2 uint64 types which encapsulates the entire address i need,
but somehow, the address I'm able to read isn't the correct one, so goes for the xfs_ext_irec_t struct, which holds the xfs_bmbt_rec_host_t struct and counters.
if someone happens to know theses structs well or have used them i would appreciate some explaining
I am trying to realize DMA in Xilinx, using DMA Engine. FPGA guys give me an AXI DMA IP-Core with DMA support, so I need to write a driver, which will transfer data from kernel to user space buffer. Here you can find an example of the driver and app, which works so:
1) process opens /dev/device_file that represents the driver
2) it maps kernel coherent memory, allocated via dma_alloc_coherent
3) fills buffer with some data
4) calls ioctl() to start dma transfer
It works fine, but i have such question - can I transfer user space buffer to kernel space (via read function from file_operations structure), prepare it (shift by PAGE_SIZE or something else) and execute dma operation ? I don't understand how to make user space memory available in DMA operation.
You probably want to implement mmap method of struct file_operations. Consider:
static int
sample_drv_mem_mmap(struct file *filep, struct vm_area_struct *vma)
{
/*
* Set your "dev" pointer here (the one you used
* for dma_alloc_coherent() invocation)
*/
struct device *dev;
/*
* Set DMA address here (the one you obtained with
* dma_alloc_coherent() via its third argument)
*/
dma_addr_t dma_addr;
/* Set your DMA buffer size here */
size_t dma_size;
/* Physical page frame number to be derived from "dma_addr" */
unsigned long pfn;
/* Check the buffer size requested by the user */
if (vma->vm_end - vma->vm_start > dma_size)
return -EINVAL;
/*
* For the sake of simplicity, do not let the user specify an offset;
* you may want to take care of that in later versions of your code
*/
if (vma->vm_pgoff != 0)
return -EINVAL;
pfn = PHYS_PFN(dma_to_phys(dev, dma_addr));
return remap_pfn_range(vma, vma->vm_start, pfn,
vma->vm_end - vma->vm_start,
vma->vm_page_prot);
}
/* ... */
static const struct file_operations sample_drv_fops = {
/* ... */
.mmap = sample_drv_mem_mmap,
/* ... */
};
Long story short, the idea is to convert the DMA (bus) address you have to a kernel physical address and then use remap_pfn_range() to do the actual mapping between the kernel and the userland.
In the user application, one should invoke mmap() to request the mapping (instead of the read / write approach) For more information on that, please refer to man 2 mmap on your system.
I'm confused about how the OS actually "opens" a file in C when you do an fopen in the code. To elaborate, suppose I had a 100 binary files (say of size 1 MB) which I open in C
FILE **fptr;
fptr = calloc(100, sizeof(FILE *));
for (ii = 0; ii < 100; ii++)
fptr[ii] = fopen(filename[ii], "rb+");
Assume that filename and ii were already defined appropriately.
Will the OS load 100 MB into memory, or does the above code just tell the program to keep these files ready for access?
The latter, no data is read from the file until needed, i.e. when you call fread() or some other I/O function.
Of course the underlying operating system might decide to speculatively read data when the file is opened, to save time later, but that's outside your control so in effect it doesn't matter. I mean that it doesn't matter because any memory used by such speculative buffering will need to be immediately made available to applications on demand.
That said, it's not as if any practical system will let the fopen() spend the time needed to read 100 MB though, that would be very bad engineering.
Also note that there might be limits on how many files a single process can open in parallel. 100 should be fine for most modern systems, though.
The file is not loaded into memory upon opening it. Instead, parts are loaded in for each read.A call to fopen should not cause reading the file content from media now fread will cause partial read (or complete read for small files) from the media. Partial read usually is equal to cache line size in cache manager.
There are few things we need to look at first
FILE
this is typedefed as below
stdio.h:
typedef struct _IO_FILE FILE;
next _IO_FILE
_IO_FILE looks like below :
libio.h:
struct _IO_FILE {
int _flags; /* High-order word is _IO_MAGIC; rest is flags. */
#define _IO_file_flags _flags
/* The following pointers correspond to the C++ streambuf protocol. */
/* Note: Tk uses the _IO_read_ptr and _IO_read_end fields directly. */
char* _IO_read_ptr; /* Current read pointer */
char* _IO_read_end; /* End of get area. */
char* _IO_read_base; /* Start of putback+get area. */
char* _IO_write_base; /* Start of put area. */
char* _IO_write_ptr; /* Current put pointer. */
char* _IO_write_end; /* End of put area. */
char* _IO_buf_base; /* Start of reserve area. */
char* _IO_buf_end; /* End of reserve area. */
/* The following fields are used to support backing up and undo. */
char *_IO_save_base; /* Pointer to start of non-current get area. */
char *_IO_backup_base; /* Pointer to first valid character of backup area */
char *_IO_save_end; /* Pointer to end of non-current get area. */
struct _IO_marker *_markers;
struct _IO_FILE *_chain;
int _fileno;
#if 0
int _blksize;
#else
int _flags2;
#endif
_IO_off_t _old_offset; /* This used to be _offset but it's too small. */
#define __HAVE_COLUMN /* temporary */
/* 1+column number of pbase(); 0 is unknown. */
unsigned short _cur_column;
signed char _vtable_offset;
char _shortbuf[1];
/* char* _save_gptr; char* _save_egptr; */
_IO_lock_t *_lock;
#ifdef _IO_USE_OLD_IO_FILE
};
and the chain of internal pointers go on, however this file structure by no means keeps the data with itself what this maintains is a snapshot of record and presents to userspace. incase when user requests to read data from file the same FILE * will be used however the data will be retrieved by a syscall (in linux case read call), the offset state however will persisted here itself to go forward , backward. most importantly this provides a very good abstraction to internal system calls.
It's implementation defined. You're not required to read the entire file in memory on fopen(). Most implementations these days use some form of mmap() to actually read the data into memory.
This is a follow-up to my other question here:
How to read vfat attributes of files in Linux using C
--
I saw this struct in linux/msdos_fs.h:
struct msdos_dir_entry {
__u8 name[8],ext[3]; /* name and extension */
__u8 attr; /* attribute bits */
__u8 lcase; /* Case for base and extension */
__u8 ctime_cs; /* Creation time, centiseconds (0-199) */
__le16 ctime; /* Creation time */
__le16 cdate; /* Creation date */
__le16 adate; /* Last access date */
__le16 starthi; /* High 16 bits of cluster in FAT32 */
__le16 time,date,start;/* time, date and first cluster */
__le32 size; /* file size (in bytes) */
};
My question is, would it be possible to populate such a struct inside my user application? My app requirement is that it should be able traverse a vfat filesystem, and get the vfat attributes (msdos_dir_entry) for each directory/file it finds.
Thanks.
Actually you can get almost all of this by combining the information you can get from fstat(), the FAT_IOCTL_GET_ATTRIBUTES and VFAT_IOCTL_READDIR_BOTH ioctls. It's not going to nice to look at though, since for the fromer two you need the file fd and for the latter two you need the fd of the dir the file is located in.
I am trying to implement the ls command in C without using the system function.
Is there a C function to get the directory/file permissions, so that I can display them?
Thanks.
The stat() system call takes a filename string and returns the following structure:
struct stat {
dev_t st_dev; /* ID of device containing file */
ino_t st_ino; /* inode number */
mode_t st_mode; /* protection */
nlink_t st_nlink; /* number of hard links */
uid_t st_uid; /* user ID of owner */
gid_t st_gid; /* group ID of owner */
dev_t st_rdev; /* device ID (if special file) */
off_t st_size; /* total size, in bytes */
blksize_t st_blksize; /* blocksize for filesystem I/O */
blkcnt_t st_blocks; /* number of blocks allocated */
time_t st_atime; /* time of last access */
time_t st_mtime; /* time of last modification */
time_t st_ctime; /* time of last status change */
};
This works equally well on directory entries as well as files and the st_mode is the specific field you're looking for.
The <sys/stat.h> header file should contain that structure and all the definitions along with #defines and/or macros for intelligently decoding the fields (see here for a sample).
If you're interested in how it's really done, you can look at the ls source code in GNU coreutils. But only use that to educate yourself. It's GPL so you can't just grab it for your own purposes and, if this is homework, your educator will almost certainly fail you if you turn in something that looks too similar to this.
Look into stat(). Sounds like you're on a Linux or POSIX system, so that should be the way.
Then look at the st_mode field of the struct stat, it contains the information about protection bits, which are often collectively called a file's "mode" (as reflected by the chmod command that changes the settings).
Going from the binary bits to a textual representation like ls' is ... an interesting challenge.
The stat functions family (stat(), lstat(), fstat()).