Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
Wikipedia says about file systems:
a filesystem is a type of data store which can be
used to store, retrieve and update a set of files. The term could
refer to the abstract data structures used to define files, or to the
actual software or firmware components that implement the abstract
ideas.
Is there a more formal definition? Is there a terminology to address the different parts?
Is a filesystem
the actual physical bit structure on the disk (or "storage device")?
the kernel code that operates on it? (or generally software or firmware components)
the kernel API that makes it possible for user programs to use it?
the mental model? (ie. abstract ideas)
the specification? (is it the "abstract data structures"?)
Do then some of these parts that make it up have distinct terms for them?
No, there is not a more formal definition of a file system, as the above definition describes fully a "system of managing files". However, there are terms for your various bullet points.
The actual physical bit structure on the disk is named a "block", a "node", or a "physical format" depending on the context. "Blocks" identifies the minimum unit of disk addressing (the actual addressing format has a different name), "Node" identifies the expected interpretation of the bits if the "block" contains file navigation data, and "physical format" identifies the expected bits in relationship to the entire file system design (this term is seldom used).
The kernel code that operates on a file system is typically called a "driver". It reads the blocks, and interprets the bits within according to the overall filesystem structure.
The kernel API that makes it possible for user programs to use a file system is typically the "file system interface", which might or might not expose the details of the filesystem implementation (depending on operating system, a possible filesystem abstraction layer, etc).
The mental model of a file system is the file system's "model" or "design"
The specification is the written documentation which describes the "model" or "design". It often contains details which help clarify if an implementation is conforming to the design. Failure to conform to the design is considered an incorrect implementation of the specification. Specifications may include required data structures, required results from particular api calls, or any other requirements.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have developed a basic kernel in assembly/c that runs a basic terminal. I have set it up to run off of an iso with grub.
I would like to continue this OS, but without a file system, I feel as if there's really nothing else I could do. After much time on the internet, I have come up with really nothing I can do to implement this.
People have said implement FAT or make a VFS, but nothing any further, nor tutorials, nor any references to anywhere.
Could someone explain how a file system works, where I can get started/where I can connect a pre-made system, and how to use it?
Also, I do not have access to standard libraries when compiling my os. I use gcc, nasm, ld, and grub-mkrescue(for the disk image). I use qemu for emulation.
EDIT to make less OT
Can someone describe, in detail how a file system works, so when I look at the sources of file systems that have been implemented, like FAT, I can understand how to apply it to my own operating system?
EDIT - Simpler
Even easier. How could I directly access the hard drive? My kernel runs completely in protected mode, so could I switch out and write directly to the hard drive. A file system could be implemented with a file looking like this:
name special char text special char
ie:
hello world.script 0x00 println "Hello, world!!" 0x00
Where you wouldn't need special segmentation, you would just look until you find the file name and the special character (something not in a string like '\0') and then read until you find the second non-string character.
Would there be a way to access the hard drive by switching in and out of protected mode or write a hard disk driver in order to implement this?
First, read wikipage on file systems to have some broad view.
The relevant resource about operating system development is OSdev (but perhaps your question is off-topic here). Kernelnewbies could also help (explaining how Linux is doing). OSdev have wikipages explaining FAT & Ext2 in details.
You could design an OS without any files (but some other persistence machinery). See this answer. You could have persistent processes (read also about application checkpointing, garbage collection, continuations, hibernation).
But you should read some good books about Operating Systems (e.g. by Tanenbaum, or the freely downloadable Operating Systems: Three Easy Pieces book). Be fluent with some existing free software OS, e.g. Linux (& POSIX), so read Advanced Linux Programming (at least to understand many concepts and get a good terminology).
IMHO, the FAT is such an ugly and inefficient file system that it is not worth looking into (except for legacy and compatibility reasons). Ext4 (see here) should be better & the wikipage on Ext2 has a nice picture.
You could adapt some library providing a file system (e.g. libext2) to your kernel.
You could perhaps adapt sqlite to work on a raw disk partition.
You might have a notion of file which is not like MSDOS (or Windows) or POSIX or <stdio.h> files. For example, it might be a sequence of fixed size records (e.g. of 1Kbyte), not a stream of bytes.
You could organize your OS as a microkernel and have file systems given by application code. Look into VSTa and HURD.
You need of course a disk driver, which fetches/writes blocks (of 4Kbytes) from your drive (disk I/O is always by blocks or disk sectors. Old small disks had 512 bytes blocks. New large disks have 4Kbytes ones, see advanced format). It should be interrupt driven and uses DMA. You need a task scheduler. AFAIU, you won't use the BIOS for this (perhaps the UEFI); you need to understand how common hardware (SATA & AHCI) works.
You should publish (today!) your toy OS as free software (e.g. under GPLv3+ on github) to get feedbacks and contributions.
You might copy (if licenses are compatible) existing code from other free software operating systems, and you certainly will study their source code to understand things.
So code some task scheduler, a page fault handler, a virtual memory, then add interrupt driven disk IO, and some file system code above that. Then you'll beginning to understand that an OS cannot be a small toy.... You might consider a microkernel or exokernel approach.
It would be simplest to use an existing open-source filesystem if the licence terms suit your needs. ELM FatFs is one such library, with no usage restrictions whatsoever. You only need to provide the device control interface layer using the provided stubs and examples.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Regarding the execution of the c program, i am aware that compiler converts the code into machine language and then it is executed.But i would like to know which services are provided by the operating system to accomplish that. say iam using fopen function, please explain me how the operating system handles it, ie. reading the file from hard disk to loading into memory...etc . for all those operations,which system calls are internally called?how the explicit functions like fopen,printf are converted into system calls?
If it is possible to view the internal system calls in context to c programming,please let me know the path to be followed to see them?
Languages typically have their own APIs as part of their run-time support (e.g. fopen() in C's standard library). These are part of the language and not strictly part of the OS itself.
The language's run-time uses the OS's lower level APIs. For example, fopen() might use the kernel API's open() function (Linux); but then it might be a createfile() function in a DLL and not something in the kernel at all (Windows). In some cases, it's nothing like that and more like a message sent to a different process (common for micro-kernels).
Regardless of where it ends up (and how), it probably finds its way to some sort of "virtual file system" layer, and depending on whether it's in the VFS's caches it may or may not get forwarded from there to code responsible for handling a file system, which may or may not forward it to some sort of storage device driver (e.g. a USB flash device driver), which in turn might forward it to another device driver (e.g. a USB controller driver).
Mostly, it can be very different for different OSs, so there is no single answer that's correct for all of them.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 7 years ago.
Improve this question
We have an asp.net website project that we are distributing to subsidiary companies.
We are also referencing a ms-RL's licensed module( dll , which we didn't built , it's on the web.) .
The MS-RL information states :
Reciprocal Grants- For any file you distribute that contains code from
the software (in source code or binary format), you must provide
recipients the source code to that file along with a copy of this
license, which license will govern that file. You may license other
files that are entirely your own work and do not contain code from the
software under any terms you choose.
Question
I find it not crystal clear :
If we have MyPage.aspx that reference JSON_MS_RL.DLL :
According to Ms-RL - Which source code we would have to supply ?
Mypage.aspx ?
JSON_MS_RL.DLL ? ( it's binary , but I assume the source code is needed)
This indeed seems to be a quite obscure passage. The problem is that the word ‘file’ is not defined, while its meaning is not obvious at all. As far as I understand you, the only ‘file’ where the library and the your own code combine together is a distribution / installation package. Does that count as single ‘file’ under Ms-RL? If does, does it counts if I would archive Ms-RL-covered software along with something else into mere *.tar.gz. If that does, it goes beyond all reasons.
I am not a lawyer to interpret it, so the only thing I can do is to refer to some explanation, that can be found over the Web.
The Code Project (that is Microsoft-oriented network for programmers) summarizes the terms of Ms-RL in that way (enumeration mine):
A Microsoft open license and a free software license. Allows for distribution of derived code so long as the modified source files are included and retain the Ms-RL.
Provides copyright protection: True
Can be used in commercial applications: True
Bug fixes / extensions must be released to the public domain: False
Provides an explicit patent license: True
Can be used in proprietary (closed source) applications: True
Is a viral licence: False
You are interested in ⑤, which is clear, and ⑥, which might require further explanation: ‘viral’ is a derogatory term for such a license that allows derivative works but forces to release them under terms of the same license.
So, according to The Code Project, you have to provide sources of Ms-RL-covered library only (either modified or not), not of your whole project. Trust that opinion on your own risk, of course.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
How can it be that a file on an ext3 partion in Linux, which, for example, has different metadata, can get transferred to a Windows NTFS and we still can open and read it correctly?
Don't you have to convert it somehow to make it compatible?
As far as I understand the metadata of the two FS are different, but what happens to these different metadata?
A file system is actually an abstract user interface to access the data behind it. It works in the same way that you can access data from a DB through a web-page.
You acess this interface with file utilities which create, list, copy, move and delete files, and alter metadata. You'll need then some NTFS utils, ext3 utils and so on (it's not a given that they will be present).
There are several aspects that the program doing the transfer (for example, nautilus) has to deal with:
-how to deal with long names and non standard characters like blank spaces, non ASCII (normally copying fails here, so better avoid this)
-endianess (the order of storing bytes). It's not the same reading 0A0B0C0D from left to right than from right to left. Both methods are at use, but the problem is old and therefore tools can deal with it, normally.
-things like Linux permissions get compromised when copying files through file systems (when transfering the file, not just accessing them through a file server like Samba). The recipient can change them to whatever he wants, being root and all. File systems like FAT don't support security at all, so as soon as you copy the file to it the security information is simply lost. Linux OSs can apply a standard set of permissions (for example, with umask, not letting any file being executable).
How a file gets copied:
Open old file to read.
Open new file to write.
Read/write bytes between files.
The file systems involved don't matter.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I'm not sure what the "general" name of something like this might be. I'm looking for a library that gives me a file format to store different types of binary data in an expanding single file.
open source, non-GPL (LGPL ok)
C interface
the file format is a single file
multiple files within using a POSIX-like file API (or multiple "blobs" within using some other API)
file/structure editing is done in-place
reliable first, performant second
Examples include:
the virtual drives of a virtual machine
whefs
HDF
CDF
NetCDF
Problems with the above:
whefs doesn't appear to be very mature, but best describes what I'm after
HDF, CDF, NetCDF are usable (also very reliable and fast), but they're rather complicated and I'm not entirely convinced of their support for opaque binary "blobs"
Edit:
Forgot to mention, one other relevant question:
Simple Virtual Filesystem in C/C++
Another similar question:
Is there an open-source alternative to Windows compound files?
Edit:
Added condition of in-place editing.
Edit:
whefs superseded by: whio_epfs
This appears to do what I was looking for: libgsf
Still need to test its reliability/performance and how cross-platform the binary format is.
It sounds like you're talking about the Linux loopback device, which lets you treat a file on a filesystem as a first-class block device (and then proceed to mkfs, mount, etc.)
(What sort of platform are you targetting? A fully-featured Unixlike? Something in the embedded space with a small footprint?)
The WxWindows library supports ZIP files (see http://docs.wxwidgets.org/stable/wx_wxarc.html#wxarc).
This has also the advantage that you can look at the contents using a ZIP manager (e.g. WINZIP).
A commercial alternative is ChillKat (http://www.chilkatsoft.com/)
If security is a concern, encrypt the file contents and mangle the file names in the ZIP archive.
Eet library from the Enlightenment project maybe?
http://en.wikipedia.org/wiki/Enlightenment_Foundation_Libraries#EET
http://docs.enlightenment.org/api/eet/html/
What about BerkeleyDB? It's not exactly a filesystem but it's quite transparent to store 'binary data' in a file. License seems to be quite permissive as well.