From Wikipedia
Disk formatting
Disk formatting is the process of preparing a data storage device such as a hard disk drive, solid-state drive, floppy disk or USB flash drive for initial use.
Mount (computing)
Mounting is a process by which the operating system makes files and directories on a storage device (such as hard drive, CD-ROM, or network share) available for user to access via the computer's file system.
I don't understand how these two sentences are different
You format a device. You mount a filesystem.
Mounting a Filesystem
Typically, to make a device ready to be written to by the system, you'd have to do the following:
Physically connect the device to the system
(optional but recommended) Create a partition on your device
Create a filesystem on the device/partition
Mount the filesystem on the device/partition to a directory on your system's filesystem, so that you can access the filesystem of the device/partition from your system.
Formatting a Device
Therefore, every filesystem is backed by some block storage device. The storage medium of a block storage device (e.g. a hard disk) is divided into many sections, called sectors. Each sector has a unique address. Whenever you write to, or read from, the device, it is done by writing/reading whole sectors. Therefore, the sector is a smallest addressable physical unit for a block storage device.
When you format a device, you are adding 'marks' onto the storage medium to mark the location of these sectors.
Source of Confusion
Formatting has nothing to do with the filesystem, or mounting. The confusion often arise from Windows users, because on Windows, the term 'formatting' also includes the creation of the filesystem. On Windows, when you format a disk, you are also creating a filesystem on it (and Windows also automatically mount devices you connect).
To distinguish between Window's version of 'formatting', and actual formatting, people have used the term low-level formatting for actual formatting, and high-level formatting for creating the filesystem.
Formatting a disk or partition usually means that you're actually creating or have already created a volume, and it is the volume that you are formatting. Formatting a volume is to write an empty filesystem with the initialised filesystem structures like the MFT to the volume.
Mount has the notion of putting something on top of the other. But what is considered to be on top and what is considered to be at the bottom varies.
For mount points, the mount directory and the existing directory tree is considered to be on the bottom, and you mount the volume on top of / to the directory. This is despite the mount points actually being logically higher than the volume in the driver stack and the process of interacting with files. You sometimes see people say mounting the filesystem to the mount point, like the Linux df command. This makes sense because the file system is indeed logically between the volume and the mount point.
For mounting filesystems to volumes, the filesystem is on top and the volume is below. You mount the filesystem onto the volume. It is actually logically on top of the volume because on Windows it is consulted by the I/O Manager before the volume and it is up to the file system to interact with the volume stack.
On Windows, the volume is mounted to its mount points (C:\ if the symbolic link is \DosDevices\C:) and then you mount the file system to the volume by formatting the volume. Formatting the volume with a file system causes the file system driver to mount the volume. The file system will then mount itself to the volume every time the volume is detected 'arrives' on the system so long as it has been formatted with that file system because the file system driver detects its filesystem on the volume.
Formatting is usually done ONCE during the lifetime of the hard drive.
The hard drive is written to, wiping away anything that was on it previously, so that it can be accepted as a filesystem.
Mounting usually happens every time the system has been restarted. It allows the operating system to access the hard drive as a filesystem. Mounting a drive does NOT alter the hard drive, although once a filesystem has been mounted it can be modified (unless it was mounted read-only) by typical filesystem operations like creating a directory/folder, creating files, modifying files, etc ....
Mounting tells the operating system which filesystem type (ext3, ext4, HFS+, NTFS, ZFS ...) the hard drive is allowing the OS kernel to map the virtual filesystem functions to the real filesystem functions.
Related
To mount vhds on windows, we can use OpenVirtualDisk then AttachVirtualDisk to attach a virtual disk, then GetVirtualDiskPhysicalPath can be used to obtain a physical path to the vhd, in the form of \\.\PhysicalDrive1.
Next we can use FindFirstVolume and friends to find the volume names the vhd contains. But this does not always work, as it can take some time for the mount manager to create a volume name symlink in the dos devices namespace to point to the physical volume.
How can we wait for this to happen, without doing busy-waits?
I have a file on tmpfs partition, which is updated alot.
I want to copy it to other partition (flash partition) before crash/reboot.
It is not an option to keep this file in the first place on the flash partition,
because this flash has limited read/write life-cycle and I'm trying to avoid excessive read/writes to it.
too many writes will damage the flash, that is why the file is on tmpfs.
regrading reboot - I can modify the reboot process to copy before reboot - is there more neat way?
regrading crash - I don't know any way to do so. any ideas?
I know that that I should not mess with files from kernel space.
Thanks
Only a Kernel panic occurs its possible that in-core data structures are already corrupted and unreliable. Ideally, your kernel is not expected to panic, if the version you are using is a stable and tested release. I would recommend capturing a vmcore using crash tool and working with the vendor on the root cause of the kernel panic.
However, if you are referring to an abrupt system shutdown dew to a power failure, etc which could possibly cause loss of the data / file stored in the memory, you could write a cron-job to sync the file to disk on intervals and tune the kernel on how frequently the dirty page get synced. Having said that, if the file you are writing to is quite important, why design it to be kept in the memory in the first place.
You should be syncing this file back to the disk once in every few seconds or in regular intervals. In this way you will not loose the complete data.
As the numbers of read/writes are heavy on the tmpfs file, it may be worth considering using a SSD for this purpose. Read about how file system transaction logs are configured to be stored on SSD drives.
Write a cron-job for syncing the tmpfs file to the SSD or disk in frequent intervals or when ever there are updates. You may want to consider changing some kernel tunables (such as, vm.dirty_expire_centisecs=0, vm.dirty_background_ratio=0) such that any dirty pages would get synced to the disk immediately. A word of caution, doing this would cause higher CPU% and I/O loads, as pages would synced to the disk frequently, although the data loss would be kept to minimal.
I understand a file-system can choose the size of blocks it uses on the disk.
On the other hand i understand that the disk is divided into LBA's.
The LBA is an address of a sector on the disk.
So whats the connection between the block used by the file system and the disk sectors (lba)?
Is there some kind of translation from a fs block and lba?
Is it different from fs to fs?
where can i read more about this?
thanks
Yes. File system usually sees a a continuous logical space without knowledge of the spindles underneath, thus it doesn't know disk LBA either. The translation work is usually done in a layer called volume, which is to hide the disk detail and present the file system a logically continuous space. For example, in Linux there's LVM (Logical Volume Manager) playing such roles.
The volume exposed to fs might not be disks. It could be constructed upon other volumes, thus sometimes come up with a very large disk.
The volume could also provide the functionality of RAID, which put several disks together that could relieve you from disk failure in some extent at the expense of performance and space efficiency.
Some file systems can manage disks directly and operate on raw disks, thus no layer of volume. As far as I know, NETAPP's WAFL is doing in that way.
I'm developping a "MP3 player"-like USB device. It is seen as a Mass Storage device by the USB host (Windows). I'd like to be able to keep the current song playing while the device is connected.
In an ideal world, the user should be able to delete every mp3 files using his file explorer but the one currently playing, which would be seen as "in use" by Windows.
The filesystem is FAT, I use FatFS for reading files on the device.
Does FAT allow such thing (mark a file as "in use") ? Any smarter idea ?
Like every filesystem that isn't specifically designed for this purpose, FAT isn't a cluster filesystem. As such, there are no provisions for mounting it from more than one host at a time. So it can't be mounted from the USB host's operating system and from the USB device's embedded operating system at the same time. The concept of a file being "in use" at the filesystem level is moot for a non-cluster filesystem.
For examples of cluster filesystems, look at OCFS2 or GFS2. But those require things like network lock managers and it is very unlikely that you can easily use them for an application like a USB device.
I will write some thing in a file/memory just before system shutdown or a service shutdown. In the next restart of system, Is it possible to access same file or same memory on the disk, before filesystem loads? Actual requirement is like this, we have a driver that sits between volume level drivers and filesystem driver...in that part of the driver code, I want to access some memory or file.
Thanks & Regards,
calvin
The logical thing here is to read/write this into the registry if it is not too big. Is there a reason you do not want to use the registry?
If you need to access large data and you are writing a volume or device filter and cannot rely on ZwOpen/Read/Write/Close functions in the kernel an approach would be to create the file in user mode, get its device name and cluster chain and store them in the registry. On the next boot, you can get the device and clusters from registry, and do direct I/O on them.
Since you want to access this before the filesystem loads, my first thought is to allocate and use a block of storage space on the hard drive outside of the filesystem. You can create a hidden mini-partition on the drive and use low-level I/O commands to read and write your data.
This is a common task in the world of embedded systems, and we often implement it by adding some sort of non-volatile memory device into the system (flash, battery-backed DRAM, etc) and reading and writing to that device. Since you likely don't have the same level of control over the available hardware as embedded developers do, the closest analogue I can think of would be to reserve a chunk of space on a physical disk that you can read from without having to mount as a filesystem. A dedicated mini-partition might work the best because if you know the size of it, you can treat it as one big raw-access buffer and can avoid having to hassle with filenames, filesystems, etc.