I began studying about file systems, especially about FAT* and NTFS.
In FAT file systems clusters may be Data or Directory clusters, and the starting cluster number of the Root directory is always known, as prior to FAT32 it was fixed at formatting time, and as with FAT32 is found in the extended BIOS block in the boot sector.
NTFS in the other hand organizes everything under the Master File Table, with a MFT record for each file and directory in the system.
The Master File Table has its first 27th position marked as reserved and the first index consists of the $MFT record, which describes the MFT itself.
I understand how NTFS keeps track of data via the resident / non-resident data attribute, while FAT uses Directory Entries to find the first cluster of the cluster chain and refer to the File Allocation Table for further processing.
Now my mind find difficult "processing" these things.
Where do I find the Root directory in a NTFS?
How is a directory represented in a MFT record? Both in a resident and non-resident way, and how do I find a sub-directory MFT record via the current MFT record?
What if cluster runs specified in the second half of the MFT record go beyond the 1024 bytes limit? (I understand this means a badly fragmented file)
I have to answer myself, as it may be useful to others.
The root directory in a NTFS is found at the index 5 of the reserved records of the MFT.
A directory MTF record in a NTFS is represented almost as a file MTF record, but instead of the Data Attribute part, Index Root and Index Allocation attributes are stored.
The Index Root contains the indexes of the MFT records in the MFT which represent files and directories stored inside the actual directory.
The Index Allocation is used in case the data runs are bigger then the 1024 bytes limit, to de-reference the data continuation to another MFT record.
Related
I am attempting to create an exFAT file system implementation for an embedded processor. I have a question about the cluster allocation of a file.
Suppose I have created a file and then written some data to the file. Lets say that the file has used 10 clusters and they are continuous on disk. As far as I understand it, exFAT has a way to mark the file as continuous in the file's directory entry.
Lets say I create more files and write data to them. Then I go back to my first file and want to write more data to the file. But now there are no free clusters at the end of the file as other files occupy them. I guess it must be possible to add more space to the file, but how is this recorded in exFAT? The original file was continuous but now it isnt. The FAT table wouldnt have been updated because of that. Does the FAT table now have to be re-written because the file is fragmented? Thanks
I did an experiment using a USB pen and a disk utility. It turns out that if you write more data to the file and it becomes fragmented, then the continuous cluster flag gets cleared in the dir entry. Then the FAT table needs to be created to indicate where the clusters for the file are located.
I understand if a file has 1 byte, it will still take up an entire block on disk (e.g. 4KB). Is the same true for a zero-length file? I am specifically wondering about NTFS but insight on other file systems welcome!
No, in case of NTFS, if file has 1 byte, it doesn't use any block. In general, if file has less than 300 bytes (approximately and in case that file record in MFT has 512 bytes - this value depends on file name length, size of MTF file record, etc.), data are located in MFT (master file table). Only if it doesn't fit in one file record (in MFT), then data are externalized to blocks (usually 4 KB).
I am required to create my own implementation of a filesystem in C. I am planning on creating a system similar to that of the FAT system. We are given one file of size 10MB, which acts as our own "disk." I understand that the FAT table stores cluster numbers, and the Root Directory stores other pertinent information about each file we create (e.g. file name, size, date and time of last modification, start block in FAT, etc.). But I am confused about how the cluster numbers are translated to physical addresses in the data region on the disk.
For example, let's say an entry in the Root Directory says that a file starts in block 100 in the FAT table, and in block 100 of the FAT table is the integer 327, which is where the next cluster of the file is located. How are these addresses translated to physical addresses in the data region of the disk? Where are these physical addresses translated and stored?
Clusters vary in size between different versions of FAT (FAT12, FAT16, and FAT32), but in general the cluster number points to a consecutively numbered cluster of whatever size is present in the format for the existing file system. As I recall (from long ago) FAT12, at least on hard disks, used 2 kibibyte clusters (made up of four 512-byte sectors each), with a maximum cluster number of 2^11 (12 bits starting with zero), so cluster 327 would be 327 * 2048 bytes from the start of the data area of the disk.
The data area includes the FAT, backup FAT, and all directories. My recollection is that each cluster entry in the FAT contains a pointer to the next cluster in the file that occupies that cluster, length of data if it's the last cluster of the file, and some other information needed in reading or writing the file, while the directory entry contains the file name, first cluster, size/date/etc..
A disk is divided into sectors. A hard disk for example has a sector size of 512 bytes. Addressing data on the disk usually uses these sectors and data is read/written in blocks of this size. The FAT filesystem groups a number of sectors into clusters. For example you could have 8 sectors per cluster. This constant is stored along with other information about the filesystem in the first few sectors of the partition. The FAT driver uses this value to compute the sector number from the cluster number. The formula is something like this:
SectorNumber = SectorsPerCluster * ClusterNumber + Constant
The constant is the sector number of the first sector of the data region of the partition. You can find the exact formula in the FAT Specification.
I was wondering about the actual (disk-)size of each MFT record. Since the number of clusters per MFT record is set in the bootsector, i guess each one has the same size.
However, each record header stores an additional value: its Allocated size (at 0x1C). As far as i could observe, this value was always equivalent to the value stored in the bootsector.
Is it possible that these two are different (and when)?
If not, the Allocated size value in each record is kind of a waste, right?
It's not actually that much of a waste. You should try to look at what happens when the number of attributes stored in the file record exceeds 1 KB. (by adding additional file names, streams, etc.) It is not clear (to me at least) for different versions of NTFS if the additional attributes are stored in the data section of the volume or in another File Record.
In previous versions of NTFS the size of a MFT File Record was equal to the size of a cluster (generally 4KB) which was a waste of space since sometimes all the attributes would take less than 1 KB of space. Since NT 5.0 (I may be wrong), after some research, Microsoft decided that all MFT File Records should be 1KB. So, one reason for storing that number may be backwards compatibility. Imagine you found an old hard drive which still used 4KB file records and you want to add some file to that drive or copy some files.
Another use for storing that number there would be that you wouldn't need to read the boot sector every time you get a file record to see what it's size should be. Imagine if you were the algorithm that has to mitigate the transfer between 4KB records to 1KB records because of backwards compatibility. If you didn't know what to expect you would have to read the boot sector to find out what size of a record to expect.
What if you didn't have access to the boot sector or you're trying to recover files from a drive that had it's boot sector wiped or has bad clusters? What would happen if the volume is on multiple extents and you're reading the MFT from one extent and the boot sector is in another extent that you don't have access to?
Usually, filesystems are designed by more than a few people over a long time. If those values would be redundant I should think they would certainly notice.
I'm trying to understand more about the ext3 filesystem and was going through the source code when I had a doubt. Would it be possible to figure out the journal inode number for a given ext3 disk using any utility that is out there?
In case you are still interested this quote from Ext4 wiki (you can find more interesting facts about ext-family there):
The journal inode is typically inode 8. The first 68 bytes of the
journal inode are replicated in the ext4 superblock. The journal
itself is normal (but hidden) file within the filesystem. The file
usually consumes an entire block group, though mke2fs tries to put it
in the middle of the disk.