I'm trying to understand more about the ext3 filesystem and was going through the source code when I had a doubt. Would it be possible to figure out the journal inode number for a given ext3 disk using any utility that is out there?
In case you are still interested this quote from Ext4 wiki (you can find more interesting facts about ext-family there):
The journal inode is typically inode 8. The first 68 bytes of the
journal inode are replicated in the ext4 superblock. The journal
itself is normal (but hidden) file within the filesystem. The file
usually consumes an entire block group, though mke2fs tries to put it
in the middle of the disk.
Related
I am attempting to create an exFAT file system implementation for an embedded processor. I have a question about the cluster allocation of a file.
Suppose I have created a file and then written some data to the file. Lets say that the file has used 10 clusters and they are continuous on disk. As far as I understand it, exFAT has a way to mark the file as continuous in the file's directory entry.
Lets say I create more files and write data to them. Then I go back to my first file and want to write more data to the file. But now there are no free clusters at the end of the file as other files occupy them. I guess it must be possible to add more space to the file, but how is this recorded in exFAT? The original file was continuous but now it isnt. The FAT table wouldnt have been updated because of that. Does the FAT table now have to be re-written because the file is fragmented? Thanks
I did an experiment using a USB pen and a disk utility. It turns out that if you write more data to the file and it becomes fragmented, then the continuous cluster flag gets cleared in the dir entry. Then the FAT table needs to be created to indicate where the clusters for the file are located.
I began studying about file systems, especially about FAT* and NTFS.
In FAT file systems clusters may be Data or Directory clusters, and the starting cluster number of the Root directory is always known, as prior to FAT32 it was fixed at formatting time, and as with FAT32 is found in the extended BIOS block in the boot sector.
NTFS in the other hand organizes everything under the Master File Table, with a MFT record for each file and directory in the system.
The Master File Table has its first 27th position marked as reserved and the first index consists of the $MFT record, which describes the MFT itself.
I understand how NTFS keeps track of data via the resident / non-resident data attribute, while FAT uses Directory Entries to find the first cluster of the cluster chain and refer to the File Allocation Table for further processing.
Now my mind find difficult "processing" these things.
Where do I find the Root directory in a NTFS?
How is a directory represented in a MFT record? Both in a resident and non-resident way, and how do I find a sub-directory MFT record via the current MFT record?
What if cluster runs specified in the second half of the MFT record go beyond the 1024 bytes limit? (I understand this means a badly fragmented file)
I have to answer myself, as it may be useful to others.
The root directory in a NTFS is found at the index 5 of the reserved records of the MFT.
A directory MTF record in a NTFS is represented almost as a file MTF record, but instead of the Data Attribute part, Index Root and Index Allocation attributes are stored.
The Index Root contains the indexes of the MFT records in the MFT which represent files and directories stored inside the actual directory.
The Index Allocation is used in case the data runs are bigger then the 1024 bytes limit, to de-reference the data continuation to another MFT record.
I was wondering about the actual (disk-)size of each MFT record. Since the number of clusters per MFT record is set in the bootsector, i guess each one has the same size.
However, each record header stores an additional value: its Allocated size (at 0x1C). As far as i could observe, this value was always equivalent to the value stored in the bootsector.
Is it possible that these two are different (and when)?
If not, the Allocated size value in each record is kind of a waste, right?
It's not actually that much of a waste. You should try to look at what happens when the number of attributes stored in the file record exceeds 1 KB. (by adding additional file names, streams, etc.) It is not clear (to me at least) for different versions of NTFS if the additional attributes are stored in the data section of the volume or in another File Record.
In previous versions of NTFS the size of a MFT File Record was equal to the size of a cluster (generally 4KB) which was a waste of space since sometimes all the attributes would take less than 1 KB of space. Since NT 5.0 (I may be wrong), after some research, Microsoft decided that all MFT File Records should be 1KB. So, one reason for storing that number may be backwards compatibility. Imagine you found an old hard drive which still used 4KB file records and you want to add some file to that drive or copy some files.
Another use for storing that number there would be that you wouldn't need to read the boot sector every time you get a file record to see what it's size should be. Imagine if you were the algorithm that has to mitigate the transfer between 4KB records to 1KB records because of backwards compatibility. If you didn't know what to expect you would have to read the boot sector to find out what size of a record to expect.
What if you didn't have access to the boot sector or you're trying to recover files from a drive that had it's boot sector wiped or has bad clusters? What would happen if the volume is on multiple extents and you're reading the MFT from one extent and the boot sector is in another extent that you don't have access to?
Usually, filesystems are designed by more than a few people over a long time. If those values would be redundant I should think they would certainly notice.
Imagine there is a file of size 5 MB. I open it in write mode in C and then fill it up with junk data of exactly 5 MB. Will the same disk sectors previously used be overwritten or will the OS select new disk sectors for the new data?
It depends on the file system.
Classically, the answer would be 'yes, the same sectors would be overwritten with the new data'.
With journalled file systems, the answer might be different. With flash drive systems, the answer would almost certainly be 'no; new sectors will be written to avoid wearing out the currently written sectors'.
The filesystem could do anything it wishes. But any real file system will write the data back to the same sectors.
Image if it didn't. Every time your wrote to a file the file system would have to find a new free sector, write to that sector, then update the file system meta data for the file to point to the new sector. This would also cause horrible file fragmentation, because writing a single sector in the middle of your contiguous 5MB file would cause it to fragment. So it's much easier to just write back to the same sector.
The only exception I can think of is JFFS2 because it was designed to support wear leveling on flash.
Now the file system will write to the same sector, but the disk hardware could write anywhere it wants. In fact on SSD/flash drives the hardware, to handle wear leveling, is almost guaranteed to write the data to a different sector. But that is transparent to the OS/file system. (It's possible on hard drives as well due to sector sparing)
Secure File Deleting in C
I need to securely delete a file in C, here is what I do:
use fopen to get a handle of the file
calculate the size using lseek/ftell
get random seed depending on current time/or file size
write (size) bytes to the file from a loop with 256 bytes written each iteration
fflush/fclose the file handle
reopen the file and re-do steps 3-6 for 10~15 times
rename the file then delete it
Is that how it's done? Because I read the name "Gutmann 25 passes" in Eraser, so I guess 25 is the number of times the file is overwritten and 'Gutmann' is the Randomization Algorithm?
You can't do this securely without the cooperation of the operating system - and often not even then.
When you open a file and write to it there is no guarantee that the OS is going to put the new file on the same bit of spinning rust as the old one. Even if it does you don't know if the new write will use the same chain of clusters as it did before.
Even then you aren't sure that the drive hasn't mapped out the disk block because of some fault - leaving your plans for world domination on a block that is marked bad but is still readable.
ps - the 25x overwrite is no longer necessary, it was needed on old low density MFM drives with poor head tracking. On modern GMR drives overwriting once is plenty.
Yes, In fact it is overwriting n different patterns on a file
It does so by writing a series of 35 patterns over the
region to be erased.
The selection of patterns assumes that the user doesn't know the
encoding mechanism used by the drive, and so includes patterns
designed specifically for three different types of drives. A user who
knows which type of encoding the drive uses can choose only those
patterns intended for their drive. A drive with a different encoding
mechanism would need different patterns.
More information is here.
#Martin Beckett is correct; there is so such thing as "secure deletion" unless you know everything about what the hardware is doing all the way down to the drive. (And even then, I would not make any bets on what a sufficiently well-funded attacker could recover given access to the physical media.)
But assuming the OS and disk will re-use the same blocks, your scheme does not work for a more basic reason: fflush does not generally write anything to the disk.
On most multi-tasking operating systems (including Windows, Linux, and OS X), fflush merely forces data from the user-space buffer into the kernel. The kernel will then do its own buffering, only writing to disk when it feels like it.
On Linux, for example, you need to call fsync(fileno(handle)). (Or just use file descriptors in the first place.) OS X is similar. Windows has FlushFileBuffers.
Bottom line: The loop you describe is very likely merely to overwrite a kernel buffer 10-15 times instead of the on-disk file. There is no portable way in C or C++ to force data to disk. For that, you need to use a platform-dependent interface.
MFT(master File Table) similar as FAT (File Allocation table),
MFT keeps records: files offsets on disk, file name, date/time, id, file size, and even file data if file data fits inside record's empty space which is about 512 bytes,1 record size is 1KB.
Note: New HDD data set to 0x00.(just let you know)
Let's say you want overwrite file1.txt OS MFT finds this file offset inside record.
you begin overwrite file1.txt with binary (00000000) in binary mode.
You will overwrite file data on disk 100% this is why MFT have file offset on disk.
after you will rename it and delete.
NOTE: MFT will mark file as deleted, but you still can get some data about this file i.e. date/time : created, modified, accessed. file offset , attributes, flags.
1- create folder in c:\ and move file and in same time rename in to folder( use rename function ) rename file to 0000000000 or any another without extention
2- overwrite file with 0x00 and check if file was overwrited
3- change date/time
4- make without attributes
5- leave file size untouched OS faster reuse empty space.
6- delete file
7- repeat all files (1-6)
8- delete folder
or
(1, 2, 6, 7, 8)
9- find files in MFT remove records of these files.
The Gutmann method worked fine for older disk technology encoding schemes, and the 35 pass wiping scheme of the Gutmann method is no longer requuired which even Gutmann acknowledges. See: Gutmann method at: https://en.wikipedia.org/wiki/Gutmann_method in the Criticism section where Gutmann discusses the differences.
It is usually sufficient to make at most a few random passes to securely delete a file (with possibly an extra zeroing pass).
The secure-delete package from thc.org contains the sfill command to securely wipe disk and inode space on a hard drive.