I need to clear sector 0 for removable media devices (custom USB memory devices) which I have been trying to clear within a WPF/C# application. My first attempt was to use DD, but I ran into problems. During the manufacturing of the devices a MBR is created at sector 0 and the volume (logical?) starts at sector 40. When I issue the following command it clears sector 40 and not sector 0:
dd bs=512 count=1 if=/dev/zero of=\.\E:
I found another version of DD here which includes a wipe utility. I tried this version and I am seeing the same behavior. I am using both HxD and Runtime's DiskExplorer that sector 40 is being cleared and not sector 0. I could use HxD or Runtime's DiskExplorer, but this needs to be scriptable.
Does anyone know of any other methods of clearing (filling) sector 0 within Windows XP SP2?? Any help would be greatly appreciated. Thanks.
Mark
Solution: My solution used WMI to find the physical drive based upon the logical drive letter. First, query the Win32_LogicalDiskToPartition class to find the logical drive I am looking for. This provides the Antecedent field which constains something like '...DeviceID="Disk #X, Partition #Y"'. Next, I query Win32_DiskDriveToDiskPartition class while searching against the Dependent field to find the match for the Antecedent field within the Win32_LogicalDiskToPartition class. Once found, the Antecedent field from Win32_LogicalDiskToPartition will yield the physical drive. I selected atzz since it is the closes to my solution. I wanted to use Eugene's suggestion, but I only had a few hours to implement this so I selected the easier of the two. I will need to revisit this at a later time though.
There are two ways to format a USB drive, from Windows standpoint:
As a floppy disk. In this case entire USB drive contains a single file system, and its boot record is located in sector 0.
As a hard drive. In this case, sector 0 contains MBR with partition table. Actual file system(s) with their individual boot records are located further on the drive.
I think you are observing the second case. Using \.\E: to identify the device, you end up accessing file system's boot record instead of MBR.
Here is how you can access sector 0 of the USB drive.
Load WinObj from here.
In WinObj, under GLOBAL??, find E:. It will be a SymbolicLink pointing to something like \Device\Harddisk2\DP(1)0-0+30.
Under GLOBAL??, find a PhysicalDrive# symlink referring to the same Harddisk# that you found on step 2. Most probably it will have the same numeric suffix as Harddisk#. E.g.: SymbolicLink PhysicalDrive2 refers to \Device\Harddisk2\DR47.
Use the PhysicalDrive# you've found in DD command:
dd bs=512 count=1 if=\\.\PhysicalDrive2 of=mbr.dat
You are trying to clear logical device E: and not physical device. Try doing the following:
call CreateFile() WinAPI function to open "\\.\PhysicalDriveX" where X is the number of the device (see Remarks in description of CreateFile function for information about how to open the physical device properly). Then use WriteFile API function to write 512 bytes at offset 0 of the opened device.
If you get permission denied error when opening the device for writing, you can take our RawDisk product (trial version will work fine for you) which lets one bypass this security measure of Windows.
upd: As for calling CreateFile from C#, see PInvoke.net.
Related
I am reading The UNIX Time-Sharing System by D. M. Ritchie and K. Thompson, where they briefly introduce the UNIX OS. In the file system section, when they talk about the "mount", they say the following 2 paragraphs. And I have a few questions about the bold and itatic content in the paragraphs.
Paragraph 1: When an I/O request is made to a file whose i-node
indicates that it is special, the last 12 device address words are
immaterial, and the first specifies an internal device name, which is
interpreted as a pair of numbers representing, respectively, a
device type and subdevice number. The device type indicates which system routine will deal with I/O on that device; the
subdevice number selects, for example, a disk drive attached to a
particular controller or one of several similar terminal interfaces.
Paragraph 2: In this environment, the implementation of the mount
system call (Section 3.4) is quite straightforward. mount maintains a
system table whose argument is the i-number and device name of the ordinary file specified during the mount, and whose
corresponding value is the device name of the indicated special file. This table is searched for each i-number/device pair that turns
up while a path name is being scanned during an open or create; if a
match is found, the i-number is replaced by the i-number of the
root directory and the device name is replaced by the table value.
From the first paragraph, I know that device name is something existing in a special file's i-node. However, why in the second paragraph it says the ordinary file also has it?
What is the system table the mount tries to maintain? In para. 2 is it indicating that the system table is part of the internal file system, and the mount process makes such a table that the entries of it are special files that point to files in the mounted external device?
Let's start with 2 observations:
mount /dev/sda2 /mnt/sda2 takes a special file /dev/sda2, and a non-special file (ordinary file) /mnt/sda2.
file name lookup has to know how to cross into another mounted filesystems.
Let's assume that /dev/sda1 is device 100, and mounted on /. Let's assume that /dev/sda2 is device 200, and mounted on /mnt/sda2. What happens when you look up /mnt/sda2/x? That file is stored as /x on /dev/sda2. Here's what happens:
Assume that the inode number of the root inode is 1 on every filesystem.
The OS looks up mnt in inode 1 of device 100, and finds e.g. inode 5.
The OS checks it's global system table of mounts to see if (device 100, inode 5) maps to something - it doesn't.
The OS looks up sda2 in inode 5 of device 100, and finds e.g. inode 17.
The OS checks it's global system table of mounts to see if (device 100, inode 17) maps to something.
Because /dev/sda2 is mounted onto /mnt/sda2, the table returns /dev/sda2, which is device 200 - we're crossing into another mounted filesystem.
The OS lookups up x in inode 1 of device 200, and finds e.g. inode 11.
Lookup returns (device 200, inode 5).
So to answer your questions:
The ordinary file is the mount point. "Ordinary" means "not a special file". Note that it is possible to mount a regular file, not just a directory. Docker uses this capability.
The system table's entries are mount points which point to mounted devices.
When a file is saved into a drive, its contents are written & then indexed. I want to get the indexes and to access the raw contents of the files.
Any idea on the method how to do it, especially for ex4 & btrfs?
UPDATE: I want to get the addresses of the extents of a file. The information about the addresses must be stored somewhere onto the disk. I want to retrieve this info, in order to map the physical location of the file contents. Any methods in order to achieve that?
UPDATE: Hello, all! Thanks for your replies. What I want is a function/command which returns me a list of extent addresses. debugfs seems the function/command with the most-relevant functionality.
It depends of the filesystem you are using. If you are running Linux you can use debufs to seek the file in the filesystem.
I have to say that all FSs are mounted through a VFS, a virtual filesystem that is like a simplified interface with the standard operations (open, close, read...). What is the meaning of that? No filesystem nor its contents(files, dirs) are opened directly from disk, when you open something, you move it to the main memory(your RAM) you do your operations and when you close something it returns to the disk drive.
Now, the question is: Can I get the absolute address in a FS? Yes, if you open your whole filesystem like open ("/dev/sdaX", 0_RDONLY); so you get the address relative to your filesystem using lseek in C for example.
And then... Can I get the same in the whole drive? No, that is because you cannot open the whole drive as a file descriptor. Remember /dev/sdaXin UNIX? Partitions and its can be opened like files because they have a virtual interface running on them.
Your last answer: Can I read really raw contents? All files are read as they appear on disk, the only thing that changes is the descriptor used by the OS and some data about how is indexed, all this as a "file header".
I hope all your questions are answered.
The current solution/workaround is to call these functions with popen:
filefrag -e /path/to/file
hdparm --fibmap /path/to/filename
Then one should simply parse the stringoutputs of these programs. It is not a real solution (i.e.: outputs at C/C++ level), but I'll accept it for now.
Sources:
https://unix.stackexchange.com/questions/106802/what-command-do-i-use-to-see-the-start-and-end-block-of-a-file-in-the-file-syste
https://serverfault.com/questions/29886/how-do-i-list-a-files-data-blocks-on-linux
I know it's possible on linux. I tried using open("E:", 0); and open("E:\\", 0); but it returns as -1. I'd like to read the DVD as a large file rather than use it as a filesystem.
There's plenty of information on the CreateFile documentation on MSDN. It's fairly simple, though: "When opening a volume or removable media drive (for example, a floppy disk drive or flash memory thumb drive), the lpFileName string should be the following form: \\.\X:. Do not use a trailing backslash \, which indicates the root directory of a drive.
I have three processes designed to run constantly in both Linux and Mac OS X environments. One process (the Downloader) downloads and stores a local copy of a large XML file every 30 seconds. Two other processes (the Workers) use the stored XML file for input. Each Worker starts and runs at random times. Since the XML file is big, it takes a long time to download. The Workers also take a long time to read and parse it.
What is the safest way to setup the processes so the Downloader doesn't clobber the stored file while the Workers are trying to read it?
For Linux and Mac OS X machines that use inode based file systems, use temporary files to store the data while its being downloaded (and is an incomplete state). Once the download is complete, move the temporary file into its final location with an atomic action.
For a little more detail, there are two main things to watch out for when one process (e.g. Downloader) writes a file that's actively read by other processes (e.g. Workers):
Make sure the Workers don't try to read the file before the Downloader has finished writing it.
Make sure the Downloader doesn't alter the file while the Workers are reading it.
Using temporary files accommodates both of these points.
For a more specific example, when the Downloader is actively pulling the XML file, have it write to a temporary location (e.g. 'data-storage.tmp') on the same device/disk* where the final file will be stored. Once the file is completely downloaded and written, have the Downloader move it to its final location (e.g. 'data-storage.xml') via an atomic (aka linearizable) rename command like bash's mv.
* Note that the reason the temporary file needs to be on the same device as the final file location is to ensure the inode number stays the same and the rename can be done atomically.
This methodology ensures that while the file is being downloaded/written the Workers won't see it since it's in the .tmp location. Because of the way renaming works with inodes, it also make sure that any Worker that opened the file continues to see the old content even if a new version of the data-storage file is put in place.
Downloader will point 'data-storage.xml' to a new inode number when it does the rename, but the Worker will continue to access 'data-storage.xml' from the previous inode number thereby continuing to work with the file in that state. At the same time, any Worker that opens a new copy 'data-storage.xml' after Downloader has done the rename will see contents from the new inode number since it's now what is referenced directly in the file system. So, two Workers can be reading from the same filename (data-storage.xml) but each will see a different (and complete) version of the contents of the file based on which inode the filename was pointed to when the file was first opened.
To see this in action, I created a simple set of example scripts that demonstrate this functionality on github. They can also be used to test/verify that using a temporary file solution works in your environment.
An important note is that it's the file system on the particular device that matters. If you are using a Linux or Mac machine but working with a FAT file system (for example, a usb thumb drive), this method won't work.
hope you can help me:
I'm trying to determine whether the device is removable or not, all i have is device name (/dev/sdc). Actually, I need to determine when the file on removable media or on local disk by full path of this file.
I've tryed to search in the
current->fs->pwd
and all I could find is a set of flags here:
*current->fs->pwd.mnt->mnt_sb->s_bdev->bd_disk->flags*
where GENHD_FL_REMOVABLE set for removable devices
But i always get the same flags set (as i understand, s_bdev always points to the same device (/dev/sda)).
So now i get the device name (/dev/sdc) that contains my file by parsing mtab, but still can't find out, removable it or not.
Is there possible way to get block_device structure by device name?
(for example, "file" structure may be obtained by calling
fd = open("name")
fl = fged(fd)
where fl points to "file" structure)
You can iterate over block devices using class_dev_iter_init and class_dev_iter_next. See the code in block/genhd.c blk_lookup_devt for usage.
Once you have the device, you can use dev_to_disk to get a struct gendisk *, in which you can check the removable flag.
Read /sys/block/dev-name/removable as it should contain 1 if the device is removable or 0 if not. (dev-name = the device's name: sda, hda, fd0, ...)