Can get size of disk sector via the Linux API/ABI? It's about the quantum of I/O disk, normally it's equal 512 bytes, but others values can be too (usually multiple 512 bytes).
Also it should not confuse to size of logical block or to size of sector of a file system.
A block device is reflected as file in a file system of an UNIX (/dev/sda, /dev/sr etc.) It means, can open that file and make some manipulations to its content like with content of the corresponded block device.
So specifically the work to a true block device similar the work to a virtual hard disk (the .vhd format for instance).
But i don't know how to get size of sector in general case.
At moment i've single solution: get the maximal CHS address and size of hard drive, both action via BIOS. But i think, it's bad idea, because portability lost
Related
I have a project where I have to update the data on disk very frequently in case of power loss. When overwriting exactly 512b (1 sector of my drive) in the file with data of equal length does the file-system mark the sectors that have been changed and update them on disk when ready to flush? Or does it write the whole file every time it flushes a change? I am mainly concerned with ext4 but I am curious if it is the same with every file-system.
If the standard is not to track changes but to overwrite the entire file is there a way to change this? Some write options?
In general with Linux files are cached in the page cache, and whether or not a page is dirty is tracked at the page level. On Intel x86 platforms, the page size is 4k, so if you dirty a 4k page, it is the 4k page which gets written back.
If you want to only overwrite a single 512 byte sector, and you have a HDD that has 512 byte sectors, you can open the file with the O_DIRECT flag and if you issue a 512 byte write, on a file offset which is a multiple of 512 bytes, and where the memory buffer from where you source the write is also 512 byte aligned, then you can bypass the page cache, and the write will go directly to the disk (hence O_DIRECT).
Note that a number of modern disks are really using a 4k physical sector, but they are emulating 512 byte sectors for backwards compatibility reasons. These disks are sometimes called 512e sectors (e for emulated). On these drives, if you do a 512 byte sector write, the disk will do a read-modify-write cycle, since the drive internally can only write 4k at a time. This will be visible to you as a performance hit, but from a functional perspective, it will otherwise look the same as a traditional, old-fashioned 512 sectored HDD.
i know the cluster size of a USB flash drive can be changed , can we change the sector size too ??
Sector size isn't a configurable parameter in ATA/SATA/SCSI/etc devices and, from my experience, USB flash drives implement one of these protocols. The sector size is reported by the device itself but, even if you could set it to something other than 512, you would likely run into a latent bug somewhere in a driver or file system package that assumed a sector size of 512.
There are real reasons for using a sector size like 512, for example, addressing of larger sectors can be done more quickly and efficiently (not just in time but in size/space as well). Throughput to these devices is also better with something like 512. Consider that, if you could set sector size to something like 16-bytes, you might have less wasted space with 16-byte sectors compared to having a number of half-full 512-byte sectors, but your throughput to the device would probably be worse. In fact, writing a single 16-byte sector would only be slightly faster than writing a single 512-byte sector. On the other hand, writing 32 16-byte sectors (to write a total of 512 bytes) would likely take longer than writing a single 512 byte sector simply due to the overhead associated with transferring multiple sectors.
I would suggest you buy a larger USB flash drive if you are worried about wasting space with 512-byte sectors.
I've been experimenting with the performance of reading and writing files on Linux, specifically O_DIRECT, and I'm wondering, both at a hard drive level and the posix/Linux API level, is it possible to write only a few bytes to a sector, without destroying the rest of the sector, and without reading it first?
My experience with disk drives is that they expect data to be sent to them in entire sectors. So, basically, there's no way of writing less than an entire sector and if you wish to change the start of a sector without changing the end, you must read the whole sector, modify and write back. That is partly to do with how the disk head interacts with the platter (for physical disks anyway. In the case of flash drives, it's more likely to be with how small a chunk of the flash can be erased in one go).
In a portable way? Probably not.
In Linux and a few other Unix-like systems, you can open the block device for the drive, seek to a position (probably aligned to the sector size) and write some data to it, but I don't know what effect it would have on the remaining portion of that block.
Your best bet is to try it out on a virtual machine and see what happens. (Obviously, you'll have to have permission to write to the block device.)
What is the significance of the file system block size? If my filesystem block size is set at, say 8K, does that mean that all read/write I/O will happen at size 8K? So if my application wants to read say 16 bytes at offset 4097 then a 4K block starting from offset 4096 will be read?
How do writes work in this case? Suppose I want to write say 64 bytes.
You are right. The block size is the unit of work for the file system. Every read and write is done in full multiples of the block size.
The block size is also the smallest size on disk a file can have. If you have a 16 byte Block size,then a file with 16 bytes size occupies a full block on disk.
The book "Practical file system design" states:
Block: The smallest unit writable by a disk or file system. Everything a
file system does is composed of operations done on blocks. A file system
block is always the same size as or larger (in integer multiples) than the
disk block size.
Normally when you have to deal with files in programming you should use Stream abstraction.
I/O operations through code are often reads and writes to streams; reading and writing from and to streams, can be buffered so that chunks of file can be read or written.
Block size on fs refers to mapping disk surface; minor the size of the single block major the number of blocks (and so the elements in the table that keeps information on allocation of files).
So OS's so can map file on disk discretely based on block size and have a smaller "map of files".
As I know this doesn't affect stream abstraction in API's of programming language.
Imagine there is a file of size 5 MB. I open it in write mode in C and then fill it up with junk data of exactly 5 MB. Will the same disk sectors previously used be overwritten or will the OS select new disk sectors for the new data?
It depends on the file system.
Classically, the answer would be 'yes, the same sectors would be overwritten with the new data'.
With journalled file systems, the answer might be different. With flash drive systems, the answer would almost certainly be 'no; new sectors will be written to avoid wearing out the currently written sectors'.
The filesystem could do anything it wishes. But any real file system will write the data back to the same sectors.
Image if it didn't. Every time your wrote to a file the file system would have to find a new free sector, write to that sector, then update the file system meta data for the file to point to the new sector. This would also cause horrible file fragmentation, because writing a single sector in the middle of your contiguous 5MB file would cause it to fragment. So it's much easier to just write back to the same sector.
The only exception I can think of is JFFS2 because it was designed to support wear leveling on flash.
Now the file system will write to the same sector, but the disk hardware could write anywhere it wants. In fact on SSD/flash drives the hardware, to handle wear leveling, is almost guaranteed to write the data to a different sector. But that is transparent to the OS/file system. (It's possible on hard drives as well due to sector sparing)