I'm using Sqlite on an embedded device (similar to Raspberry pi3) with a emmc. Emmc handle the wear leveling at hadrware level.
I'm using sqlite in WAL mode.
I have to set the correct database page size in order to reduce writes on the emmc memory.
What kind of page size value should i use in order to reduce writes on memory (small or large)?
Related
I'm in charge of choosing a file system for an embedded Linux device.
The device is a Freescale iMX6 running with a eMMC NAND flash memory and a kernel v3.10.17.
I plan to partition the Flash as decribed below:
Partition #1: kernel - not mounted
Partition #2: rootfs - mounted at "/" in read-only mode
Partition #3: userdata - mounted at "/home" in read-write mode
"/var" and "/tmp" directories will be mounted as tmpfs.
In some previous embedded linux projects, I used to use UBIFS on NAND flashes that were not eMMC NAND flash.
Since eMMC NAND flashes include a wear leveling feature, UBIFS should not be used with them as UBIFS' wear leveling feature may interfere with the one used by the eMMC NAND flashes.
I was planning to use ext2 or ext3 for the Partition #2 (rootfs) and ext3 for the Partition #3. I was wondering if ext3 is robust enough so my data won't get corrupted easily after a power failure of a hardreset reboot.
Does anyone have a strong backgroung with all of this and could help me to figure out what file system would be the best ?
Thanks.
I use ext4 file-system on an eMMC device that contains user data in read/write mode on an embedded-linux system.
The system shuts down by hard-reset several times a day for months now. have not witnessed problems with data consistency yet.
cramfs and squashfs are popular for read-only embedded filesystems, because they are highly compressed in storage.
For read-write filesystems, the "normal" ones you might find on a standard Linux desktop install work well (ext3, ext4, etc.). Read about them and pick one that has a balance of overhead and error-correction, depending on what you need for your device.
For the most part the popularity of these filesystems is independent of the hardware you're using as storage -- drivers are used to actually write to the hardware; the filesystem is an abstraction layer above this.
Your comment about ubifs being inappropriate since the driver already does wear-levelling sounds correct to me. UBIFS is weird in that way. Other filesystems are pretty storage-agnostic.
I understand a file-system can choose the size of blocks it uses on the disk.
On the other hand i understand that the disk is divided into LBA's.
The LBA is an address of a sector on the disk.
So whats the connection between the block used by the file system and the disk sectors (lba)?
Is there some kind of translation from a fs block and lba?
Is it different from fs to fs?
where can i read more about this?
thanks
Yes. File system usually sees a a continuous logical space without knowledge of the spindles underneath, thus it doesn't know disk LBA either. The translation work is usually done in a layer called volume, which is to hide the disk detail and present the file system a logically continuous space. For example, in Linux there's LVM (Logical Volume Manager) playing such roles.
The volume exposed to fs might not be disks. It could be constructed upon other volumes, thus sometimes come up with a very large disk.
The volume could also provide the functionality of RAID, which put several disks together that could relieve you from disk failure in some extent at the expense of performance and space efficiency.
Some file systems can manage disks directly and operate on raw disks, thus no layer of volume. As far as I know, NETAPP's WAFL is doing in that way.
I'm looking for realization of embbeded file system to store pieces of data in nand flash memory addressable by name. Target processor is ColdFire v2 running uC/OS-II, that's why huge FlashFX from datalight.com doesn't fit.
Such realization must be robust to occasional power off and erroneous of nand storage.
Thank you
If you haven't looked at uC/FS with journaling enabled, then I would start there.
http://micrium.com/page/products/rtos/fs
journaling file systems are supposed to maintain the integrity of the FS during system failures.
http://en.wikipedia.org/wiki/Journaling_file_system
I have a CE 6.0 project on a PXA310 where I need to be able to download OS updates (nk.bin) via Wi-Fi and safely flash the new OS to my device. I'm open to other suggestions about how to do this, but I'm considering saving the nk.bin to my file system in NAND flash, then restarting and have the bootloader locate the file in the file system and flash it to the BINFS partition. Is this possible, and if so, can you give me an outline of what I'd need to do?
One caveat is that this needs to be very robust since the devices are deployed in the field and are not field serviceable. I need to be sure that if the OS flash fails (due to power failure, etc.) that upon reboot the bootloader can try again. That is why I'd like to store the downloaded image in persistent flash and avoid having to re-download the image.
Technically just about anything is possible. For this strategy what you would need is code for your bootloader to mount the NAND flash as a drive and have a FAT driver so that it can traverse that file system and find the image. That is a lot of work if you don't already have it.
THe other option is to just store it in flash outside of the file system in a known address location. That's a lot easier from the bootloader perspective as all you have to do is map to the address and copy. Of course it makes the writes more challenging because then you're doing it from the OS and you have to disable any other flash accesses completely while you do your write to prevent corruption by two threads sending flash commands to the chip at the same time.
In either case, if you have the space it's a good idea to store a "known-good" image elsewhere too, so that if the new image has a problem (fails a checksum or x number of load attempts fails) then you have a working OS that the bootloader can fall back to.
Clearly a lot depends on your hardware setup, but we've done this without making the Bootloader support the Flash Filesystem.
In our product, the OS image is loaded from Flash to execute from RAM -- I think most WinCE devices work this way nowadays. So to update the OS we use a special Flash driver which lets an application, running under WinCE, update the OS blocks in the Flash -- then all you need is a hard reboot and the Bootloader loads the new flash image into RAM in order to execute it. We've found this pretty reliable in the field (with some not-very-technical end-users!).
A special Flash driver was needed because the MS Flash Filesystem drivers have no access to the OS image sectors of the Flash, in order to prevent trashing the OS by accident.
You do need to load the NK.BIN into some memory which the OS programming application can read, normally the NAND Flash, but if you had enough RAM it could just go into the root of the filestore. However either way you can delete it when you've finished programming the OS sectors before the reboot so it's only a temporary requirement.
I will write some thing in a file/memory just before system shutdown or a service shutdown. In the next restart of system, Is it possible to access same file or same memory on the disk, before filesystem loads? Actual requirement is like this, we have a driver that sits between volume level drivers and filesystem driver...in that part of the driver code, I want to access some memory or file.
Thanks & Regards,
calvin
The logical thing here is to read/write this into the registry if it is not too big. Is there a reason you do not want to use the registry?
If you need to access large data and you are writing a volume or device filter and cannot rely on ZwOpen/Read/Write/Close functions in the kernel an approach would be to create the file in user mode, get its device name and cluster chain and store them in the registry. On the next boot, you can get the device and clusters from registry, and do direct I/O on them.
Since you want to access this before the filesystem loads, my first thought is to allocate and use a block of storage space on the hard drive outside of the filesystem. You can create a hidden mini-partition on the drive and use low-level I/O commands to read and write your data.
This is a common task in the world of embedded systems, and we often implement it by adding some sort of non-volatile memory device into the system (flash, battery-backed DRAM, etc) and reading and writing to that device. Since you likely don't have the same level of control over the available hardware as embedded developers do, the closest analogue I can think of would be to reserve a chunk of space on a physical disk that you can read from without having to mount as a filesystem. A dedicated mini-partition might work the best because if you know the size of it, you can treat it as one big raw-access buffer and can avoid having to hassle with filenames, filesystems, etc.