I need raw read data from and write data into drive..not via existed filesystem like fat32 or something....i just wanna raw write read......
I was told in Windows i can use CreateFile WriteFile and ReadFile APIs to access data in drive directly... but I dont know in Linux whether there r similar functions....
dont tell me to use hardware driver programming, coz i am just a newbie in linux programming..
thanx a lot....
Files /dev/hd* and /dev/sd*, depending on your setup, contain raw disk data. You need root account usually.
Use mount command to find out which disks/partitions are mapped to which files.
Related
I am new to MariaDB and need to do below activity.
We are using MariaDB as datatbase and we need to read a txt file from ftp location. Then load into a table. This has to be scheduled to read the file on a regular interval.
After searching I got LOAD DATA INFILE to be used, but it has the limitation that, it can't be used in Events.
Any suggestions/samples on this would be great help.
Thanks
Nitin
You import it and read it using the local path, MariaDB does basic file support, in no case it supports FTP transactions
LOAD DATA can only read a "file". But maybe the OS can play games...
What Operating System? If the OS can hide the fact that FTP is under the covers, then LOAD DATA will be none the wiser.
I am working at an OS independent file manager, in C. I managed to copy files, links and directories, but I am not sure how to copy devices. I would appreciate any help.
To create a device file, use the mknod(2) syscall. The struct stat structure will give you the major and minor device numbers for an existing device in st_rdev.
Having said that, there is little value in "copying" a device because a device doesn't contain anything useful. The major and minor numbers are specific to the OS on which they exist.
It's not really a useful feature, IMHO. tar(1) needs to be able to do it as part of backing up a system, and setup programs need to be able to create them for you when setting up your system, but few people need to deal directly with device files these days.
Also, modern Linux systems are going to dynamic device files, created on the fly. You plug in a device and the device files appear; you unplug it and they disappear. There is really no use in being able to copy these dynamic files.
dd is your friend (man dd)
dd if=/dev/sda1 of=/some_file_or_equally_sized_partition bs=8192
if you want to copy the device-file itself, do this:
cp -p device-filename new-filename
e.g.:
cp -p /dev/sda1 /tmp/sda1
those are both equivalent device files, and can be used to access the device.
If you're want to do this from C, use mknod() .. see "man 2 mknod"
This might be useful
cp -dpR devices /destination_directory
cp -dpR console /mnt/dev
You don't. Just filter them out of the view such that it can't be done.
Use the stat function to determine the file type.
Check if you've the udev package, if you do, chances are that devices are generated on the fly, from the package description:
udev - rule-based device node and kernel event manager
udev is a collection of tools and a daemon to manage events received from the
kernel and deal with them in user-space. Primarily this involves creating and
removing device nodes in /dev when hardware is discovered or removed from the
system.
Events are received via kernel netlink messaged and processed according to
rules in /etc/udev/rules.d and /lib/udev/rules.d, altering the name of the
device node, creating additional symlinks or calling other tools and programs
including those to load kernel modules and initialise the device.
I need to create a "virtual file", a file that if some program access that file, e can answer the size, permissions, and 'say' what is writen in it.
Why?
I have a Virtual Machine that have and Virtual Hard Disk, and i need that file be a 'link' to other and far location (unknow by Virtual Machine Program)
When the VM try to read, i need to 'say' what is writen, and when it´s try to write, i need to store in other location.
In most cases the file is in a network and splited in many computers (like a Network FileSystem) then i need to know every change/access/read in file and deal with the request.
I can't modify/inject the target program (third-party).
I found a question in this site about Virtual File, but i need to know in real time all changes in the file.
I searched for File Hooks and found nothing.
I tryed to use a virtual driver, but it´s hard to find some code (like Daemon Tools, but with Read/Write).
I thought in a remote folder (or mapped drive) with FTP, but the file size is TOO big (10-50 GB) and i need to read specific sections of the file.
Thanks in advance for any help.
(Windows)
What you are looking for is called a Filter Driver.
I've data in MP4 format which needs to be copied to DVD on Linux platform. Now I am creating MP4 file on hard disk and then burning that file to DVD using growisofs command.
It would be more efficient if I didn't have to write the MP4 data to hard disk before they are burned to DVD. Please let me know if there is a way to write in memory data to DVD using C program.
By reimplementing the tasks growisofs performs. DVDs are different to randomly accessible storage. First the data to be burnt onto the blank medium must be prepared into in a certain format, namely ISO9660, this includes a certain error correction scheme. The result of this is a complete Track. In the ISO9660 scheme it's not possible to record single files, only whole file systems. Once you got the FS you must implement the whole program for controlling the recording process.
This is what growisofs does. Now you could take the source of growisofs and replace the code it uses to read the files with code to read from some shared memory. But then you must make sure, that your program can deliver the data continously, without falling into pauses. Once started, the recording process should not be interrupted.
Anyway: If you're under Linux your program could provide the filesystem structure through FUSE.
Is it possible to read damaged media (cd, hdd, dvd,...) even if windows explorer bombs out?
What I mean to ask is, whether there is a set of APIs or something that can access the disk at a very low level (below explorer?) and read whatever can be retrieved even if it is only partial, especially if you can still see the file is there from explorer, but can't do anything with it because it is damaged somehow (scratch on cd, etc)?
The main problem with Windows Explorer is that it doesn't support resuming copying after a read error. Most superficially scratched CDs, for example, will fail on different areas of the disk every time you eject and reinsert them.
Therefore, with a utility that supports resuming copy operations, it is possible to read the entire contents of a damaged CD with by doing "eject/reload/resume" a few times.
In fact, this is what a utility I wrote does, and I've never needed anything fancier to read scratched disks. (It simply uses ReadFile and WriteFile.)
One step lower would be opening the raw partition (i.e. disk image) by passing a string such as "\.\F:" (note: slashes are literal here) to CreateFile. It would allow you to read raw sectors from a drive, but reconstructing files from that data would be hard.
In fact, the "\.\" syntax allows you to open devices in the "\GLOBAL??" branch of the Windows Object Manager namespace as if they were files. It's not unlike calling dd with /dev/x as a parameter. There is also a "\Device" branch, but that's only accessible via DeviceIoControl() (i.e. ioctl()), meaning there's no simple ReadFile()/WriteFile() interface.
Anything lower level than that would be device-specific, I guess; like reading raw CD-ROM data (including ECC bits) the way some CD-burning programs do. You'd have to do some research on the specific media (CD, flash, DVD) and what your hardware allows you to do on them.
Note: The backslashes seem to get lost on the way to the web page; you need to pass "backslash backslash dot backslash DeviceName" to CreateFile. You need to escape them, too, of course.
If you want to do it, do it from the Linux side - see: http://sourceforge.net/projects/monkeycity/ opensource
or ready made app and freeware too: http://www.theabsolute.net/sware/dskinv.html
the first step is dd_rescue. After that, you're free to try anything to reconstruct the data.
And there's GNU ddrescue
GNU ddrescue is a data recovery tool. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying to rescue the good parts first in case of read errors.
Make sure to use the 3-arg version (manual):
ddrescue [options] infile outfile [mapfile]
That is, do use a mapfile even if it's optional, because:
If you use the mapfile feature of ddrescue, the data is rescued very efficiently, (only the needed blocks are read). Also you can interrupt the rescue at any time and resume it later at the same point. The mapfile is an essential part of ddrescue's effectiveness. Use it unless you know what you are doing.
And it's also included in Cygwin and Homebrew.
I don't know what layer exists between Windows Explorer and the Win32 APIs. You can try to write a program with the Win32 File I/O stuff. If that doesn't work, then you have to write your own device driver to get any lower.
I've had some luck from the linux side, or using BartPE (http://www.nu2.nu/pebuilder/), but just seeing the file doesn't always mean the file is going to be recoverable, whether you're trying from Windows or Linux. You're best bet might be to use a trial of a recovery program.
I have had two disks start to disintegrate on me. From the pattern of unreadable sectors I think they had internal flaking of their emulsion. WinXP Explorer just threw up its hands and said the drive didn't even exist.
In both cases I used "GetDataBack for NTFS" from Runtime Software (http://www.runtime.org/). You can download a free trial which will show you what you could get back if you paid for it. When I bought it it was $49, but I see it is now $79.
This program is amazing. It's not necessarily fast as it will reread some sectors over and over, trying to get a consensus value from multiple tries, but when it's done you can get back stuff that you thought was gone forever. I had one drive that it took over 10 hours to analyze, but when it was done I got back over 97% of a 500GB drive. Definitely worth the price.
Another great tool is Beyond Compare. I have rev 2.5.3, but it is currently at 3.?? and costs $30. They have a full-functionality, 30-day trail. It does a great job of copying large quantities of files (and only those that need to be copied) and, unlike Explorer, it doesn't blow up if something fails. It's sort of like a visual rsync for Windows, if you're familiar with that program from the Samba people.
I have no connection with either of the comapnies mentioned other than being a very satisfied customer.
The gold standard for recovering data from a magnetic storage device would have to be SpinRite. It's a commerical app though, so you probably wouldn't learn much from it.
If you have a Linux machine around, I can recommend dvdisaster. It is originally meant for creating error correction files, but it also reads DVDs into an image and ignores read errors; and you can use different drives one after another to get missing sectors filled in the image.