I have a set of .DAT files present along side a set of .IDX files with the same name.
The goal is to be able to open these files and read its contents, parsing it into a new format. The problem: I have no idea what database the data is being stored in! The files contain no headers or clues, they are binary, and the resource from which I have received these has no idea as to its storage mechanism.
So the question is: What are some common databases which store databases in .DAT files and store their indexes in .IDX files with the same name? Is there an application I can use in Linux or Windows which can detect the database?
EDIT :-
File names:
price.dat
price.idx
Here is a hex dump of the beginning of the .DAT file:
030D04806420500FFE3E0500002078581001C000738054E0C0099804138100402550080442090082403C101F7406010080C0A010201002010C006FC0246C0403FE00B041C051F0091BFE042F812FE054F8177E066F81BFE078F8207E08AF824FE09CF8297E0AEF82DFE0C0F8327E0D2F836FE0E4F83B7E0F6F83FE5FEFF47C06608480FA91F003C0213101F1BFDFE804220100F500D2A00388430801E04028D4390D128B46804024010A067269FCA546003C0844060E11F084B9E1377850
Here is a hex dump of the beginning of the .IDX file:
030D04805820100FFD7E0000397FEB60050410007300246A3060068220009BE0401030088B3903F740E010C80402410281402030094004C708004DC058880FFC052F015EBFE042F812FE054F8177E066F81BFE078F8207E08AF824FE09CF8297E0AEF82DFE0C0F8327E0D2F836FE0E4F83B7E0F6F83FFE108F8447E11AF848FE12CF84D7E13EF851FE150F8567E162F85AFE174F85F7E186F863FE198F8687E1AAF86CFE1BCF8717E1CEF875FE1E0F87A7E1F2F87EF5FEFF005E30901714
Both files uniquely start out with 030D04806420500FF wonder if this is a good start?
Did a quick search on Google but it didn't return anything...
END EDIT :-
Any other ideas?
Thanks much in advance!
There is a faircom ODBC driver called 'ctreeODBC_RO.exe' which should be capable.
Related
For example, if we have the following file: file.txt that after the compression is now file.new (new is the new extension) , how to obtain that .txt extension, that is forgotten?
I need that to decompress the file.
In general, if you lose the file name extension you can't get it back. It's as simple as this.
However, there might be chances depending on the compression format. Some formats do store the original file name (along with other informations) in the compressed file. And the "decompressor" will be able to recreate those properties.
Anyway, it's good practise to name a compressed file with an additional extension, in your case file.txt.new.
Oh, and you don't need to know the file name extension to uncompress the compressed file. Just uncompress it and give it a temporary name. As #MarcoBonelli said, file contents and file name extensions have no fixed relation. They are just a convention to handle them conveniently.
For example: You can rename a EXE to DOCX. Windows will show the Word icon but it is still an executable. Windows will not attempt to run it, though.
To know what a file contains can be difficult. The magic number Marco linked to might give you some hint.
I need to store data in a file in this format
word, audio, jpeg
How would I store that all in one file? Is it even possible do would I need to store links to other data files in place of the audio and jpeg. Would I need a custom file format?
1. Your own filetype
As mentioned by #Ken White you would need to be creating your own custom file format for this sort of thing, which would then mean creating your own parser type. This could be achieved in almost any language you wanted but since you are planning on using word format, then maybe C# would be best for you. However, this technique could be quite complicated and take a relatively large amount of time to thoroughly test your file compresser / decompressor, but may be best depending on your needs.
2. Command line utilities
Another way to go about this would be to use a bash script to combine all of the files into one file, and then decompress it at the other end. For example the steps could involve:
Combine files using windows copy / linux cat command on command line
Create a metdata file of your own that says how many files are in this custom file, and how much memory each one takes up (could be a short XML or JSON file for example...)
Use the linux split command or install a Windows command line file splitter program (here's just one example) to split the file back into whatever components have made it up.
This way you only have to create a really small file type, and let the OS utilities handle the combining of them for you.
Example on Windows:
Copy all of the files in your current directory into one output file called 'file.custom'
copy /b * file.custom
Generate custom file format describing metadata (i.e. get the file size on disk in C# example here). This is just maybe what I would do in JSON. SO formatting was being annoying so here's a link (Copy paste it into an editor or online JSON viewer).
Use a decompress windows / linux command line tool to decompress each files to the exact length (and export it back to the exact name) specified in the JSON (metadata) file. (More info on splitting files on this post).
3. ZIP files
You could always store all of the files in a compressed zip file, and then just use a zip compressor, expander as and when you like to retreive any number of file formats stored within.
I found a couple of examples of :
Combining multiple files into one ZIP file in only C# .net,
Unzipping ZIP files in C#
Zipping & Unzipping with only windows built-in utilities
Zipping & Unzipping in Linux command line
Good Zipping/Unzipping library in Java
Zipping/Unzipping in Python
Is it possible to open a flatfile when only part of the file name is known?
I have files in a directory that have a timestamp appended to the filename, is it possible to open it by specifiying the known part of the filename (excluding timestamp)?
Is it possible with a PLSQL only approach?
There is a dbms_ package which allows you to get a directory listing for the directory (or you can implement your own in a java stored procedure - google!) This will allow you to find the file you are looking for - if necessary choose which is the relevant file and then process.
See http://notdennis.wordpress.com/2013/07/03/listing-directory-files-plsql/
Is there a possibility to include a text file in a .exe file? or atleast hide it from the user view? I mean let us take an example,I have a target.exe file in which it opens and reads the contents of a data.txt file and yes its working perfect with my computer But when i transfer these target.exe file without a data.txt file to some other computer where you dont have the required data.txt file.It results an error and the thing I want you to know is data.txt file has some information like example a contact info which is a confidential text.When someone runs these .exe file he should enter the name and the data is displayed about the contact info but it works only if you have data.txt file. But i want the data.txt file to be hidden it cannot be accessed normally.The data in data.txt file can be accessed only through .exe file.How could I solve it? and remember i should give my friends only the .exe file and using that .exe file they can save thier data and display contact info. Does any one have any idea to do it?
yes, you can include any user data into recource and link it with your .exe
Resources in Windows
Resource compiler reference
LoadResource
If the exe opens the file, then it is nearly impossible to prevent users from accessing the contents of that file. If you store it as a raw resource, then one can use a resource editor to view it. If you do some sort of basic encryption, then using ProcExplorer, one could view string in the process to see the information when the program executes. You could use DRM style protections, but that seems like overkill.
The answer to your general question is yes, you can store resources in an EXE file and then the EXE can open and load those resources at runtime.
I have a little utility that converts .dbc files to .csv files, trouble is, somewhere in the conversion some data is lost/destroyed/whatever. I input a.dbc into converter, it produces a.csv. I delete a.dbc,and then run a.csv back through the converter, and I come back with a "slightly" different .dbc file then I had started with.
Does anyone know any better way of converting these files? Without loss of information..
I open both files in HexCMP (compares two hex files, show's you the differences) and the differences are totally random through out the file.
Sounds like this is nothing more than a buggy utility.
If you convert the same .dbc file to a .csv file twice in a row, do you get the exact same .csv file? If you run the .csv through twice do you get the same .dbc file out both times? That would at least tell you which side of the conversion the bugs are in.
Do you have access to FoxPro to export the file as a CSV directly from FoxPro without using the utility? That would allow you to compare the CSV file created from FoxPro versus your utility to try and narrow down where the problem is.