I want to define the MIME type of *.txt files: text/txt, so that Tika can apply a more specific parser than the one used for text/plain files.
The glob *.txt is included in the definition of the type text/plain in tika-mimetypes.xml. Moreover, it seems to me that you cannot redefine a MIME type in custom-mimetypes.xml, only add new globs or magic patterns. Additionally, if I define the text/txt type in tika-mimetypes.xml as a subtype of text/plain with only the glob *.txt, Tika still detects a txt file as text/plain.
Is it absurd to define a subtype of text/plain only for txt files? If not, is it possible to define it only with custom-mimetypes.xml? If not, what is the easiest way to extend tika so that it can parse txt files differently than (let's say) STEP 3D CAD .stp files or .cfg files?
The use case in detail: I have a large source of data composed of (recursive) archives. Some plain text files are huge and I don't want Tika to parse them. However, I want to keep all the txt files.
Edit: specify that I don't want to keep .cfg files either (*.cfg is a glob of text/plain)
Related
For example, if we have the following file: file.txt that after the compression is now file.new (new is the new extension) , how to obtain that .txt extension, that is forgotten?
I need that to decompress the file.
In general, if you lose the file name extension you can't get it back. It's as simple as this.
However, there might be chances depending on the compression format. Some formats do store the original file name (along with other informations) in the compressed file. And the "decompressor" will be able to recreate those properties.
Anyway, it's good practise to name a compressed file with an additional extension, in your case file.txt.new.
Oh, and you don't need to know the file name extension to uncompress the compressed file. Just uncompress it and give it a temporary name. As #MarcoBonelli said, file contents and file name extensions have no fixed relation. They are just a convention to handle them conveniently.
For example: You can rename a EXE to DOCX. Windows will show the Word icon but it is still an executable. Windows will not attempt to run it, though.
To know what a file contains can be difficult. The magic number Marco linked to might give you some hint.
I need to store data in a file in this format
word, audio, jpeg
How would I store that all in one file? Is it even possible do would I need to store links to other data files in place of the audio and jpeg. Would I need a custom file format?
1. Your own filetype
As mentioned by #Ken White you would need to be creating your own custom file format for this sort of thing, which would then mean creating your own parser type. This could be achieved in almost any language you wanted but since you are planning on using word format, then maybe C# would be best for you. However, this technique could be quite complicated and take a relatively large amount of time to thoroughly test your file compresser / decompressor, but may be best depending on your needs.
2. Command line utilities
Another way to go about this would be to use a bash script to combine all of the files into one file, and then decompress it at the other end. For example the steps could involve:
Combine files using windows copy / linux cat command on command line
Create a metdata file of your own that says how many files are in this custom file, and how much memory each one takes up (could be a short XML or JSON file for example...)
Use the linux split command or install a Windows command line file splitter program (here's just one example) to split the file back into whatever components have made it up.
This way you only have to create a really small file type, and let the OS utilities handle the combining of them for you.
Example on Windows:
Copy all of the files in your current directory into one output file called 'file.custom'
copy /b * file.custom
Generate custom file format describing metadata (i.e. get the file size on disk in C# example here). This is just maybe what I would do in JSON. SO formatting was being annoying so here's a link (Copy paste it into an editor or online JSON viewer).
Use a decompress windows / linux command line tool to decompress each files to the exact length (and export it back to the exact name) specified in the JSON (metadata) file. (More info on splitting files on this post).
3. ZIP files
You could always store all of the files in a compressed zip file, and then just use a zip compressor, expander as and when you like to retreive any number of file formats stored within.
I found a couple of examples of :
Combining multiple files into one ZIP file in only C# .net,
Unzipping ZIP files in C#
Zipping & Unzipping with only windows built-in utilities
Zipping & Unzipping in Linux command line
Good Zipping/Unzipping library in Java
Zipping/Unzipping in Python
I read the mimetype for .docx file is application/vnd.openxmlformats-officedocument.wordprocessingml.document. But when I upload a .docx file(one that I just created, not from a zip file) and check for its mimetype in my application using
String mimeType = Magic.getMagicMatch(file1, false).getMimeType();
I get Mimetype as application/zip.
I get the same result when I try to upload a .jar file.
I mean this way, how can I check if the user is uploading a msword or a jar file to my application?
All of the .*x Office variants (.docx, .pptx, and so on) are XML-based content which is wrapped in a ZIP "container" to keep them compact, and your library is detecting the ZIP header correctly but then either not checking for, or failing to find, the additional information that would allow it to distinguish those from a ZIP file containing whatever random data someone put into it.
Similarly, the JAR file format is an extension of the ZIP file format, so if the library does not know to check for the "special type of ZIP" case, it would simply report it as a ZIP file.
I'd like to make a dynamically generated .ini file available for download, is there a standardized value for the Content-Type of .ini files?
I've found some places saying it's text/plain, but I'm a bit dubious.
Checking with the IANA list of MIME Media Types shows us that .ini is not listed in the list of media types (and in particular, in the list of Text Media Types.)
Additionally, the Wikipedia entry on .ini files explains that .ini files are really just "simple text files."
One way to verify these findings is to run the following command on a given .ini file (in *nix, at least):
$ file --mime-type php.ini.default
which gives us the following result:
php.ini.default: text/plain
I have written the C code and have used libxml to parse the xml file by providing its absolute file , now my requirement is to pass only the folder name and to pick all the xm files one by one and parse them completely. is there any specific method through which libxml takes all the files from the folder or can we mention files sperated by comma or any other delimiter in the configuration file through which libxml picks the path of the file to parse.
You should look at FindFirstFile and FindNextFile, since you're on Windows.