I have an application where I can upload files and add metadata to the file. This metadata information is stored in a database, but parts of the added information is encoded somehow (sadly I have no access to the source code).
The raw representation of the metadata in the Oracle database is as follows:
00000009010000000000000000512005B69801505B000000010000000700000040000000010000000A0100000006496D616765000000003C000000010000000A010000000A696D6167652F706E670000000027000000030000000501000000010000000500000001010000000B64653A3132332E706E6700000002A8000000030000000501000000030000000700000001010000000E737461636B6F766572666C6F770000000042000000010000000A010000001844433078303166363565396420307830303033336433640000000A2600000001000000020100033D3D0000003E000000010000000A0100000021346266653539343939343631356333323861613736313431636337346134353900
Whereas the raw sequence
737461636B6F766572666C6F77
corresponds to
stackoverflow
The query
select UTL_RAW.CAST_TO_VARCHAR2(<raw_data>) from dual;
returns the string below:
Here the values of the metadata are shown. But the names/identifier of the properties are unreadable. The corresponding name/identifier of stackoverflow should be test or a foreign key to a table that contains test. The other data contains additional information about the file (like the checksum, title or mime type)
Is it possible to retrieve the unreadable data (identifier) from the raw string?
RAW columns are not always containing a string, since the results it looks like that the content is binary data, more exactly a jpg file which has a string header in it but among binary information.
Converting it to a varchar will generate invalid charcode that are represented as rectangular boxes.
What you are doing here with varchar is the equivalent of opening a binary file, i.e a winword.doc or even a .jpeg by using Notepad.
To be able to get the content you need to treat it as image, not as varchar.
You can obtain the jpg file by using PLSQL as described here:
http://www.dba-oracle.com/t_extract_jpg_image_photo_sql_file.htm
Eventually it is possible to get all the content without loss in a char datatype using the following:
select RAWTOHEX(<raw_data>) from dual;
This will return the whole content as character value containing its hexadecimal equivalent and should not present any invalid ANSI character which is rapresented with a rectangular box.
Indeed you will not be able to read anymore "stackoverflow" or any other text since you will get only a sequence of HEX values.
You will need then from your program to convert it to binary/image and treat it properly.
Both "A01" and "101" are used to preface a 4 byte length followed by the Text, which is null terminated
00000009 010000000000000000512005B69801505B000000010000000700000040000000010000000A01
00000006 496D61676500 Image
0000003C 000000010000000A01
0000000A 696D6167652F706E6700 image/png
00000027 00000003000000050100000001000000050000000101
0000000B 64653A3132332E706E6700 de:123.png
000002A8 00000003000000050100000003000000070000000101
0000000E 737461636B6F766572666C6F7700 stackoverflow
00000042 000000010000000A01
00000018 444330783031663635653964203078303030333364336400
D C 0 x 0 1 f 6 5 e 9 d 0 x 0 0 0 3 3 d 3 d
00000A26 00000001000000020100033D3D0000003E000000010000000A01
00000021 346266653539343939343631356333323861613736313431636337346134353900
4 b f e 5 9 4 9 9 4 6 1 5 c 3 2 8 a a 7 6 1 4 1 c c 7 4 a 4 5 9
Related
Trying to open a dxf file format (actual project in C++), I could understand the basic structure of the file, but I can't manage to find how a cube is actually represented.
For a cube in CAD, I expect at least 9 values:
X Y Z position
A B C rotation
W H D size of the cube
I expect X, Y, Z, A, B and C to be in the ENTITY section,
But Looking at example files, I see many settings, the preview image (taking a significant space in the file), layouts, etc.. But nothing that I can match to how the cube is actually build.
Question:
How to represent/read a cube in a dxf file?
More info
Here is the documentation about the file format:
http://help.autodesk.com/view/OARX/2018/ENU/?guid=GUID-235B22E0-A567-4CF6-92D3-38A2306D73F3
Here is an example of file with a cube (created with BricsCAD). Unfortunately, I can't embed the file, as it is too big.
The "cube" is of 20mm x 25mm x 30mm.
https://download.escain.org/example_cube_20_25_30_mm.dxf
I checked LibreCAD source-code, but it does not manage 3D models. Also, libdxfrw library is too generic (it just call the interface callback with the full Entity data).
https://github.com/LibreCAD/LibreCAD_3
https://github.com/LibreCAD/libdxfrw
This cube is embedded binary ACIS data (3DSOLID) and can not be interpreted without the libraries from Spatial Inc. For more information see my answer to another question:
How I can parse nurbs surface from dxf file? Or do you know library(for js, if exists or any other language) for parsing it?
EDIT: Find binary data of ACIS entities
Starting with R2013/AC1027 Modeler Geometry of ACIS data is stored in the section ACDSDATA in a ACDSRECORD these records have no handle, instead they have an ID. The record of your 3DSOLID starts at line 22393 and has the ID 10:
0
ACDSRECORD
90
1
2
AcDbDs::ID
280
10
320
D2 <<< handle to 3DSOLID
2
ASM_Data
280
15
94
9259 <<< size in bytes
310
41534D2042696E61... <<< binary data as multiple tags of group code 310
This is your 3DSOLD with handle D2 which starts at line 2187:
0
3DSOLID
5
D2 <<< handle of your 3DSOLID
330
1F
100
AcDbEntity
8
0
100
AcDbModelerGeometry
290
0
2
{00000000-0000-0000-0000-000000000000}
100
AcDb3dSolid
350
0
As you see there is no association from the 3DSOLID to the binary content as ACDSRECORD in the ACDSDATA section.
I have no knowledge of a table (DICTIONARY) that links this data together. The only way I know is to search all ACDSRECORD in the ACDSDATA section for links (group code 320) to ACIS objects.
FYI: In DXF versions prior to R2013 the ACIS data is stored in the entity itself as ascii text with a lousy xor "encryption". All my Knowledge about the DXF format is baked into my Python package: ezdxf.
I am working with a publicly available database in which four files are there : They are all .txt documents. How can I put them in a .mat format ? I am giving a simple example:
A.txt file
1 2 3 4 5 6
7 8 9 1 2 3
4 5 6 7 8 9
1 2 3 4 9 8
So I need to form a matrix with 4 rows and 6 columns. The data in the txt format is separated by 'space' delimiter. The rows are separated by 'newline'. Typically the .txt documents that I will handle will have sizes 130x1000, 3200x58, etc. Can anyone please help me regarding this? The publicly database is available at : click link. Please download the dataset under the topic "Multimodal Texture Dataset".
You can load the .txt file into MatLab:
load audio.txt
then save them
save audio audio
(the first "audio" is the ".mat" file, the second "audio" is the name of the variable stored in it.
Hope this helps.
I have a custom file which contains data in a format like below
prop1: value1
prop2: value2
prop3: value 2
Table Instance 1
A B C D E
10 11 12 13 14
12 13 11 12 20
Table Instance 2
X Y Z
1 3 4
3 4 0
Table Instance 3
P R S
2 3 5
5 5 0
I want to be able to parse this file and map the contents to a POCO. I was really excited about working with CSV type provider in F#, but then I quickly realized that it might be not possible to use that in my case. Should I have to write my own parser in this case? (Iterate through each like and parse the values into its appropriate properties in POCO)
Thanks
Kay
If that's a one-of file format, I would just write a parser by hand. Split the file into separate tables, throw away the title and header, then String.Split each row and parse the resulting array into a record type specific for the table.
If that file format is more or less standardized and you expect that you'll need to parse it more often and in different contexts (and/or you're feeling adventurous), you can always write your own type provider.
I am trying to copy all of the records from a data file (STUDMARKS) into my physical file (MARKS) using the CPYF command.
A R MARKSR TEXT('Marks Records')
A STUDENTID 9S 0 COLHDG('Student' 'ID')
A COURSE_CD 6A COLHDG('Course' 'Code')
A FINAL_MARK 3S COLHDG('Final' 'Mark')
A DATERUN L COLHDG('Date' 'Run')
A K STUDENTID
A K COURSE_CD
This is what I currently have in my MARKS.pf. The STUDMARKS.pf-dta file has the first three records already defined, the DATERUN record get filled with the date of use.
CPYF FROMFILE(IBC233LIB/STUDMARKS) TOFILE(DS233B32/MARKS) MBROPT(*REPLACE) FMTOPT(*MAP *DROP)
The above is the CPYF command that I ran after creating MARKS.pf, and after doing a RUNQRY to see all the records I've noticed that all but the COURSE_CD have been filled. COURSE_CD is completely blank.
I did some research before hand and did a DSPFFD on both members to ensure that the record lengths and types were all the same, which they were. I did notice, however, that in STUDMARKS.pf-dta that all the records had a buffer length which was equivalent to the field length. The STUDENTID field in MARKS.pf was the only one to not share this property, where the field length is 9, but the buffer length is only 5. I'm not sure if it's the reason why I'm having such difficulty, and the matter is almost certainly less so than what I'm making it out to be, but I've been at this for quite some time and a just can't seem to copy records from one member to another.
It's incredibly frustrating, and help would be greatly appreciated
I took screen shots of the DSPFFD commands for both files
For STUDMARKS
And For MARKS
EDIT
Just now seeing the spelling error! Smashing my head against the desk but I almost guarantee that is the problem. All of your answers were very informative and helpful though, so thank you very much
EDITEDIT
for others, despite the fact that I did change the names when recompiling the program, it will not work unless you delete the file first and THEN compile it. Very frustrating, but that's just how it is...
So DLTF [file name] and then recompile
As James noted, the differences in buffer length for STUDENTID are due to one file having it defined as packed and the other having it defined as zoned.
This won't matter to CPYF as both are a compatible numeric and CPYF will map between them as you've seen.
However, this proves that there's more than just a missing field different between the two files. Use DSPFFD and look at post the definitions of COURSE_ID from both files.
I'd bet either the names are different or the types are.
What you are experiencing is the difference between a packed and a signed decimal field.
More than likely you forgot to specify a datatype in position 35 of the DDS specification for the STUDENTID field in the MARKS file.
For example:
A STUDENTID 9S 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID ZONED 9 0 9 1 Both Student
ID
A STUDENTID 9 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID ZONED 9 0 5 1 Both Student
ID
A STUDENTID 9P 0 COLHDG('Student' 'ID')
Data Field Buffer Buffer Field Column
Field Type Length Length Position Usage Heading
STUDENTID PACKED 9 0 5 1 Both Student
ID
The explanation for this behaviour can be found in the DDS reference in the section Data type for physical and logical files (position 35):
For physical files, if you do not specify a data type or duplicate one from a referenced field, the operating system assigns the following defaults:
A (character) if the decimal positions 36 through 37 are blank.
P (packed decimal) if the decimal positions 36 through 37 contain a number in the range 0 through 63.
Because the data types are different the FMTOPT(*MAP *DROP) tells the CPYF command to silenty drop and default any non-matching fields.
The odd thing is the file field description identifies the field as ZONED when it is really PACKED.
The *DROP value for the FMTOPT parameter excludes like named fields that do not have the same attribute and relative position in both files. The COURSE_CD field has a different position in the receiving file.
Does anyone know how I can reuse inline data in Gnuplot, I've been googling it and can't find nothing everything suggests to input the data gain? Basically reuse the '-' file.
in place of a bare replot, you can use refresh if you're using gnuplot 4.3 or newer. If you actually want to add more data to be plotted, I think you're out of luck.
e.g.
plot '-' u 1:2
1 2
2 3
e
set label "Hello World!" at 1.5,2.5
refresh
since I stumbled over this old question via Google...
There are two ways to having "inline data" (data in the gnuplot file):
the special filename '-', which reads the lines immediately following the plot command. This data can only be used once.
named datablocks with here documents, which can be reused:
$Data << EOD
0 0 0
1 1 1
2 2 4
3 3 9
4 4 16
EOD
plot $Data using 1:2 title 'linear' with linespoints, \
$Data using 1:3 title 'quadratic' with linespoints
See http://gnuplot.info/docs_5.5/loc3521.html