After struggling for days, I'd like to ask the community :)
I have two exe files. Both at the same size and pretty big (around 80MB).
The first exe is the original file, which has been compiled some time ago. I also have the corresponding PDB file.
The second file is almost identical to the first, but it has been binary patched at several locations.
I managed to decompile the files with Cutter (Radare2 GUI), but thinking about the file-size, it's almost impossible to go through every single method.
With an Hex Editor (Hex Fiend), I managed to get every single offset location, where the patch has been applied. Now I'm trying to find these offsets with Cutter to decompile the locations.
The Question is: How?
When I open the Hexdump in Cutter, it seems that everything is not in place and I might need an offset. Setting an offset at the loading-screen in Cutter seems not to do anything. Am I using it wrong?
Example:
HexFiend
Offset Hex Data Text
474942-474956 47 65 74 4D 6F 64 75 6C 65 48 61 6E 64 6C 65 41 GetModuleHandleA
Cutter
Offset Hex Data Text
0x0060713E-0x0060714D 47 65 74 4D 6F 64 75 6C 65 48 61 6E 64 6C 65 41 GetModuleHandleA
474942 is 0x73F3E, but that's far away from 0x0060713E
Maybe I need other tools?
Any help is appreciated!
Related
I'm trying to scan the list of available networks and enumerate the security type for each SSID. I'm at a point where I can issue a SIOCSIWSCAN ioctl and parse the results. However, when I try to differentiate between a WEP network and an open network, I seem to be getting the same type of IE from the AP.
For example, I configured my Dlink DIR-655 router to be of open type vs WEP.
Since the network is a WEP network, I look for the hex byte 0xDD to tell me that this is an IE describing a WPA/WEP/open network. For this case, I only get one byte that says 0xDD for both open and WEP networks and the corresponding IE looks the same for both:
DD 18 00 50 F2 02 01 01 83 00 03 A4 00 00 27 A4 00 00 42 43 5E 00 62 32 2F 00
Does that mean that the router doesn't populate information about open networks under the byte 0xDD and I should be looking somewhere else?
PS: I've been reverse engineering the source from iwlist to tell me how to read the IEs returned. But they only seem to be describing WPA and WPA2 networks
It's not a duplicate. I've read em all.
I have a Nokia-N8-00. It's music player supports USLT (UnSynchronised Lyrics/Text). I use a tool called spotdl (https://github.com/Ritiek/Spotify-Downloader) that fetches song titles from "spotify" and downloads them from other sources (generally youtube) and merges metadata as well.
The problem is then, the music downloaded by that tool have lyrics on all my devices except N8. Fortunately, I got a music that had embedded lyrics that is supported on my phone too. I then analyzed both the files and found that in binary sequence, they have a very little difference (just for USLT section but they are different songs). The differences are :-
The one that supports :
55 53 4C 54 00 00 0A 56 00 00 03 58 58 58
The one that doesn't :
55 53 4C 54 00 00 07 38 00 00 01 58 58 58
(These sequences are for "USLT" declaration in the file)
I think it's an encoding difference. If I am right, what encoding is present and in which one? If it's not encoding, what is it?
I know these sequences can't elaborate the situation. So, here are the files I'm trying https://github.com/gaurav712/music.
I don't need supported USLT, I am just curious about it as I wanna make an implementation of it in C (I don't need language specific help though).
Here is what I got:
55 53 4C 54
Translates to:
USLT
So we got that right. Now, I believe we can merge that result with this answer:
Frame ID $xx xx xx xx (four characters)
Size $xx xx xx xx
Flags $xx xx
Encoding $xx
Text
(Taken from: ID3v2 Specification)
(or see this: https://web.archive.org/web/20161022105303/http://id3.org/id3v2-chapters-1.0)
Now, I couldn't get this from the source (because the site was down) but there is also this:
Encoding flag explanation:
• $00 ISO-8859-1 [ISO-8859-1]
• $01 UTF-16 [UTF-16]
• $02 UTF-16BE [UTF-16]
• $03 UTF-8 [UTF-8]
So, according to these findings (which I'm not too sure about), the one that is supported is UTF-8 encoded and the one not supported is UTF-16.
EDIT
I've downloaded and viewed your mp3 files for further inspection. Here are my new findings:
First of all, we were correct about the encodings:
UTF-8 is on supported:
UTF-16 is on unsupported:
Does this mean you can just turn '01' into '03' and it'll magically work? I doubt. It depends on the driver. What if the driver sees '\x00' bytes and iterprets it as end of string (as in end of USLT payload). To test this, you can try manually converting the encoding on the file (by removing extra bytes).
Secondly, running eyeD3 on linux on both files, I recovered that:
supported.mp3 -> ID3 v2.4
unsupported.mp3 -> ID3 v2.3
Perhaps that's an issue?
Also, note that the location of USLT tag in both files are different:
supported.mp3:
unsupported.mp3:
On linux, there are further tools to give you extra information, if need be:
mp3info, id3info, id3tool, exiftool, eyeD3, lltag
Are a couple examples. However, I think the main problem is in the text encoding. I was able to recover the lyrics quite fine using the above tools. But some of the tools give different answers because of ID3 version being different and so on.
This file works fine (UTF-8):
$ cat ok.txt
291054 Ţawī Rifā
This file causes an error (UTF-8):
$ cat bad.txt
291054 Ţawī Rifā‘
Here's the message:
$ freebcp 'DB.dbo.table' in bad.txt ... -c
Starting copy...
Msg 20050, Level 4
Attempt to convert data stopped by syntax error in source field
Msg 4895, Level 16, State 2
Server '...', Line 1
Unicode data is odd byte size for column 2. Should be even byte size.
Msg 20018, Level 16
General SQL Server error: Check messages from the SQL Server
The only difference is the last character, which is unicode 2018 (left single quotation mark)
Any idea what is causing this error?
The SQL Server uses UTF-16LE (though TDS starts with UCS-2LE and switches over I believe)
The column in question is nvarchar(200)
Here's the packet sent right before the error:
packet.c:741:Sending packet
0000 07 01 00 56 00 00 01 00-81 02 00 00 00 00 00 08 |...V.... ........|
0010 00 38 09 67 00 65 00 6f-00 6e 00 61 00 6d 00 65 |.8.g.e.o .n.a.m.e|
0020 00 69 00 64 00 00 00 00-00 09 00 e7 90 01 09 04 |.i.d.... ...ç....|
0030 d0 00 34 04 6e 00 61 00-6d 00 65 00 d1 ee 70 04 |Ð.4.n.a. m.e.Ñîp.|
0040 00 13 00 62 01 61 00 77-00 2b 01 20 00 52 00 69 |...b.a.w .+. .R.i|
0050 00 66 00 01 01 18 - |.f....|
Update: This issue has apparently been fixed in FreeTDS v1.00.16, released 2016-11-04.
I can reproduce your issue using FreeTDS v1.00.15. It definitely looks like a bug in freebcp that causes it to fail when the last character of a text field has a Unicode code point of the form U+20xx. (Thanks to #srutzky for correcting my conclusion as to the cause.) As you noted, this works ...
291054 Ţawī Rifā
... and this fails ...
291054 Ţawī Rifā‘
... but I found that this also works:
291054 Ţawī Rifā‘x
So, an ugly workaround would be to run a script against your input file that would append a low-order non-space Unicode character to each text field (e.g., x which is U+0078, as in the last example above), use freebcp to upload the data, and then run an UPDATE statement against the imported rows to strip off the extra character.
Personally, I would be inclined to switch from FreeTDS to Microsoft's SQL Server ODBC Driver for Linux, which includes the bcp and sqlcmd utilities when installed using the instructions described here:
https://gallery.technet.microsoft.com/scriptcenter/SQLCMD-and-BCP-for-Ubuntu-c88a28cc
I just tested it under Xubuntu 16.04, and although I had to tweak the procedure a bit to use libssl.so.1.0.0 instead of libssl.so.0.9.8 (and the same for libcrypto), once I got it installed the bcp utility from Microsoft succeeded where freebcp failed.
If the SQL Server ODBC Driver for Linux will not work on a Mac then another alternative would be to use the Microsoft JDBC Driver 6.0 for SQL Server and a little bit of Java code, like this:
connectionUrl = "jdbc:sqlserver://servername:49242"
+ ";databaseName=myDb"
+ ";integratedSecurity=false";
String myUserid = "sa", myPassword = "whatever";
String dataFileSpec = "C:/Users/Gord/Desktop/bad.txt";
try (
Connection conn = DriverManager.getConnection(connectionUrl, myUserid, myPassword);
SQLServerBulkCSVFileRecord fileRecord = new SQLServerBulkCSVFileRecord(dataFileSpec, "UTF-8", "\t", false);
SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(conn)) {
fileRecord.addColumnMetadata(1, "col1", java.sql.Types.NVARCHAR, 50, 0);
fileRecord.addColumnMetadata(2, "col2", java.sql.Types.NVARCHAR, 50, 0);
bulkCopy.setDestinationTableName("dbo.freebcptest");
bulkCopy.writeToServer(fileRecord);
} catch (Exception e) {
e.printStackTrace(System.err);
}
This issue has nothing to do with UTF-8 given that the data being transmitted, as shown in the transmission packet (bottom of the question) is UTF-16 Little Endian (just as SQL Server would be expecting). And it is perfectly good UTF-16LE, all except for the missing final byte, just like the error message implies.
The problem is most likely a minor bug in freetds that incorrectly applies logic meant to strip off trailing spaces from variable length string fields. There are no trailing spaces, you say? Well, if it hadn't gotten chopped off then it would be a little clearer (but, if it hadn't gotten chopped off there wouldn't be this error). So, let's look at what the packet to see if we can reconstruct it.
The error in the data is probably being overlooked because the packet contains an even number of bytes. But not all fields are double-byte, so it doesn't need to be an even number. If we know what the good data is (prior to the error), then we can find a starting point in the data and move forwards. It is best to start with Ţ as it will hopefully be above the 255 / FF value and hence take 2 bytes. Anything below will have a 00 and many of the characters have that on both sides. While we should be able to assume Little Endian encoding, it is best to know for certain. To that end, we need at least one character that has two non-00 bytes, and bytes that are different (one of the character is 01 for both bytes and that does not help determine ordering). The first character of this string field, Ţ, confirms this as it is Code Point 0162 yet shows up as 62 01 in the packet.
Below are the characters, in the same order as the packet, their UTF-16 LE values, and a link to their full details. The first character's byte sequence of 62 01 gives us our starting point, and so we can ignore the initial 00 13 00 of line 0040 (they have been removed in the copy below for readability). Please note that the "translation" shown to the right does not interpret Unicode, so the 2-byte sequence of 62 01 is displayed as 62 by itself (i.e. lower-case Latin "b") and 01 by itself (i.e. non-printable character; displayed as ".").
0040 xx xx xx 62 01 61 00 77-00 2b 01 20 00 52 00 69 |...b.a.w .+. .R.i|
0050 00 66 00 01 01 18 ?? - |.f....|
Ţ -- 62 01 -- http://unicode-table.com/en/0162/
a -- 61 00 -- http://unicode-table.com/en/0061/
w -- 77 00 -- http://unicode-table.com/en/0077/
ī -- 2B 01 -- http://unicode-table.com/en/012B/
-- 20 00 -- http://unicode-table.com/en/0020/
R -- 52 00 -- http://unicode-table.com/en/0052/
i -- 69 00 -- http://unicode-table.com/en/0069/
f -- 66 00 -- http://unicode-table.com/en/0066/
ā -- 01 01 -- http://unicode-table.com/en/0101/
‘ -- 18 20 -- http://unicode-table.com/en/2018/
As you can see, the last character is really 18 20 (i.e. a byte-swapped 20 18 due to the Little Endian encoding), not 01 18 as it might appear if reading the packet starting at the end. Somehow, the final byte -- hex 20 -- is missing, hence the Unicode data is odd byte size error.
Now, 20 by itself, or followed by 00, is a space. This would explain why #GordThompson was able to get it working by adding an additional character to the end (the final character was no longer trimmable). This could be further proven by ending with another character that is a U+20xx Code Point. For example, if I am correct about this, then ending with ⁄ -- Fraction Slash U+2044 -- would have the same error, while ending with ⅄ -- Turned Sans-Serif Capital Y U+2144 -- even with the ‘ just before it, should work just fine (#GordThompson was kind enough to prove that ending with ⅄ did work, and that ending with ⁄ resulted the same error).
If the input file is null (i.e. 00) terminated, then it could simply be the 20 00 ending sequence that does it, in which case ending with a newline might fix it. This can also be proven by testing a file with two lines: line 1 is the existing row from bad.txt, and line 2 is a line that should work. For example:
291054 Ţawī Rifā‘
999999 test row, yo!
If the two-line file shown directly above works, that proves that it is the combination of a U+20xx Code Point and that Code Point being the last character (of the transmission more than of the file) that exposes the bug. BUT, if this two-line file also gets the error, then it proves that having a U+20xx Code Point as the last character of a string field is the issue (and it would be reasonable to assume that this error would happen even if the string field were not the final field of the row, since the null terminator for the transmission has already been ruled out in this case).
It seems like either this is a bug with freetds / freebcp, or perhaps there is a configuration option to not have it attempt trimming trailing spaces, or maybe a way to get it to see this field as being NCHAR instead of NVARCHAR.
UPDATE
Both #GordThompson and the O.P. (#NeilMcGuigan) have tested and confirmed that this issue exists regardless of where the string field is in the file: in the middle of a row, at the end of the row, on the last row, and not on the last row. Hence it is a general issue.
And in fact, I found the source code and it makes sense that the issue would happen since there is no consideration for multi-byte character sets. I will file an Issue on the GitHub repository. The source for the rtrim function is here:
https://github.com/FreeTDS/freetds/blob/master/src/dblib/bcp.c#L2267
Regarding this statement:
The SQL Server uses UTF-16LE (though TDS starts with UCS-2LE and switches over I believe)
From an encoding stand-point, there is really no difference between UCS-2 and UTF-16. The byte sequences are identical. The only difference is in the interpretation of Surrogate Pairs (i.e. Code Points above U+FFFF / 65535). UCS-2 has the Code Points used to construct Surrogate Pairs reserved, but there was no implementation at that time of any Surrogate Pairs. UTF-16 simply added the implementation of the Surrogate Pairs in order to create Supplementary Characters. Hence, SQL Server stores and retrieves UTF-16 LE data without a problem. The only issue is that the built-in functions don't know how to interpret Surrogate Pairs unless the Collation ends with _SC (for Supplementary Characters), and those Collations were introduced in SQL Server 2012.
This might be an encoding issue of the source file.
As you are using non-standard characters, the source file should be unicode by itself probably. Other encodings use a differing count of bytes (one up to three) to encode one single character. E.g. your Unicode 2018 is 0xE2 0x80 0x98 in UTF-8.
Your packet ends with .R.i.f....| while there should be your ā‘. And the error shows Server '...', Line 1.
Try to find out the encoding of your source file (look at big and little endian too) and try to convert your file to a sure unicode format.
This might solve it:
inf your /etc/freetds/freetds.conf
add:
client charset = UTF-8
also found this about the flag use utf-16
use utf-16 Instead of using UCS-2 for database wide
character encoding use UTF-16. Newer Windows versions use this
encoding instead of UCS-2. This could result in some issues if clients
assume that a character is always 2 bytes.
I am using the Telpo TPS300 POS terminal to try and read information from a VISA bank Card. The terminal comes with C libraries so sending commands is a lot more easier. However the when I run the SELECT APDU command
(00 A4 04 00 )Lc=0, it returns the following hex data 18byte long as below
6F 10 84 08 A0 00 00 00 03 00 00 00 A5 04 9F 65 01 FF.
I read the ISO 7816-4 specification and the EMV specification and from the look of things, my returned data seems to be lacking one of the mandatory tag 88 as specified in EMV Specification 11.3.4
When I try to SELECT the returned DF name i.e one with tag 84 (A0 00 00 00 03 00 00 00), it returns the same information. All other commands were not successful as well specifically I tried READ RECORD, VERIFY, GET PROCESSING OPTIONS, GET CHALLENGE and they all return the SW 6D 00 (Instruction code not supported or invalid). I just want to retrieve user info from the card and perform an offline authentication of the PIN using the verify command.
I have looked around the web but no one seems to answer me. I have read the standard ISO 7816-4 and EMV Specification again and again on the commands and response interactions but no luck so far because I can't go beyond this step (SELECT command response)
I am using the Telpo TPS300 POS terminal to try and read information from a VISA bank Card
As you said you tried with blank card , here information is coming from card is correct.
when you send select command like,
00 A4 04 00 00 , it select ISD - Issuer Security Domain and return ISD AID i.e. A0 00 00 00 03 00 00 00 with tag 9F 65 that means -Maximum length of data field in command message
Recv - 6F 10 84 08 A0 00 00 00 03 00 00 00 A5 04 9F 65 01 FF
what you receive showing AID of ISD and value of tag 9F65. it seems correct.
my returned data seems to be lacking one of the mandatory tag 88 as specified in EMV Specification 11.3.4
Tag 88 -SFI of the Directory Elementary File is come out from card when you select PSE directory with using command,
00 A4 04 00 0E 315041592E5359532E4444463031 (select PSE command)
it will give you tag 88 if PSE is installed in the card.
I tried READ RECORD, VERIFY, GET PROCESSING OPTIONS, GET CHALLENGE and they all return the SW 6D 00 (Instruction code not supported or invalid).
for reading emv card, EMV application should be installed and personalize in the card then only you can get information from the card with using sequence of commands. try- how to read emv card
it gives basic idea to read emv card with sequence of commands.
hope it helps you..
I have a number of data files created by many different programs. Is there a way to determine the database and version of the database that was used to create the data file.
For example, I'd like to identify which files are created from Microsoft Access, dBASE, FileMaker, FoxPro, SQLite or others.
I really just want to somehow quickly scan the files, and display information about them, including source Database and Version.
For reference, I'm using Delphi 2009.
First of all, check the file extension. Take a look at the corresponding wikipedia article, or other sites.
Then you can guess the file format from its so called "signature".
This is mostly the first characters content, which is able to identify the file format.
You've an updated list at this very nice Gary Kessler's website.
For instance, here is how our framework identify the MIME format from the file content, on the server side:
function GetMimeContentType(Content: Pointer; Len: integer;
const FileName: TFileName=''): RawUTF8;
begin // see http://www.garykessler.net/library/file_sigs.html for magic numbers
result := '';
if (Content<>nil) and (Len>4) then
case PCardinal(Content)^ of
$04034B50: Result := 'application/zip'; // 50 4B 03 04
$46445025: Result := 'application/pdf'; // 25 50 44 46 2D 31 2E
$21726152: Result := 'application/x-rar-compressed'; // 52 61 72 21 1A 07 00
$AFBC7A37: Result := 'application/x-7z-compressed'; // 37 7A BC AF 27 1C
$75B22630: Result := 'audio/x-ms-wma'; // 30 26 B2 75 8E 66
$9AC6CDD7: Result := 'video/x-ms-wmv'; // D7 CD C6 9A 00 00
$474E5089: Result := 'image/png'; // 89 50 4E 47 0D 0A 1A 0A
$38464947: Result := 'image/gif'; // 47 49 46 38
$002A4949, $2A004D4D, $2B004D4D:
Result := 'image/tiff'; // 49 49 2A 00 or 4D 4D 00 2A or 4D 4D 00 2B
$E011CFD0: // Microsoft Office applications D0 CF 11 E0 = DOCFILE
if Len>600 then
case PWordArray(Content)^[256] of // at offset 512
$A5EC: Result := 'application/msword'; // EC A5 C1 00
$FFFD: // FD FF FF
case PByteArray(Content)^[516] of
$0E,$1C,$43: Result := 'application/vnd.ms-powerpoint';
$10,$1F,$20,$22,$23,$28,$29: Result := 'application/vnd.ms-excel';
end;
end;
else
case PCardinal(Content)^ and $00ffffff of
$685A42: Result := 'application/bzip2'; // 42 5A 68
$088B1F: Result := 'application/gzip'; // 1F 8B 08
$492049: Result := 'image/tiff'; // 49 20 49
$FFD8FF: Result := 'image/jpeg'; // FF D8 FF DB/E0/E1/E2/E3/E8
else
case PWord(Content)^ of
$4D42: Result := 'image/bmp'; // 42 4D
end;
end;
end;
if (Result='') and (FileName<>'') then begin
case GetFileNameExtIndex(FileName,'png,gif,tiff,tif,jpg,jpeg,bmp,doc,docx') of
0: Result := 'image/png';
1: Result := 'image/gif';
2,3: Result := 'image/tiff';
4,5: Result := 'image/jpeg';
6: Result := 'image/bmp';
7,8: Result := 'application/msword';
else begin
Result := RawUTF8(ExtractFileExt(FileName));
if Result<>'' then begin
Result[1] := '/';
Result := 'application'+LowerCase(Result);
end;
end;
end;
end;
if Result='' then
Result := 'application/octet-stream';
end;
You can use a similar function, from the GAry Kessler's list.
There are lots of database engines with hundreds (if not thousands) of versions and formats. (Binary, CSV, XML...) Many of them are encrypted to protect the content. It is quite "impossible" to identify every database and every format and it is a subject of constant changes.
So first of all you have to limit your task to a list of database engines you want to scan. Thats what i would do...
First, I do not believe you could do more in a "quick scan" than provide a "possible format". Also, it's very difficult to imagine that any quick technique could be reliable.
DBASE files commonly use the extension .dbf. There are variants of the dBase file format used by FoxPro, and Clipper. Wikipedia documents these as xBase. Any dBase library that can open dBase files will also probably be able to (a) show that this is in fact a true dBase file by opening it, and (b) allow you to see which supported variants of the xBase file format are in use.
Access files are usually using the .mdb file format, but can be encrypted with a password. You could probably write your own library that could postiively identify the internal content as being of the "Jet database engine" (internal type of file used by Access) but not read the content, but I doubt that short of cracking the password, you could do this reliably.
FileMaker files can have many file extensions, and their internal file formats are not well documented. According to wikipedia, .fm .fp3 .fp5 and .fp7 are common file extensions. You will have similar "password" problems with filemaker databases, as with Access. I am not aware of any way to read filemaker files in delphi except through ODBC, and even then, I don't think you could provide an "omni-reader" in Delphi that was powered by ODBC, since ODBC requires careful setup and knowledge of the originating file into an odbc data source before it becomes readable through ODBC. Browse/Discovery is not a phase that is supported by ODBC.
SQLite files can have any file extension at all. The easiest way to try to detect it would be to load/open the file using SQLite and see if it opens.
The rest of the list is more or less infinite, and the technique would be the same. Just keep rolling more database engines and access layer libraries into your Katamari Damaci Database Detector Tool.
If you want to start with old database formats as you seem to be, I would investigate using BDE (ancient, but hey, you're talking about ancient stuff), plus ADO, to try to auto-detect and open files.