I’m trying to pull numbers out of a proprietary database (Lacerte tax software).
They are all whole numbers, stored in 4 bytes. I have put numbers into the program, and then checked out the file with a hex editor. This has allowed me to see how they are stored.
Here are some examples:
-100 = 00 00 59 C0
-4 = 00 00 10 C0
-3 = 00 00 08 C0
-2 = 00 00 00 C0
-1 = 00 00 F0 BF
0 = 00 00 00 00
1 = 00 00 F0 3F
2 = 00 00 00 40
3 = 00 00 08 40
4 = 00 00 10 40
5 = 00 00 14 40
6 = 00 00 18 40
7 = 00 00 1C 40
8 = 00 00 20 40
9 = 00 00 22 40
10 = 00 00 24 40
100 = 00 00 59 40
1,000,000 = 80 84 2E 41
Does anybody have any idea how to convert these hex numbers from the database into decimals?
Related
Given:
A machine running Debian 10 and using the french keyboard mapping.
A dump of keystrokes correspoding to "azerty" obtained using:
sudo cat /dev/input/by-id/usb-Logitech_USB_Keyboard-event-kbd > test_keylogger_azerty
The following code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <linux/types.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <xcb/xcb.h>
#include <xcb/xcb_keysyms.h>
#include <X11/Xlib.h>
#include <linux/input.h>
int main(int ac, char **av)
{
int fd, col;
xcb_connection_t *connection;
xcb_key_symbols_t *symbols;
struct input_event ev;
if (ac != 3) {
printf("Usage: %s <dump file> <symbol table index>\n", av[0]);
exit(EXIT_FAILURE);
}
connection = xcb_connect(NULL, NULL);
symbols = xcb_key_symbols_alloc(connection);
col = atoi(av[2]);
fd = open(av[1], O_RDONLY);
while (read(fd, (void *) &ev, sizeof(ev))) {
if (ev.type == EV_KEY && ev.value == 1)
printf("type: %x code: %x value: %2x => %s\n",
ev.type, ev.code, ev.value,
XKeysymToString(xcb_key_symbols_get_keysym(symbols, ev.code, col)));
}
printf("\n");
close(fd);
return(EXIT_SUCCESS);
}
A test run with xev tell me that: a == 24, b == 25 [...] y == 29.
Nevertheless, when i run my program, the obtained code don't match (and, as a result, the guessed corresponding symbol doesn't match too):
$ ./reverse_kl test_keylogger_azerty 0
type: 1 code: 10 value: 1 => egrave
type: 1 code: 11 value: 1 => underscore
type: 1 code: 12 value: 1 => ccedilla
type: 1 code: 13 value: 1 => agrave
type: 1 code: 14 value: 1 => parenright
type: 1 code: 15 value: 1 => equal
type: 1 code: 1c value: 1 => t
type: 1 code: 1d value: 1 => y
type: 1 code: 2e value: 1 => l
if i modify my code that way:
$ diff reverse_kl.c{.ref,}
33c33
< XKeysymToString(xcb_key_symbols_get_keysym(symbols, ev.code , col)));
---
> XKeysymToString(xcb_key_symbols_get_keysym(symbols, ev.code + 8 , col)));
It works better:
$ ./reverse_kl test_keylogger_azerty 0
type: 1 code: 10 value: 1 => a
type: 1 code: 11 value: 1 => z
type: 1 code: 12 value: 1 => e
type: 1 code: 13 value: 1 => r
type: 1 code: 14 value: 1 => t
type: 1 code: 15 value: 1 => y
type: 1 code: 1c value: 1 => Return
type: 1 code: 1d value: 1 => Control_L
type: 1 code: 2e value: 1 => c
Also, note that, for key 'a', i have the code 10 and adding 8 still don't give me 24 but it's working...
If anyone could help me understanding this strange behaviour ...
EDIT:
Added an hexadecimal dump of the trace as requested:
0000000 66 35 4d 5d 00 00 00 00 2b 7c 04 00 00 00 00 00
0000020 04 00 04 00 28 00 07 00 66 35 4d 5d 00 00 00 00
0000040 2b 7c 04 00 00 00 00 00 01 00 1c 00 00 00 00 00
0000060 66 35 4d 5d 00 00 00 00 2b 7c 04 00 00 00 00 00
0000100 00 00 00 00 00 00 00 00 66 35 4d 5d 00 00 00 00
0000120 a7 01 0e 00 00 00 00 00 04 00 04 00 14 00 07 00
0000140 66 35 4d 5d 00 00 00 00 a7 01 0e 00 00 00 00 00
0000160 01 00 10 00 01 00 00 00 66 35 4d 5d 00 00 00 00
0000200 a7 01 0e 00 00 00 00 00 00 00 00 00 00 00 00 00
0000220 67 35 4d 5d 00 00 00 00 22 11 01 00 00 00 00 00
0000240 04 00 04 00 14 00 07 00 67 35 4d 5d 00 00 00 00
0000260 22 11 01 00 00 00 00 00 01 00 10 00 00 00 00 00
0000300 67 35 4d 5d 00 00 00 00 22 11 01 00 00 00 00 00
0000320 00 00 00 00 00 00 00 00 67 35 4d 5d 00 00 00 00
0000340 fc fe 03 00 00 00 00 00 04 00 04 00 1a 00 07 00
0000360 67 35 4d 5d 00 00 00 00 fc fe 03 00 00 00 00 00
0000400 01 00 11 00 01 00 00 00 67 35 4d 5d 00 00 00 00
0000420 fc fe 03 00 00 00 00 00 00 00 00 00 00 00 00 00
0000440 67 35 4d 5d 00 00 00 00 ab 31 06 00 00 00 00 00
0000460 04 00 04 00 1a 00 07 00 67 35 4d 5d 00 00 00 00
0000500 ab 31 06 00 00 00 00 00 01 00 11 00 00 00 00 00
0000520 67 35 4d 5d 00 00 00 00 ab 31 06 00 00 00 00 00
0000540 00 00 00 00 00 00 00 00 67 35 4d 5d 00 00 00 00
0000560 1d e7 07 00 00 00 00 00 04 00 04 00 08 00 07 00
0000600 67 35 4d 5d 00 00 00 00 1d e7 07 00 00 00 00 00
0000620 01 00 12 00 01 00 00 00 67 35 4d 5d 00 00 00 00
0000640 1d e7 07 00 00 00 00 00 00 00 00 00 00 00 00 00
0000660 67 35 4d 5d 00 00 00 00 23 58 0a 00 00 00 00 00
0000700 04 00 04 00 08 00 07 00 67 35 4d 5d 00 00 00 00
0000720 23 58 0a 00 00 00 00 00 01 00 12 00 00 00 00 00
0000740 67 35 4d 5d 00 00 00 00 23 58 0a 00 00 00 00 00
0000760 00 00 00 00 00 00 00 00 67 35 4d 5d 00 00 00 00
As a reminder, below is the format of the input_event structure:
struct input_event {
struct timeval time;
__u16 type;
__u16 code;
__s32 value;
};
And, other reminder, keyboard stroke obey the following rules:
type == EV_KEY == 0x01
value may be 0 for event "key released", 1 for "key pressed" and 2 is for "auto-repeat".
Here is my hex code:
42 4D C6 00 00 00 00 00 00 00 76 00 00 00 28 00
00 00 0A 00 00 00 0A 00 00 00 01 00 04 00 00 00
00 00 50 00 00 00 12 0B 00 00 12 0B 00 00 10 00
00 00 10 00 00 00 FF 00 00 00 00 FF 00 00 00 00
42 00 5A 5A 84 00 00 00 FF 00 FF 00 FF 00 00 FF
FF 00 08 FF FF 00 5A FF FF 00 FF FF FF 00 FF FF
FF 00 FF FF FF 00 FF FF FF 00 FF FF FF 00 FF FF
FF 00 FF FF FF 00 92 59 00 16 47 00 00 00 25 90
01 64 61 00 00 00 59 90 11 64 61 00 00 00 99 00
16 48 11 00 00 00 90 01 64 61 11 00 00 00 00 16
64 61 00 00 00 00 01 16 46 10 09 00 00 00 11 64
41 00 99 00 00 00 16 64 11 09 95 00 00 00 66 48
10 09 53 00 00 00
I know that the pixel "assignment" starts with the first line being (10 pixels wide):
92 59 00 16 47 00 00 00
I need to count how many times each colour is in the image, but I am unable to pull the individual integer value (ie: just the 9, then just the 2, then just the 5, and so on). The only value I am able to pull is "92" then "59" then "00"...
This is my code for that segment (the offset is 118 and the total hex values remaining are 80):
int nbr_each[NBRCOLOURS];
int ch, pixel;
fseek(fptr, 118, SEEK_SET);
for (count = 0; count < 81; count++)
{
pixel = fgetc(fptr);
nbr_each[pixel] = nbr_each[pixel] + 1;
}
fgetc will get you the individual characters.
first = fgetc(fptr); // '9'
second = fgetc(fptr); // '2'
space = fgetc(fptr); // ' '
Then convert each digit to a number 0..9 by subtracting off '0':
first -= '0';
second -= '0';
Then to count each digit, something like this:
nbr_each[first]++;
nbr_each[second]++;
I'm trying to connect to SQL Server with Windows Authentication. Microsoft C# and C sources can be found at NetCpp.
In Delphi I have code like this:
function TTDS7SSPI.MakeSPN: string;
const
szBracketedInstanceFormatString = '%s/[%s]:%s';
szBracketedEmptyInstanceFormatString = '%s/[%s]%s';
szClearInstanceFormatString = '%s/%s:%s';
szClearEmptyInstanceFormatString = '%s/%s%s';
szBracketedFormatString = '%s/[%s]:%d';
szClearFormatString = '%s/%s:%d';
var
NeedBrackets: Boolean;
FmtString: string;
begin
NeedBrackets := Pos(':', FHostName) > 0;
if FInstanceName <> '' then begin
// Make an instance name based SPN, i.e. MSSQLSvc/FQDN:instancename
if NeedBrackets then begin
if FInstanceName = '' then
FmtString := szBracketedEmptyInstanceFormatString
else
FmtString := szBracketedInstanceFormatString;
end
else begin
if FInstanceName = '' then
FmtString := szClearEmptyInstanceFormatString
else
FmtString := szClearInstanceFormatString;
end;
Result := Format(FmtString, [SQL_SERVICECLASS, FHostName, FInstanceName]);
end
else begin
// Make a TCP port based SPN, i.e. MSSQLSvc/FQDN:TcpPort
Assert(FPort > 0);
if NeedBrackets then
FmtString := szBracketedFormatString
else
FmtString := szClearFormatString;
Result := Format(FmtString, [SQL_SERVICECLASS, FHostName, FPort]);
end;
end;
function TTDS7SSPI.GetAuth: TBytes;
var
pkgInfo: PSecPkgInfo;
SecBuf: SecBuffer;
BuffDesc: SecBufferDesc;
status: SECURITY_STATUS;
attrs: Cardinal;
tsExpiry: TTimeStamp;
const
NEG_STR: WideString = 'Negotiate'; // 'NTLM'; // 'Kerberos';
begin
Result := nil;
status := QuerySecurityPackageInfo({$IFDEF FPC}PSecChar{$ELSE}PSecWChar{$ENDIF}(NEG_STR), pkgInfo);
if status <> SEC_E_OK then
raise Exception.CreateFmt('Couldn''t query package info for %s, error %X', [NEG_STR, status]);
FMaxMessageLen := pkgInfo.cbMaxToken; // 4096;
FreeContextBuffer(pkgInfo);
TTimeStamp(tsExpiry).QuadPart := 0;
status := AcquireCredentialsHandle(nil, {$IFDEF FPC}PSecChar{$ELSE}PSecWChar{$ENDIF}(NEG_STR), SECPKG_CRED_BOTH, // SECPKG_CRED_OUTBOUND
nil, nil, nil, nil, #FCred, tsExpiry); // tsExpiry as var parameter
if status <> SEC_E_OK then
raise Exception.CreateFmt('AcquireCredentialsHandle error %X', [status]);
BuffDesc.ulVersion := SECBUFFER_VERSION;
BuffDesc.cBuffers := 1;
BuffDesc.pBuffers := #SecBuf;
SecBuf.BufferType := SECBUFFER_TOKEN;
SetLength(Result, FMaxMessageLen);
SecBuf.pvBuffer := #Result[0];
SecBuf.cbBuffer := FMaxMessageLen;
{status := QueryCredentialsAttributes(#FCred, SECPKG_CRED_ATTR_NAMES, #attrName);
if status = SEC_E_OK then
FSPN := PWideChar(attrName.sUserName)
else}
// For DAC use "localhost" instead of the server name (Microsoft)
FSPN := WideString(MakeSPN);
FContextAttrib := ISC_REQ_DELEGATE or ISC_REQ_MUTUAL_AUTH or ISC_REQ_INTEGRITY or ISC_REQ_EXTENDED_ERROR;
// ISC_REQ_CONFIDENTIALITY or ISC_REQ_REPLAY_DETECT or ISC_REQ_CONNECTION;
// $8C03C;
// ISC_REQ_MUTUAL_AUTH or ISC_REQ_IDENTIFY or ISC_REQ_CONFIDENTIALITY or ISC_REQ_REPLAY_DETECT or ISC_REQ_SEQUENCE_DETECT or ISC_REQ_CONNECTION or ISC_REQ_DELEGATE;
status := InitializeSecurityContext(#FCred, nil, {$IFDEF FPC}PSecChar{$ELSE}PSecWChar{$ENDIF}(FSPN),
FContextAttrib,
0, SECURITY_NATIVE_DREP, nil, 0, #FCredCtx, #BuffDesc, attrs, #tsExpiry);
if status <= 0 then
raise Exception.CreateFmt('InitializeSecurityContext error %X', [status]);
if (status = SEC_I_COMPLETE_NEEDED) or (status = SEC_I_COMPLETE_AND_CONTINUE) {or (status = SEC_I_CONTINUE_NEEDED)} then begin
status := CompleteAuthToken(#FCredCtx, #BuffDesc);
if status <> SEC_E_OK then begin
FreeCredentialsHandle(#FCred);
Result := nil;
raise Exception.CreateFmt('CompleteAuthToken error %X', [status]);
end;
end
else if (status <> SEC_E_OK) and (status <> SEC_I_CONTINUE_NEEDED) then begin
// SEC_I_CONTINUE_NEEDED
// The client must send the output token to the server and wait for a return token.
// The returned token is then passed in another call to InitializeSecurityContext (Negotiate). The output token can be empty
FreeCredentialsHandle(#FCred);
Result := nil;
raise Exception.CreateFmt('InitializeSecurityContext error %X', [status]);
end;
SetLength(Result, SecBuf.cbBuffer);
end;
function TTDS7SSPI.ParseServerResponse(Buf: TBytes): TBytes;
var
InSecBuff, OutSecBuff: SecBuffer;
InBuffDesc, OutBuffDesc: SecBufferDesc;
status: SECURITY_STATUS;
attrs: Cardinal;
tsExpiry: TTimeStamp;
begin
Assert((Length(Buf) >= 32) or (Length(Buf) <= Integer(FMaxMessageLen)));
InBuffDesc.ulVersion := SECBUFFER_VERSION;
InBuffDesc.cBuffers := 1;
InBuffDesc.pBuffers := #InSecBuff;
OutBuffDesc.ulVersion := SECBUFFER_VERSION;
OutBuffDesc.cBuffers := 1;
OutBuffDesc.pBuffers := #OutSecBuff;
Assert(Length(Buf) > 0);
InSecBuff.BufferType := SECBUFFER_TOKEN;
InSecBuff.pvBuffer := #Buf[0];
InSecBuff.cbBuffer := Length(Buf);
OutSecBuff.BufferType := SECBUFFER_TOKEN;
SetLength(Result, FMaxMessageLen);
OutSecBuff.pvBuffer := #Result[0];
OutSecBuff.cbBuffer := Length(Result);
status := InitializeSecurityContext(#FCred, #FCredCtx, {$IFDEF FPC}PSecChar{$ELSE}PSecWChar{$ENDIF}(FSPN),
FContextAttrib,
0, SECURITY_NATIVE_DREP, #InBuffDesc, 0, #FCredCtx, #OutBuffDesc, attrs, #tsExpiry);
if status <> SEC_E_OK then begin
Result := nil;
raise Exception.CreateFmt('InitializeSecurityContext error %X', [status]);
end
else
SetLength(Result, OutSecBuff.cbBuffer);
end;
The SPN I got is like MSSQLSvc/3R-XP:MSSQL2008 (client and server both on 3R-XP, instance MSSQL2008). InitializeSecurityContext has status SEC_I_CONTINUE_NEEDED. Everything works without errors except that the server does not return any of the rows from the query, only TDS_DONE.
The SQL Server log says:
Login succeeded for user '3R-XP\me'. Connection made using Windows authentication. [CLIENT: 192.168.0.100]
Also I tried to compare OLEDB and mine data sent and received. I can't see the first packet sent by OLEDB due to SSL encryption. Mine SSPI login data
4E 54 4C 4D 53 53 | NTLMSS
50 00 01 00 00 00 97 B2 08 E2 09 00 09 00 2D 00 | P.............-.
00 00 05 00 05 00 28 00 00 00 05 01 28 0A 00 00 | ......(.....(...
00 0F 33 52 2D 58 50 57 4F 52 4B 47 52 4F 55 50 | ..3R-XPWORKGROUP
The server response of OLEDB (connect to another PC due to the fact that WinPCAP can only work with real adapters, so the host name is 'hp-6320' and the client name is '3R-Win7' here) is:
000000 04 01 00 A5 00 00 01 00 ED 9A 00 4E 54 4C 4D 53 | ...........NTLMS
000010 53 50 00 02 00 00 00 0E 00 0E 00 38 00 00 00 15 | SP.........8....
000020 82 8A E2 A3 6E FC 4B 59 86 13 D6 00 00 00 00 00 | ....n.KY........
000030 00 00 00 54 00 54 00 46 00 00 00 05 01 28 0A 00 | ...T.T.F.....(..
000040 00 00 0F 48 00 50 00 2D 00 36 00 33 00 32 00 30 | ...H.P.-.6.3.2.0
000050 00 02 00 0E 00 48 00 50 00 2D 00 36 00 33 00 32 | .....H.P.-.6.3.2
000060 00 30 00 01 00 0E 00 48 00 50 00 2D 00 36 00 33 | .0.....H.P.-.6.3
000070 00 32 00 30 00 04 00 0E 00 68 00 70 00 2D 00 36 | .2.0.....h.p.-.6
000080 00 33 00 32 00 30 00 03 00 0E 00 68 00 70 00 2D | .3.2.0.....h.p.-
000090 00 36 00 33 00 32 00 30 00 06 00 04 00 01 00 00 | .6.3.2.0........
0000A0 00 00 00 00 00 | .....
SQL Server response with my code (machine '3R-XP')
04 01 00 89 00 00 01 00 | ........
ED 7E 00 4E 54 4C 4D 53 53 50 00 02 00 00 00 0A | .~.NTLMSSP......
00 0A 00 38 00 00 00 15 C2 8A E2 B0 17 7A 15 A4 | ...8.........z..
21 2A 96 38 E6 3D 01 00 00 00 00 3C 00 3C 00 42 | !*.8.=.....<.<.B
00 00 00 05 01 28 0A 00 00 00 0F 33 00 52 00 2D | .....(.....3.R.-
00 58 00 50 00 02 00 0A 00 33 00 52 00 2D 00 58 | .X.P.....3.R.-.X
00 50 00 01 00 0A 00 33 00 52 00 2D 00 58 00 50 | .P.....3.R.-.X.P
00 04 00 0A 00 33 00 52 00 2D 00 58 00 50 00 03 | .....3.R.-.X.P..
00 0A 00 33 00 52 00 2D 00 58 00 50 00 00 00 00 | ...3.R.-.X.P....
00 | .
It looks the same. But after that second InitializeSecurityContext OLEDB returns the value
000000 11 01 01 A2 00 00 01 00 4E 54 4C 4D 53 53 50 00 | ........NTLMSSP.
000010 03 00 00 00 18 00 18 00 78 00 00 00 FA 00 FA 00 | ........x.......
000020 90 00 00 00 0E 00 0E 00 58 00 00 00 04 00 04 00 | ........X.......
000030 66 00 00 00 0E 00 0E 00 6A 00 00 00 10 00 10 00 | f.......j.......
000040 8A 01 00 00 15 82 88 E2 06 01 B1 1D 00 00 00 0F | ................
000050 18 B1 57 6E 0F 9B BE 6A AF 2A D4 76 8D B2 19 72 | ..Wn...j.*.v...r
000060 33 00 52 00 2D 00 57 00 69 00 6E 00 37 00 6D 00 | 3.R.-.W.i.n.7.m.
000070 65 00 33 00 52 00 2D 00 57 00 49 00 4E 00 37 00 | e.3.R.-.W.I.N.7.
000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
000090 00 00 00 00 00 00 00 00 3B 97 82 77 95 74 1E 7C | ........;..w.t.|
0000A0 A8 D1 C5 2F 5F 82 7A 9C 01 01 00 00 00 00 00 00 | .../_.z.........
0000B0 EE 4C 92 1E 68 10 D1 01 B3 93 23 3B A9 14 0C EF | .L..h.....#;....
0000C0 00 00 00 00 02 00 0E 00 48 00 50 00 2D 00 36 00 | ........H.P.-.6.
0000D0 33 00 32 00 30 00 01 00 0E 00 48 00 50 00 2D 00 | 3.2.0.....H.P.-.
0000E0 36 00 33 00 32 00 30 00 04 00 0E 00 68 00 70 00 | 6.3.2.0.....h.p.
0000F0 2D 00 36 00 33 00 32 00 30 00 03 00 0E 00 68 00 | -.6.3.2.0.....h.
000100 70 00 2D 00 36 00 33 00 32 00 30 00 06 00 04 00 | p.-.6.3.2.0.....
000110 01 00 00 00 08 00 30 00 30 00 00 00 00 00 00 00 | ......0.0.......
000120 01 00 00 00 00 20 00 00 9B 51 53 D8 0E 0F C8 EB | ..... ...QS.....
000130 F9 11 AB 3D B3 FB 86 F6 D0 D2 97 3C 4C F7 E0 48 | ...=.......<L..H
000140 C4 BF 2F 60 DC CA AB 10 0A 00 10 00 14 5E 11 19 | ../`.........^..
000150 42 DC 79 32 B1 DC 04 C0 C9 48 8D 2C 09 00 2A 00 | B.y2.....H.,..*.
000160 4D 00 53 00 53 00 51 00 4C 00 53 00 76 00 63 00 | M.S.S.Q.L.S.v.c.
000170 2F 00 68 00 70 00 2D 00 36 00 33 00 32 00 30 00 | /.h.p.-.6.3.2.0.
000180 3A 00 31 00 34 00 33 00 33 00 00 00 00 00 00 00 | :.1.4.3.3.......
000190 00 00 7D 45 28 4F E6 4B 38 90 BD F6 91 61 A7 E8 | ..}E(O.K8....a..
0001A0 8D 26 | .&
while for my code it returns
11 01 00 50 00 00 00 00 4E 54 4C 4D 53 53 50 00 | ...P....NTLMSSP.
03 00 00 00 00 00 00 00 48 00 00 00 00 00 00 00 | ........H.......
48 00 00 00 00 00 00 00 48 00 00 00 00 00 00 00 | H.......H.......
48 00 00 00 00 00 00 00 48 00 00 00 00 00 00 00 | H.......H.......
48 00 00 00 15 C2 88 E2 05 01 28 0A 00 00 00 0F | H.........(.....
As you can see all structures are empty (size 0, allocated 0, offset 48). Is there something wrong? How to fix that stuff? I've tried different flags etc already, the results are the same or even worse. OLEDB works so it seems that the server is configured properly.
With WinAPIOverride I found that authentication uses Bindings (see QueryContextAttributes SECPKG_ATTR_UNIQUE_BINDINGS) retrieved from SSL handshake in negotiation InitializeSecurityContext as SECBUFFER_CHANNEL_BINDINGS member.
So far I made SSPI based SSL handshake, got Bindings that looks like
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00 00 00 00 00 00 00 00 17 00 00 00 20 00 00 00 | ............ ...
74 6C 73 2D 75 6E 69 71 75 65 3A 66 55 6F 05 7F | tls-unique:fUo.
DD 90 31 4F 87 02 52 | ..1O..R
found that those empty NTLMSSP message seems proper (with some extra at the end) while client and server on same machine, ODBC driver sends like
A1 77 30 75 A0 03 0A 01 01 A2 5A 04 58 4E 54 4C .w0u......Z.XNTL
4D 53 53 50 00 03 00 00 00 00 00 00 00 58 00 00 MSSP.........X..
00 00 00 00 00 58 00 00 00 00 00 00 00 58 00 00 .....X.......X..
00 00 00 00 00 58 00 00 00 00 00 00 00 58 00 00 .....X.......X..
00 00 00 00 00 58 00 00 00 15 C2 88 E2 0A 00 5A .....X.........Z
29 00 00 00 0F 9E 3F 5C EE FF F1 AF 9A 44 4C 3A ).....?\.....DL:
6F C3 20 0F 8B A3 12 04 10 01 00 00 00 9C B1 60 o. ............`
36 3B 84 96 09 00 00 00 00 6;.......
remote last authentication data looks like (ODBC driver)
4E 54 4C 4D 53 53 50 00 03 00 00 00 18 00 18 00 NTLMSSP.........
78 00 00 00 3C 01 3C 01 90 00 00 00 0E 00 0E 00 x...<.<.........
58 00 00 00 04 00 04 00 66 00 00 00 0E 00 0E 00 X.......f.......
6A 00 00 00 10 00 10 00 CC 01 00 00 15 82 88 E2 j...............
0A 00 5A 29 00 00 00 0F E0 87 5F 85 21 5A 73 17 ..Z)......_.!Zs.
04 6C 1A F5 9C BA F7 42 33 00 52 00 2D 00 57 00 .l.....B3.R.-.W.
69 00 6E 00 37 00 6D 00 65 00 33 00 52 00 2D 00 i.n.7.m.e.3.R.-.
57 00 49 00 4E 00 37 00 00 00 00 00 00 00 00 00 W.I.N.7.........
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
37 F2 38 62 5B 9C 7E 07 6F 89 9F 33 B2 92 3C 5C 7.8b[.~.o..3..<\
01 01 00 00 00 00 00 00 47 39 01 AD AC 4F D1 01 ........G9...O..
0D 36 47 06 7E 70 B8 A4 00 00 00 00 02 00 18 00 .6G.~p..........
48 00 50 00 2D 00 45 00 4C 00 49 00 54 00 45 00 H.P.-.E.L.I.T.E.
42 00 4F 00 4F 00 4B 00 01 00 18 00 B.O.O.K.....
while SSPI with Bindings looks slightly bigger (8 first bytes here is TDS packet header)
11 01 01 EE 00 00 00 00 4E 54 4C 4D 53 53 50 00 | ........NTLMSSP.
03 00 00 00 18 00 18 00 78 00 00 00 46 01 46 01 | ........x...F.F.
90 00 00 00 0E 00 0E 00 58 00 00 00 04 00 04 00 | ........X.......
66 00 00 00 0E 00 0E 00 6A 00 00 00 10 00 10 00 | f.......j.......
D6 01 00 00 15 82 88 E2 0A 00 5A 29 00 00 00 0F | ..........Z)....
AD A5 C9 05 8C 25 E1 A9 C5 3E 17 BD 3D 19 E3 EB | .....%...>..=...
33 00 52 00 2D 00 57 00 69 00 6E 00 37 00 6D 00 | 3.R.-.W.i.n.7.m.
65 00 33 00 52 00 2D 00 57 00 49 00 4E 00 37 00 | e.3.R.-.W.I.N.7.
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
00 00 00 00 00 00 00 00 02 0B C5 A2 01 17 DB AC | ................
D8 26 9E 1B AF A1 77 32 01 01 00 00 00 00 00 00 | .&....w2........
B8 28 90 0F AD 4F D1 01 41 F1 DF 7C BE 85 5D B6 | .(...O..A..|..].
00 00 00 00 02 00 18 00 48 00 50 00 2D 00 45 00 | ........H.P.-.E.
4C 00 49 00 54 00 45 00 42 00 4F 00 4F 00 4B 00 | L.I.T.E.B.O.O.K.
01 00 18 00 48 00 50 00 2D 00 45 00 4C 00 49 00 | ....H.P.-.E.L.I.
54 00 45 00 42 00 4F 00 4F 00 4B 00 04 00 18 00 | T.E.B.O.O.K.....
48 00 50 00 2D 00 45 00 6C 00 69 00 74 00 65 00 | H.P.-.E.l.i.t.e.
62 00 6F 00 6F 00 6B 00 03 00 18 00 48 00 50 00 | b.o.o.k.....H.P.
2D 00 45 00 6C 00 69 00 74 00 65 00 62 00 6F 00 | -.E.l.i.t.e.b.o.
6F 00 6B 00 07 00 08 00 B8 28 90 0F AD 4F D1 01 | o.k......(...O..
06 00 04 00 02 00 00 00 08 00 30 00 30 00 00 00 | ..........0.0...
00 00 00 00 01 00 00 00 00 20 00 00 DC 75 9C 98 | ......... ...u..
70 C7 28 D7 BC C7 1E 14 48 70 0E 3B 8B A4 94 7C | p.(.....Hp.;...|
32 05 44 FD 85 5F D3 54 DB 6C 84 22 0A 00 10 00 | 2.D.._.T.l."....
B1 3B 92 CC 6C 5B E2 CD 0F 24 19 5F 6F 73 47 73 | .;..l[...$._osGs
09 00 3E 00 4D 00 53 00 53 00 51 00 4C 00 53 00 | ..>.M.S.S.Q.L.S.
76 00 63 00 2F 00 48 00 50 00 2D 00 45 00 4C 00 | v.c./.H.P.-.E.L.
49 00 54 00 45 00 42 00 4F 00 4F 00 4B 00 3A 00 | I.T.E.B.O.O.K.:.
4D 00 53 00 53 00 51 00 4C 00 32 00 30 00 30 00 | M.S.S.Q.L.2.0.0.
38 00 00 00 00 00 00 00 00 00 00 00 00 00 F0 46 | 8..............F
20 EB 45 EC C8 67 9F E3 45 45 9C 79 76 47 | .E..g..EE.yvG
QueryContextAttributes(#FCtxHandle, SECPKG_ATTR_NEGOTIATION_INFO, #NegInfo) returns state SECPKG_NEGOTIATION_COMPLETE so everything suppose to be fine, server log shows that "Authentication successful" but there is still not enough rights to get results of queries or server errors like "Cannot find the object "all_types" because it does not exist or you do not have permissions" while simple queries like "SET LOCK TIMEOUT 100" runs without errors.
So my thoughts that Windows Authentication in the eyes of own creator doesn't looks secure enough to allow it to some third-party applications. Guest account enabled and have permissions to read/write data and it works through ODBC driver.
I have been using SQL for like 10 years but now I realised I never knew how the client actually receive and process data it gets from the server.
My question is, how does the result from Microsoft SQL Server actually look like in raw format? The same as that result from HTTP server contains HTTP headers and a Content-Type header to tell what the body format is (mostly HTML for web pages).
The protocol name is TDS (Tabular Data Stream).
Some documentation is available at MSDN.
There is a very basic example of data being transferred for simple query select 'foo' as 'bar'
Request
Packet header (type, legth, etc)
01 01 00 5C 00 00 01 00
Packet data
16 00 00 00 - headers total length
12 00 00 00 - first header length
02 00 - type
00 00 00 00 00 00 00 01 00 00 00 00 - data
0A 00 73 00 65 00 6C 00 65 00 63 00 74 00 20 00
27 00 66 00 6F 00 6F 00 27 00 20 00 61 00 73 00
20 00 27 00 62 00 61 00 72 00 27 00 0A 00 20 00
20 00 20 00 20 00 20 00 20 00 20 00 20 00 - sql
Response
Packet header (type, legth, etc)
04 01 00 33 00 00 01 00
Packet data
columns metadata
81 - record id
01 - count
first column
00 00 00 00 00 - user type
20 00 - flags
A7 - type
03 00 - length
09 04 D0 00 34 - colation
03 - column name length
62 00 61 00 72 00 - column name bytes
rows
D1 - record id
03 00 - length
66 6F 6F - value
ending data
FD - record id
10 00 - status
C1 00
01 00 00 00 00 00 00 00 - rows total
We can also look at parser implementation thanks to reference sources.
and can it be configured not to happen?
I'm usually finding myself saving a result of a query as a .csv and processing it later on my Unix machine. The characters being null separated makes me have to filter those chars and is a bit of a pain.
So, these are the questions:
Why is this so?
EDIT:
Because it outputs in UTF-16 by default. Easiest conversion would then be:
iconv -f utf-16 -t utf-8 origFile.csv > newFile.csv
Can it be disabled somehow? How?
Here's a piece of a hexdump of a file thus generated. Each char is followed by a
null char (00):
00000cf0 36 00 36 00 32 00 0d 00 0a 00 36 00 38 00 34 00 |6.6.2.....6.8.4.|
00000d00 30 00 36 00 32 00 31 00 36 00 0d 00 0a 00 36 00 |0.6.2.1.6.....6.|
00000d10 38 00 34 00 30 00 36 00 33 00 36 00 34 00 0d 00 |8.4.0.6.3.6.4...|
00000d20 0a 00 36 00 38 00 34 00 30 00 36 00 38 00 34 00 |..6.8.4.0.6.8.4.|
00000d30 32 00 0d 00 0a 00 36 00 38 00 34 00 30 00 37 00 |2.....6.8.4.0.7.|
00000d40 30 00 32 00 31 00 0d 00 0a 00 36 00 38 00 34 00 |0.2.1.....6.8.4.|
00000d50 30 00 37 00 37 00 39 00 37 00 0d 00 0a 00 36 00 |0.7.7.9.7.....6.|
00000d60 38 00 34 00 30 00 37 00 39 00 32 00 31 00 0d 00 |8.4.0.7.9.2.1...|
00000d70 0a 00 36 00 38 00 34 00 30 00 38 00 32 00 34 00 |..6.8.4.0.8.2.4.|
00000d80 31 00 0d 00 0a 00 36 00 38 00 34 00 30 00 38 00 |1.....6.8.4.0.8.|
00000d90 36 00 36 00 31 00 0d 00 0a 00 36 00 38 00 34 00 |6.6.1.....6.8.4.|
00000da0 30 00 38 00 37 00 35 00 31 00 0d 00 0a 00 36 00 |0.8.7.5.1.....6.|
00000db0 38 00 34 00 31 00 30 00 32 00 35 00 34 00 0d 00 |8.4.1.0.2.5.4...|
00000dc0 0a 00 36 00 38 00 34 00 31 00 30 00 34 00 34 00 |..6.8.4.1.0.4.4.|
The file is being outputted in Unicode, not ASCII. Unicode uses twice as many bits to represent each character, hence the preceding 00's.
There might be an option to save as ANSI or ASCII, which should use 8 bit characters.
I know this is an old post...but for new visitors...
When you are saving data from Microsoft SQL Management Studio, you will notice that the 'Save' button has a little arrow next to it. If you select the little arrow you can select 'Save With Encoding...' this will allow you to select the encoding you desire.
On Unix, I suggest the use of iconv -futf-16le -tutf-8 to filter your output. :-)