A few weeks ago I began experimenting with smart cards for an university project. The exercise was as easy as using a health insurance card (the German one, "eGK") and find out what's saved on it and read out the "public" parts which can be read out without prior authenticating in any way.
Fourtunately the structure of the card is well documented and the documentation can easily be downloaded from the internet. What I basically want to to now is selecting the EF (MF / DF.HCA / EF.VD) and read out the data. The file I want to read out is a binary file. According to the documentation, the first 2x4 bytes represent the start/end-offsets of two parts of the file. Each part should be gzip-compressed XML.
At first, I used "scriptor" from the pcsc-lite tools to try out a few commands, to see if it works and if I get some reasonable results. This all worked out pretty well :)
$ scriptor
No reader given: using Towitoko Chipdrive USB 00 00
Using T=1 protocol
Reading commands from STDIN
reset
> RESET
< OK: 3B DD 97 FF 81 B1 FE 45 1F 03 00 64 04 05 08 03 73 96 21 D0 00 90 00 C8
00 a4 04 0c 06 d2 76 00 00 01 02 # Select DF
> 00 a4 04 0c 06 d2 76 00 00 01 02
< 90 00 : Normal processing.
00 a4 02 0c 02 d0 02 # Select EF
> 00 a4 02 0c 02 d0 02
< 90 00 : Normal processing.
00 b0 00 00 08 # Read out start/end offsets
> 00 b0 00 00 08
< 00 08 01 AD 01 AE 02 7C 90 00 : Normal processing.
As you can see, this would be over 400 Bytes to read out (and decompress them afterwards), so I decided I need to write some program which reads this out for me. With the help of this blogpost: http://ludovicrousseau.blogspot.de/2010/04/pcsc-sample-in-c.html I was quickly able to send my first commands to the card using C. The basic flow of my program is:
Establish a context
Get a reader (the program prints out the reader in use and it's the only and the right one ;) )
Get the card (protocol is T=1)
Reset the card
Tell PC/SC I need a transaction
Transmit the commands
End transaction
Here are the fields I send to the SC:
// CLA INS P1 P2 LEN_SEND
BYTE cmdSelectDF[] = {0x00, 0xA4, 0x04, 0x0C, 0x06,
// 00 SELECT DF/AID first/noanswer
// <- HEADER] ----- [DATA ->
// D6 D5 D4 D3 D2 D1
0xD2, 0x76, 0x00, 0x00, 0x01, 0x02};
// CLA INS P1 P2 LEN_SEND
BYTE cmdSelectEF[] = {0x00, 0xA4, 0x02, 0x0C, 0x02,
// 00 SELECT EF/FID first/noanswer
// <- HEADER] ----- [DATA ->
// D2 D1
0xD0, 0x02};
// CLA INS P1 P2 LEN_RECV
BYTE cmdReadOffsets[] = {0x00, 0xB0, 0x00, 0x00, 0x08};
// 00 READ BINARY OFFSET1 OFFSET2 BYTES
The first two commands transmit just fine and I always get a 0x90 0x00 back. Unfortunately, the ReadOffsets-Transmit returns with an error: "Transaction failed". The log says the following:
00000011 winscard.c:1613:SCardTransmit() Send Protocol: T=1
00042572 winscard.c:1658:SCardTransmit() UnrefReader() count was: 2
00000030 winscard_svc.c:656:ContextThread() TRANSMIT rv=0x0 for client 6
03000349 winscard_svc.c:356:ContextThread() Received command: TRANSMIT from client 6
00000043 readerfactory.c:798:RFReaderInfoById() RefReader() count was: 1
00000012 winscard.c:1613:SCardTransmit() Send Protocol: T=1
00055517 ifdwrapper.c:553:IFDTransmit() Card not transacted: 612
00000027 winscard.c:1638:SCardTransmit() Card not transacted: 0x80100016
00000010 winscard.c:1658:SCardTransmit() UnrefReader() count was: 2
00000031 winscard_svc.c:656:ContextThread() TRANSMIT rv=0x80100016 for client 6
00000297 winscard_svc.c:348:ContextThread() Client die: 6
00000029 winscard.c:230:SCardReleaseContext() Releasing Context: 0x420027B7
Does anyone have an idea what this means and why it happens? What am I doing wrong?
Thank you very much in advance!
Edit: I have some news for you. I found some #ifdefs in the towitoko driver source code for debugging purposes. Now, I have a much more detailed log file for you. The parts with [[[]]] are only in the error log file. As you can see, there isn't any difference in the bytes submitted and the bytes received at all!
01942065 winscard_svc.c:356:ContextThread() Received command: TRANSMIT from client 6
00000021 readerfactory.c:798:RFReaderInfoById() RefReader() count was: 1
00000008 winscard.c:1613:SCardTransmit() Send Protocol: T=1
IFD: Setting baudrate to 9600
IFD: Transmit: 0 40 7 0 A4 2 C 2 D0 2 3D
IO: Sending: 6F B 5 5A
IO: Sending: 0
IO: Sending: 40 7 0 A4 2 C 2 D0 2 3D
IO: Receiving: 0
IO: Receiving: 40 2 90
IFD: Receive: 0 40 2 90
IO: Receiving: 0 D2
IFD: Receive: 0 D2
00042861 winscard.c:1658:SCardTransmit() UnrefReader() count was: 2
00000011 winscard_svc.c:656:ContextThread() TRANSMIT rv=0x0 for client 6
IO: Sending: 3 7
IO: Receiving: 42 87
IFD: Status = card / no change
IO: Sending: 3 7
IO: Receiving: 40 83
IFD: Status = card / no change
[...]
02773187 winscard_svc.c:356:ContextThread() Received command: TRANSMIT from client 6
00000032 readerfactory.c:798:RFReaderInfoById() RefReader() count was: 1
00000009 winscard.c:1613:SCardTransmit() Send Protocol: T=1
IFD: Setting baudrate to 9600
IFD: Transmit: 0 0 5 0 B0 0 0 8 BD
IO: Sending: 6F 9 5 52
IO: Sending: 0
IO: Sending: 0 5 0 B0 0 0 8 BD
IO: Receiving: 0
IO: Receiving: 0 A 0
IFD: Receive: 0 0 A 0
IO: Receiving: 8 1 AD 1 AE 2 7C 90 0 EF
IFD: Receive: 8 1 AD 1 AE 2 7C 90 0 EF
[[[ 00054674 ifdwrapper.c:553:IFDTransmit() Card not transacted: 612
00000011 winscard.c:1638:SCardTransmit() Card not transacted: 0x80100016 ]]]
00054759 winscard.c:1658:SCardTransmit() UnrefReader() count was: 2
00000011 winscard_svc.c:656:ContextThread() TRANSMIT rv=0x0 for client 6
So, where does this error message come from?
Another thing I've learned is that this error doesn't seem to be reader specific. I've tried the program on a friend's internal card reader (Windows says it's a Broadcom reader), which yields to the same results. Because the only case I can think of is an error in my C program, here's the source code with the basic macros and the connection part:
// Parts taken from http://ludovicrousseau.blogspot.de/2010/04/pcsc-sample-in-c.html
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <PCSC/pcsclite.h>
#include <PCSC/winscard.h>
#include <PCSC/wintypes.h>
#define EGK_RECV_BUFSIZE 258
#define PCSC_ERROR(debugmsg, retval) \
if(retval != SCARD_S_SUCCESS) { \
fprintf(stderr, "PC/SC Error at %s: %s\n", debugmsg, pcsc_stringify_error(retval)); \
return 1; \
}
#define TRANS_RESPONSE \
printf("Command response: "); \
for(i = 0; i < dwRecvLen; i++) { \
printf("%02X ", recvBuffer[i]); \
} printf("\n");
#define CHECK_SUCCESS \
if(dwRecvLen >= 2) { \
if(recvBuffer[dwRecvLen-2] == 0x90 && recvBuffer[dwRecvLen-1] == 0x00) { \
printf("Command success!\n"); \
} \
}
#define SCARD_TRANSMIT(cmd) \
SCardTransmit(scHandle, &scSendProto, cmd, sizeof(cmd), NULL, recvBuffer, &dwRecvLen);
int main(void)
{
LONG retval = 0; // = long
SCARDCONTEXT scContext; // = LONG
LPTSTR scReaders; // = LPSTR = char *
SCARDHANDLE scHandle; // = LONG (specific smartcard)
DWORD dwReaders, dwCurProto, dwRecvLen; // = unsigned long
SCARD_IO_REQUEST scSendProto;
BYTE recvBuffer[EGK_RECV_BUFSIZE];
// BYTE cmdXXX[] = {...}; ... SEE ABOVE!
// Get Context
retval = SCardEstablishContext(SCARD_SCOPE_SYSTEM, NULL, NULL, &scContext);
PCSC_ERROR("1 Establish Context", retval);
// Get Reader
retval = SCardListReaders(scContext, NULL, NULL, &dwReaders);
PCSC_ERROR("2.1 Get Readers", retval);
scReaders = calloc(dwReaders, sizeof(DWORD));
retval = SCardListReaders(scContext, NULL, scReaders, &dwReaders);
PCSC_ERROR("2.2 Get Readers", retval);
printf("Reader name: %s\n", scReaders);
// Get Card
retval = SCardConnect(scContext, scReaders, SCARD_SHARE_SHARED, SCARD_PROTOCOL_T0 | SCARD_PROTOCOL_T1, &scHandle, &dwCurProto);
PCSC_ERROR("3 SCardConnect", retval);
// Reset
retval = SCardReconnect(scHandle, SCARD_SHARE_SHARED, SCARD_PROTOCOL_T0 | SCARD_PROTOCOL_T1, SCARD_RESET_CARD, &dwCurProto);
PCSC_ERROR("4 RESET Card", retval);
dwRecvLen = EGK_RECV_BUFSIZE;
if(dwCurProto == SCARD_PROTOCOL_T0) {
scSendProto = *SCARD_PCI_T0;
} else if(dwCurProto == SCARD_PROTOCOL_T1) {
scSendProto = *SCARD_PCI_T1;
} else {
fprintf(stderr, "No known protocol selected\n");
return 1;
}
The Transmit-Part looks the same for every command. Example:
printf("READ BINARY -> First 8 Bytes (Offsets)\n");
retval = SCARD_TRANSMIT(cmdReadOffsets);
PCSC_ERROR("8 Transmit READ BINARY", retval);
TRANS_RESPONSE;
CHECK_SUCCESS; printf("\n");
Edit: Sorry this edit comes so late, I totally forgot about this post! I managed to get it working in the end by rewriting my program in Java. I know this is not quite a satisfying answer for those who may stumble upon this question. At least we can be sure now that the error lies somewhere inside the C code. Should anyone know why the program doesn't work, feel free to answer anyway, I would be really interested in knowing which mistake I made :) !
Related
TL;DR: One ESP32 broadcasts via BLE (already working), another ESP32 listens. I am unable to parse the received advertisements correctly, i.e. can't extract the manufacturer specific data!
Goal: One ESP32 (call A) broadcasts an advertisement containing manufacturer specific data (MSD), which is received by another ESP32 (call B) who prints that data to the console.
I am using the new RISC-V based ESP32C3 which supports Bluetooth 5.0, though everything I do is based on Bluetooth 4.2.
Where I am:
A can broadcast a valid advertisement (checked with an Ubertooth/Wireshark)
B receives something from A, though the packet only very loosely corresponds to the (correct) packet received by the Ubertooth.
Code:
Structs used to set up A:
// Struct defining advertising parameters
static esp_ble_adv_params_t ble_adv_params = {
.adv_int_min = 0x0800,
.adv_int_max = 0x0900,
.adv_type = ADV_TYPE_NONCONN_IND,
.own_addr_type = BLE_ADDR_TYPE_PUBLIC,
.channel_map = ADV_CHNL_ALL,
.adv_filter_policy = ADV_FILTER_ALLOW_SCAN_ANY_CON_ANY, // NO IDEA ABOUT THAT ONE, ESPECIALLY GIVEN THAT WE SEND NONCONNECTABLE AND NONSCANNABLE ADVERTISEMENTS!
};
Struct used to define the payload of the advertisement:
esp_ble_adv_data_t ble_adv_data =
{
.set_scan_rsp = false,
.include_name = true, // "Name" refers to the name passed as an argument within "esp_ble_gap_set_device_name()"
.include_txpower = false,
.min_interval = 0xffff, // Not sure what those are for, as the chosen advertisement packets are non-connectable...
.max_interval = 0xFFFF,
.appearance = 64, // 64 is appearance ID of phone. Only to maybe be able to find it on my Galaxy phone
.manufacturer_len = ble_adv_payload_len,
.p_manufacturer_data = (uint8_t *) ble_adv_payload, // Currently just some human-readable string used for debugging
.service_data_len = 0,
.p_service_data = NULL,
.service_uuid_len = 0,
.p_service_uuid = NULL,
.flag = 0
};
As the Ubertooth receives correct packets sent by A, I reckon A has been set up correctly.
Struct used to define scanning behavior of B:
// Struct defining scanning parameters
static esp_ble_scan_params_t ble_scan_params = {
.scan_type = BLE_SCAN_TYPE_PASSIVE, // Don't send scan requests upon receiving an advertisement
.own_addr_type = BLE_ADDR_TYPE_PUBLIC, // Use (static) public address, makes debugging easier
.scan_filter_policy = BLE_SCAN_FILTER_ALLOW_ONLY_WLST, // Consider all advertisements
.scan_interval = 0x50, // Time between each scan window begin
.scan_window = 0x30, // Length of scan window
.scan_duplicate = BLE_SCAN_DUPLICATE_DISABLE // Filters out duplicate advertisements, e.g. if an advertisement received k times, it is only reported once
};
The majority of the remaining code is just boilerplate, the only really relevant part is the callback function of B, which gets called whenever a GAP-Event occurs (GAP-Events can be found in esp_gap_ble_api.h, beginning on line 138).
B's callback function:
void esp_ble_callback_fun(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param)
{
// Do a case split on the different events. For now, only "ESP_GAP_BLE_SCAN_RESULT_EVT" is of interest
switch (event) {
case ESP_GAP_BLE_SCAN_RESULT_EVT:
if ((param->scan_rst).search_evt != ESP_GAP_SEARCH_INQ_RES_EVT) {
printf(
"B: Callback function received a non-\"ESP_GAP_SEARCH_INQ_RES_EVT\" event!\n");
return;
}
// Copy the parameter UNION
esp_ble_gap_cb_param_t scan_result = *param;
// Create a POINTER to the "bda" entry, which is accessed by interpreting the scan_result UNION as a scan_rst STRUCT
// Note that "esp_bd_addr_t" is a typedef for a uint8_t array!
esp_bd_addr_t *ble_adv_addr = &scan_result.scan_rst.bda;
printf("\n-------------------------\nMessage: \n");
uint8_t adv_data_len = scan_result.scan_rst.adv_data_len;
uint8_t *adv_data = scan_result.scan_rst.ble_adv;
printf("Message length: %i\n", adv_data_len);
printf("Message body:\n"); // NOT SO SURE ABOUT THIS!
for(int i = 0; i < adv_data_len; ++i)
{
printf("%X", adv_data[i]);
}
printf("\n-------------------------\n");
break; // #suppress("No break at end of case")
default:
// NOT SUPPORTED! JUST IGNORE THEM!
break;
}
}
Sample output:
Serial output by B:
Message:
Message length: 22
Message body:
31940069414C4943454FF486579512FFFFFFFF
Packet as received in Wireshark:
Frame 78135: 61 bytes on wire (488 bits), 61 bytes captured (488 bits) on interface /tmp/pipe, id 0
PPI version 0, 24 bytes
Version: 0
Flags: 0x00
.... ...0 = Alignment: Not aligned
0000 000. = Reserved: 0x00
Header length: 24
DLT: 251
Reserved: 36750c0000620900e05b56b811051000
Bluetooth
Bluetooth Low Energy Link Layer
Access Address: 0x8e89bed6
Packet Header: 0x1c22 (PDU Type: ADV_NONCONN_IND, ChSel: #2, TxAdd: Public)
.... 0010 = PDU Type: ADV_NONCONN_IND (0x2)
...0 .... = RFU: 0
..1. .... = Channel Selection Algorithm: #2
.0.. .... = Tx Address: Public
0... .... = Reserved: False
Length: 28
Advertising Address: Espressi_43:3e:d6 (7c:df:a1:43:3e:d6)
Advertising Data
Appearance: Generic Phone
Device Name: ALICE
Manufacturer Specific
Slave Connection Interval Range: 81918.8 - 81918.8 msec
Connection Interval Min: 65535 (81918.8 msec)
Connection Interval Max: 65535 (81918.8 msec)
CRC: 0x905934
0000 00 00 18 00 fb 00 00 00 36 75 0c 00 00 62 09 00 ........6u...b..
0010 e0 5b 56 b8 11 05 10 00 d6 be 89 8e 22 1c d6 3e .[V........."..>
0020 43 a1 df 7c 03 19 40 00 06 09 41 4c 49 43 45 04 C..|..#...ALICE.
0030 ff 48 65 79 05 12 ff ff ff ff 09 9a 2c .Hey........,
For a packet without MSD, I computed the longest common subsequence between the binary representation of a packet received by B (i.e. content of adv_data) and the packet received by the Ubertooth. They only had 46 bits in common, a weird number for sure!
My questions:
Am I right in assuming that adv_data, i.e. scan_result.scan_rst.ble_adv holds the raw BLE packet? The definition of ble_adv (esp_gap_ble_api.h, line 936) is incredibly confusing IMO, as it is called "Received EIR" despite EIRs only being introduced in Bluetooth 5.0...
How can I extract the MSD from a received BLE advertisement?
EIR was introduced a long time ago and was present in Bluetooth 4.0.
You should use %02X when printing hex strings since that will include leading zeros.
ble_adv contains only the EIR content, not the whole packet.
EIR uses length, type, value encoding. Your manufacturing data is encoded like this:
4 (length)
0xff (manufacturer data)
Hey (content)
Note that the two bytes of the manufacturer data content should be a Bluetooth SIG registered company id.
My problem is that I cannot get the AES 256 CTR output from the C code below to match the output from the OpenSSL command below.
The C code produces this:
5f b7 18 d1 28 62 7f 50 35 ba e9 67 a7 17 ab 22
f9 e4 09 ce 23 26 7b 93 82 02 d3 87 eb 01 26 ac
96 2c 01 8c c8 af f3 de a4 18 7f 29 46 00 2e 00
The OpenSSL command line produces this:
5f b7 18 d1 28 62 7f 50 35 ba e9 67 a7 17 ab 22
3c 01 11 bd 39 14 74 76 31 57 a6 53 f9 00 09 b4
6f a9 49 bc 6d 00 77 24 2d ef b9 c4
Notice the first 16 bytes are the same because the nonceIV was the same, however, when the nonceIV is updated on the next iteration, then XOR'd with the plaintext, the next 16 bytes differ and so on...?
I cannot understand why that happens? Anyone know why the hex codes are different after the first 16 byte chunk?
Disclaimer: I'm no C expert.
Thanks!!
Fox.txt
The quick brown fox jumped over the lazy dog
Then run the following OpenSSL command to create foxy.exe
openssl enc -aes-256-ctr -in fox.txt -out foxy.exe -K 603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4 -iv f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff -nosalt -nopad -p
Here's what foxy.exe contains:
5f b7 18 d1 28 62 7f 50 35 ba e9 67 a7 17 ab 22
3c 01 11 bd 39 14 74 76 31 57 a6 53 f9 00 09 b4
6f a9 49 bc 6d 00 77 24 2d ef b9 c4
Here's the code.
#include <Windows.h>
// What is AES CTR
//
// AES - CTR (counter) mode is another popular symmetric encryption algorithm.
//
// It is advantageous because of a few features :
// 1. The data size does not have to be multiple of 16 bytes.
// 2. The encryption or decryption for all blocks of the data can happen in parallel, allowing faster implementation.
// 3. Encryption and decryption use identical implementation.
//
// Very important note : choice of initial counter is critical to the security of CTR mode.
// The requirement is that the same counter and AES key combination can never to used to encrypt more than more one 16 - byte block.
// Notes
// -----
// * CTR mode does not require padding to block boundaries.
//
// * The IV size of AES is 16 bytes.
//
// * CTR mode doesn't need separate encrypt and decrypt method. Encryption key can be set once.
//
// * AES is a block cipher : it takes as input a 16 byte plaintext block,
// a secret key (16, 24 or 32 bytes) and outputs another 16 byte ciphertext block.
//
// References
// ----------
// https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Counter_.28CTR.29
// https://www.cryptopp.com/wiki/CTR_Mode#Counter_Increment
// https://modexp.wordpress.com/2016/03/10/windows-ctr-mode-with-crypto-api/
// https://msdn.microsoft.com/en-us/library/windows/desktop/jj650836(v=vs.85).aspx
// http://www.cryptogrium.com/aes-ctr.html
// http://www.bierkandt.org/encryption/symmetric_encryption.php
#define IV_SIZE 16
#define AES_BLOCK_SIZE 16
typedef struct _key_hdr_t {
PUBLICKEYSTRUC hdr; // Indicates the type of BLOB and the algorithm that the key uses.
DWORD len; // The size, in bytes, of the key material.
char key[32]; // The key material.
} key_hdr;
// NIST specifies two types of counters.
//
// First is a counter which is made up of a nonce and counter.
// The nonce is random, and the remaining bytes are counter bytes (which are incremented).
// For example, a 16 byte block cipher might use the high 8 bytes as a nonce, and the low 8 bytes as a counter.
//
// Second is a counter block, where all bytes are counter bytes and can be incremented as carries are generated.
// For example, in a 16 byte block cipher, all 16 bytes are counter bytes.
//
// This uses the second method, which means the entire byte block is treated as counter bytes.
void IncrementCounterByOne(char *inout)
{
int i;
for (i = 16 - 1; i >= 0; i--) {
inout[i]++;
if (inout[i]) {
break;
}
}
}
void XOR(char *plaintext, char *ciphertext, int plaintext_len)
{
int i;
for (i = 0; i < plaintext_len; i++)
{
plaintext[i] ^= ciphertext[i];
}
}
unsigned int GetAlgorithmIdentifier(unsigned int aeskeylenbits)
{
switch (aeskeylenbits)
{
case 128:
return CALG_AES_128;
case 192:
return CALG_AES_192;
case 256:
return CALG_AES_256;
default:
return 0;
}
}
unsigned int GetKeyLengthBytes(unsigned int aeskeylenbits)
{
return aeskeylenbits / 8;
}
void SetKeyData(key_hdr *key, unsigned int aeskeylenbits, char *pKey)
{
key->hdr.bType = PLAINTEXTKEYBLOB;
key->hdr.bVersion = CUR_BLOB_VERSION;
key->hdr.reserved = 0;
key->hdr.aiKeyAlg = GetAlgorithmIdentifier(aeskeylenbits);
key->len = GetKeyLengthBytes(aeskeylenbits);
memmove(key->key, pKey, key->len);
}
// point = pointer to the start of the plaintext, extent is the size (44 bytes)
void __stdcall AESCTR(char *point, int extent, char *pKey, char *pIV, unsigned int aeskeylenbits, char *bufOut)
{
HCRYPTPROV hProv;
HCRYPTKEY hSession;
key_hdr key;
DWORD IV_len;
div_t aesblocks;
char nonceIV[64];
char tIV[64];
char *bufIn;
bufIn = point;
memmove(nonceIV, pIV, IV_SIZE);
SetKeyData(&key, aeskeylenbits, pKey);
CryptAcquireContext(&hProv, NULL, NULL, PROV_RSA_AES, CRYPT_VERIFYCONTEXT | CRYPT_SILENT);
CryptImportKey(hProv, (PBYTE)&key, sizeof(key), 0, CRYPT_NO_SALT, &hSession);
aesblocks = div(extent, AES_BLOCK_SIZE);
while (aesblocks.quot != 0)
{
IV_len = IV_SIZE;
memmove(tIV, nonceIV, IV_SIZE);
CryptEncrypt(hSession, 0, FALSE, 0, (BYTE *)tIV, &IV_len, sizeof(tIV));
XOR(bufIn, tIV, AES_BLOCK_SIZE);
IncrementCounterByOne(nonceIV);
bufIn += AES_BLOCK_SIZE;
aesblocks.quot--;
}
if (aesblocks.rem != 0)
{
memmove(tIV, nonceIV, IV_SIZE);
CryptEncrypt(hSession, 0, TRUE, 0, (BYTE *)tIV, &IV_len, sizeof(tIV));
XOR(bufIn, tIV, aesblocks.rem);
}
memmove(bufOut, point, extent);
CryptDestroyKey(hSession);
CryptReleaseContext(hProv, 0);
}
I was able to get this working by the suggested pseudocode on the M$ CryptEncrypt() remarks section https://msdn.microsoft.com/en-us/library/windows/desktop/aa379924(v=vs.85).aspx:
// Set the IV for the original key. Do not use the original key for
// encryption or decryption after doing this because the key's
// feedback register will get modified and you cannot change it.
CryptSetKeyParam(hOriginalKey, KP_IV, newIV)
while(block = NextBlock())
{
// Create a duplicate of the original key. This causes the
// original key's IV to be copied into the duplicate key's
// feedback register.
hDuplicateKey = CryptDuplicateKey(hOriginalKey)
// Encrypt the block with the duplicate key.
CryptEncrypt(hDuplicateKey, block)
// Destroy the duplicate key. Its feedback register has been
// modified by the CryptEncrypt function, so it cannot be used
// again. It will be re-duplicated in the next iteration of the
// loop.
CryptDestroyKey(hDuplicateKey)
}
Here's the updated code with the two new lines added:
HCRYPTKEY hDuplicateKey;
boolean final;
while (aesblocks.quot != 0)
{
CryptDuplicateKey(hOriginalKey, NULL, 0, &hDuplicateKey);
IV_len = IV_SIZE;
memmove(tIV, nonceIV, IV_len);
final = (aesblocks.quot == 1 && aesblocks.rem == 0) ? TRUE : FALSE;
CryptEncrypt(hDuplicateKey, 0, final, 0, (BYTE *)tIV, &IV_len, sizeof(tIV));
XOR(bufIn, tIV, AES_BLOCK_SIZE);
IncrementCounterByOne(nonceIV);
bufIn += AES_BLOCK_SIZE;
aesblocks.quot--;
CryptDestroyKey(hDuplicateKey);
}
if (aesblocks.rem != 0)
{
CryptDuplicateKey(hOriginalKey, NULL, 0, &hDuplicateKey);
final = TRUE;
memmove(tIV, nonceIV, IV_SIZE);
CryptEncrypt(hDuplicateKey, 0, final, 0, (BYTE *)tIV, &IV_len, sizeof(tIV));
XOR(bufIn, tIV, aesblocks.rem);
CryptDestroyKey(hDuplicateKey);
}
I'm not familiar with the Microsoft APIs, but I believe that CryptEncrypt() uses CBC mode by default - so the output from the first block of encryption is automatically being fed into the input for the second block. You are building CTR mode yourself form scratch (which incidentally is generally not an advisable thing to do - you should use the capabilities of crypto libraries rather than "roll your own" crypto). To get the expected output you probably need to get CryptEncrypt to use AES in ECB mode - which I believe can be done using CryptptSetKeyParam (https://msdn.microsoft.com/en-us/library/aa380272.aspx) and setting KP_MODE to CRYPT_MODE_ECB.
Make sure your input file doesn't contain any extra characters like new line etc. Openssl will include those extra characters while encrypting.
I am working with one ZigBee module based on 32-bit ARM Cortex-M3. But my question is not related to ZigBee protocol itself. I have access to the source code of application layer only which should be enough for my purposes. Lower layer (APS) passes data to application layer within APSDE-DATA.indication primitive to the following application function:
void zbpro_dataRcvdHandler(zbpro_dataInd_t *data)
{
DEBUG_PRINT(DBG_APP,"\n[APSDE-DATA.indication]\r\n");
/* Output of raw bytes string for further investigation.
* Real length is unknown, 50 is approximation.
*/
DEBUG_PRINT(DBG_APP,"Raw data: \n");
DEBUG_PRINT(DBG_APP,"----------\n");
for (int i = 0; i < 50; i++){
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data+i));
}
DEBUG_PRINT(DBG_APP,"\n");
/* Output of APSDE-DATA.indication primitive field by field */
DEBUG_PRINT(DBG_APP,"Field by field: \n");
DEBUG_PRINT(DBG_APP,"----------------\n");
DEBUG_PRINT(DBG_APP,"Destination address: ");
for (int i = 0; i < 8; i++)
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data->dstAddress.ieeeAddr[i]));
DEBUG_PRINT(DBG_APP,"\n");
DEBUG_PRINT(DBG_APP,"Destination address mode: 0x%02x\r\n",*((uint8_t*)data->dstAddrMode));
DEBUG_PRINT(DBG_APP,"Destination endpoint: 0x%02x\r\n",*((uint8_t*)data->dstEndPoint));
DEBUG_PRINT(DBG_APP,"Source address mode: 0x%02x\r\n",*((uint8_t*)data->dstAddrMode));
DEBUG_PRINT(DBG_APP,"Source address: ");
for (int i = 0; i < 8; i++)
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data->srcAddress.ieeeAddr[i]));
DEBUG_PRINT(DBG_APP,"\n");
DEBUG_PRINT(DBG_APP,"Source endpoint: 0x%02x\r\n",*((uint8_t*)data->srcEndPoint));
DEBUG_PRINT(DBG_APP,"Profile Id: 0x%04x\r\n",*((uint16_t*)data->profileId));
DEBUG_PRINT(DBG_APP,"Cluster Id: 0x%04x\r\n",*((uint16_t*)data->clusterId));
DEBUG_PRINT(DBG_APP,"Message length: 0x%02x\r\n",*((uint8_t*)data->messageLength));
DEBUG_PRINT(DBG_APP,"Flags: 0x%02x\r\n",*((uint8_t*)data->flags));
DEBUG_PRINT(DBG_APP,"Security status: 0x%02x\r\n",*((uint8_t*)data->securityStatus));
DEBUG_PRINT(DBG_APP,"Link quality: 0x%02x\r\n",*((uint8_t*)data->linkQuality));
DEBUG_PRINT(DBG_APP,"Source MAC Address: 0x%04x\r\n",*((uint16_t*)data->messageLength));
DEBUG_PRINT(DBG_APP,"Message: ");
for (int i = 0; i < 13; i++){
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data->messageContents+i));
}
DEBUG_PRINT(DBG_APP,"\n");
bufm_deallocateBuffer((uint8_t *)data, CORE_MEM);
}
APSDE-DATA.indication primitive is implemented by following structures:
/**
* #brief type definition for address (union of short address and extended address)
*/
typedef union zbpro_address_tag {
uint16_t shortAddr;
uint8_t ieeeAddr[8];
} zbpro_address_t;
/**
* #brief apsde data indication structure
*/
PACKED struct zbpro_dataInd_tag {
zbpro_address_t dstAddress;
uint8_t dstAddrMode;
uint8_t dstEndPoint;
uint8_t srcAddrMode;
zbpro_address_t srcAddress;
uint8_t srcEndPoint;
uint16_t profileId;
uint16_t clusterId;
uint8_t messageLength;
uint8_t flags; /* bit0: broadcast or not; bit1: need aps ack or not; bit2: nwk key used; bit3: aps link key used */
uint8_t securityStatus; /* not-used, reserved for future */
uint8_t linkQuality;
uint16_t src_mac_addr;
uint8_t messageContents[1];
};
typedef PACKED struct zbpro_dataInd_tag zbpro_dataInd_t;
As a result I receive next:
[APSDE-DATA.indication]
Raw data:
---------
00 00 00 72 4c 19 40 00 02 e8 03 c2 30 02 fe ff 83 0a 00 e8 05 c1 11 00 11 08 58 40 72 4c ae 53 4d 3f 63 9f d8 51 da ca 87 a9 0b b3 7b 04 68 ca 87 a9
Field by field:
---------------
Destination address: 00 00 00 28 fa 44 34 00
Destination address mode: 0x12
Destination endpoint: 0xc2
Source address mode: 0x12
Source address: 13 01 12 07 02 bd 02 00
Source endpoint: 0xc2
Profile Id: 0xc940
Cluster Id: 0x90a0
Message length: 0x00
Flags: 0x00
Security status: 0x04
Link quality: 0x34
Source MAC Address: 0x90a0
Message: ae 53 4d 3f 63 9f d8 51 da ca 87 a9 0b
From this output I can see that while raw string has some expected values, dispatched fields are totally different. What is the reason of this behavior and how to fix it? Is it somehow related to ARM architecture or wrong type casting?
I don't have access to implementation of DEBUG_PRINT, but we can assume that it works properly.
There's no need to dereference in your DEBUG_PRINT statements, for example
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data->dstAddress.ieeeAddr[i]));
should be simply
DEBUG_PRINT(DBG_APP,"%02x ", data->dstAddress.ieeeAddr[i]);
so on and so forth...
Consider this code:
DEBUG_PRINT(DBG_APP,"%02x ",*((uint8_t*)data->dstAddress.ieeeAddr[i]));
Array subscripting and direct and indirect member access have higher precedence than does casting, so the third argument is equivalent to
*( (uint8_t*) (data->dstAddress.ieeeAddr[i]) )
But data->dstAddress.ieeeAddr[i] is not a pointer, it is an uint8_t. C permits you to convert it to a pointer by casting, but the result is not a pointer to the value, but rather a pointer interpretation of the value. Dereferencing it produces undefined behavior.
Similar applies to your other DEBUG_PRINT() calls.
When I first run I would add 3 records, which gives me a count of 3. Then I fwrite the count into a bin file, and the records into a bin file, then I close the program.
When I reopen it and then I fread in and it will give me my 3 records and a count of 3. But from there on, no matter if I back up or when I read in, it will give me the same count 3 and 3 records, though since the count isn't being updated either this may be why fread is only reading in the first time records.
I am not sure why the counter isn't updating. Both fread and fwrite are returning = success so I am not sure what`s up.
void backUp(PAYROLL employee[], long int *pCounter)
{
FILE *counter;
errno_t result1 = fopen_s(&counter, "c:\\myFiles\\counter.bin", "a+b");
if (result1 == 0){
fwrite(pCounter, sizeof(long int), 1, counter);
fclose(counter);
}
else
printf("Back up of counter failed! error:%d",result1);
FILE *record;
errno_t result2 = fopen_s(&record, "c:\\myFiles\\record.bin", "a+b");
if (result2 == 0){
fwrite(employee, *pCounter *sizeof(PAYROLL), 1, record);
fclose(record);
}
else
printf("Back up of record failed! error:%d", result2);
}
void upload(PAYROLL employee[], long int *pCounter)
{
FILE *counter;
errno_t result1 = fopen_s(&counter, "c:\\myFiles\\counter.bin", "a+b");
if (result1 == 0){
result = fread(pCounter, sizeof(long int), 1, counter);
fclose(counter);
printf("Counter:%d", *pCounter);
}
else
printf("Upload up of counter failed!");
FILE *record;
errno_t result2 = fopen_s(&record, "c:\\myFiles\\record.bin", "r+b");
if (result2 == 0)
{
result2 = fread(employee, *pCounter *sizeof(PAYROLL), 1, record);
printf("Upload successful!\n");
fclose(record);
}
else
printf("Error opening file!");
}
Transferring the most salient comments into an answer.
Weathervane commented:
How do you know that fread and fwrite are returning "success" when you have not checked their return value?
Jude commented:
I look through the debugger and step in to the function, result is giving me their success return values (if that's how it works).
Weathervane commented:
You still need that in the program. Without that sort of checking, your code will be blown over by a puff of wind.
Dmitri correctly observed:
Looks like everywhere you open in append mode "a+b" you should probably be using something else ("rb" in upload() and "wb" in backUp() possibly?)
Jude commented:
I don't understand, is there a specific function for error checking? As I had always thought that error checking was just looking at what goes in the value of result and then I can go check what the value means?
Look at the specification of fread() and
fwrite(). They return the number of records written or read, which may be less than the number requested. If you get a short write, then you have a problem — maybe out of disk space. If you get a short read, it may be that you requested 100 records but there were only 1 or 10 or 99 available to read (or there was an error). If you don't capture and check the return value, you've no idea what happened.
Jude commented:
I see they read and write 1, but it still stores the first 3 elements of my struct array. I assume it's one because it's only writing my array?
fread() (and fwrite() too) give you considerable flexibility because you can supply the size of an item and the number of items separately. You use:
result2 = fread(employee, *pCounter *sizeof(PAYROLL), 1, record);
This tells fread() to read 1 item of size *pCounter * sizeof(PAYROLL). You will get a result of 1 (success) or 0 (failure). You could have specified:
result2 = fread(employee, sizeof(PAYROLL), *pCounter, record);
which would tell you how many records of size sizeof(PAYROLL) were read, up to a maximum of the value in *pCounter. You might get 0 or 1 or …
Here is some workable code that does more or less what's required. The main() program demonstrates working with 1, 2 and 3 records (and the names are a few kings and queens of England, along with the year of their ascension to the throne as their employee ID number). I had to create a minimal payroll structure since the question didn't provide one.
#include <stdio.h>
#include <string.h>
#include <errno.h>
typedef struct PAYROLL
{
long emp_id;
char emp_name[32];
} PAYROLL;
static const char counter_bin[] = "counter.bin";
static const char records_bin[] = "records.bin";
static
void backUp(PAYROLL employee[], long int *pCounter)
{
FILE *counter = fopen(counter_bin, "wb");
if (counter != 0){
fwrite(pCounter, sizeof(long int), 1, counter);
fclose(counter);
}
else
fprintf(stderr, "Back up of counter failed! error: %d %s\n", errno, strerror(errno));
FILE *record = fopen(records_bin, "wb");
if (record != 0){
fwrite(employee, *pCounter *sizeof(PAYROLL), 1, record);
fclose(record);
}
else
fprintf(stderr, "Back up of records failed! error: %d %s\n", errno, strerror(errno));
}
static
void upload(PAYROLL employee[], long int *pCounter)
{
FILE *counter = fopen(counter_bin, "rb");
if (counter != 0){
size_t result = fread(pCounter, sizeof(long int), 1, counter);
fclose(counter);
if (result != 0)
printf("Counter: %ld\n", *pCounter);
else
fprintf(stderr, "Failed to read counter\n");
}
else
fprintf(stderr, "Upload up of counter failed!\n");
FILE *record = fopen(records_bin, "r+b");
if (record != 0)
{
size_t result2 = fread(employee, *pCounter * sizeof(PAYROLL), 1, record);
if (result2 == 1)
printf("Upload successful!\n");
else
fprintf(stderr, "Failed to read records!\n");
fclose(record);
}
else
fprintf(stderr, "Error opening file!");
}
int main(void)
{
PAYROLL emps[] =
{
{ 1066, "William the Conqueror" },
{ 1819, "Victoria" },
{ 1689, "William and Mary" },
};
for (int i = 1; i <= 3; i++)
{
long emp_count = i;
printf("Employee count = %ld\n", emp_count);
backUp(emps, &emp_count);
upload(emps, &emp_count);
for (int j = 0; j < emp_count; j++)
printf("%4ld: %s\n", emps[j].emp_id, emps[j].emp_name);
}
return 0;
}
Note that I've factored out the file names so that you only have to change a single line to change the files used. Sample output:
$ Employee count = 1
Counter: 1
Upload successful!
1066: William the Conqueror
Employee count = 2
Counter: 2
Upload successful!
1066: William the Conqueror
1819: Victoria
Employee count = 3
Counter: 3
Upload successful!
1066: William the Conqueror
1819: Victoria
1689: William and Mary
$ odx counter.bin
0x0000: 03 00 00 00 00 00 00 00 ........
0x0008:
$ odx records.bin
0x0000: 2A 04 00 00 00 00 00 00 57 69 6C 6C 69 61 6D 20 *.......William
0x0010: 74 68 65 20 43 6F 6E 71 75 65 72 6F 72 00 00 00 the Conqueror...
0x0020: 00 00 00 00 00 00 00 00 1B 07 00 00 00 00 00 00 ................
0x0030: 56 69 63 74 6F 72 69 61 00 00 00 00 00 00 00 00 Victoria........
0x0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x0050: 99 06 00 00 00 00 00 00 57 69 6C 6C 69 61 6D 20 ........William
0x0060: 61 6E 64 20 4D 61 72 79 00 00 00 00 00 00 00 00 and Mary........
0x0070: 00 00 00 00 00 00 00 00 ........
0x0078:
$
(odx is just a hex dump program. Pick your own program that does an equivalent job — od -c is a fallback, though I don't particularly like its formatting.)
I see many faults in your program.
Firstly, you are writing long ints and PAYROLL structures directly to a file. You should never do that, as structures and integers have a machine-dependent representation and if you write the files on one machine (let's say 32-bit machine) and read them on another machine (let's say 64-bit machine) then you might run into problems.
Secondly, you're not checking the return value of fread(). It should be always checked.
Thirdly, you're assigning the return value of fread() to an errno_t. Are you sure you really want to do that?
If you want to have the answer to the actual problem, consider updating the source code to fix the mistakes I pointed out and consider improving the English language in your question. Furthermore, you should provide a complete example, i.e. one that contains the definition of PAYROLL. When you know the actual return value of fread(), perhaps the problem will be easier to track down then.
Trying to build Xuggler under Windows. Xuggler is core native code functions wrapped into Java for sound processing purposes (including ffmpeg).
My Windows is x64 Win 7 prof, but all used libraries are 32bit. I am running build procedure under MinGW/MSys, from under Msys shell with the followinf script:
#!/bin/sh
export JAVA_HOME=/C/Program\ Files\ \(x86\)/Java/jdk1.6.0_25
export XUGGLE_HOME=/C/Xuggler
PATH=$XUGGLE_HOME/bin:/C/Program\ Files\ \(x86\)/Java/jdk1.6.0_25/bin:/d/APPS/msysgit/msysgit/bin/git:/D/APPS/MinGW/bin:/bin:/D/APPS/apa che-ant-1.8.2/bin:/D/Users/Dims/Design/MinGW/Util:$PATH
ant -Dbuild.m64=no run-tests
Ant target contains some tests at the end, which give an error. The error follows
[exec] Running 6 tests..
[exec] In StdioURLProtocolHandlerTest::testRead:
[exec] ../../../../../../../../../test/csrc/com/xuggle/xuggler/io/StdioURLProtocolHandlerTest.cpp:108: Error: Expected (4546420 == totalBytes), found (4546420 != 1042)
[exec] In StdioURLProtocolHandlerTest::testReadWrite:
[exec] ../../../../../../../../../test/csrc/com/xuggle/xuggler/io/StdioURLProtocolHandlerTest.cpp:185: Error: Expected (4546420 == totalBytes), found (4546420 != 1042)
[exec] In StdioURLProtocolHandlerTest::testSeek:
[exec] ../../../../../../../../../test/csrc/com/xuggle/xuggler/io/StdioURLProtocolHandlerTest.cpp:139: Error: Expected (4546420 == totalBytes), found (4546420 != 1042)
[exec] .
[exec] Failed 3 of 6 tests
[exec] Success rate: 50%
[exec] FAIL: xugglerioTestStdioURLProtocolHandler.exe
UPDATE 1
The test code is follows:
int32_t totalBytes = 0;
do {
unsigned char buf[2048];
retval = handler->url_read(buf, (int)sizeof(buf));
if (retval > 0)
totalBytes+= retval;
} while (retval > 0);
VS_TUT_ENSURE_EQUALS("", 4546420, totalBytes);
While the url_read code is follows:
int
StdioURLProtocolHandler :: url_read(unsigned char* buf, int size)
{
if (!mFile)
return -1;
return (int) fread(buf, 1, size, mFile);
}
I don't understand, under what circumstances it can return 1042??? May be 64 bits play here somehow?
UPDATE 2
I printed out filename used and it was
d:/......./../../../test/fixtures/testfile.flv
the path is correct, but started with d:/ not with /d/
Can this play a role under Msys?
UPDATE 3
I have compared the readen bytes with real content of the test file and found, that fread() skips some bytes for some reason. Don't know which bytes yet, probably these are CR/LF
UPDATE 4
Not related with CR/LF I guess.
Original bytes are
46 4C 56 01 05 00 00 00 09 00 00 00 00 12 00 00 F4 00 00 00 00 00 00 00 02 00 0A 6F 6E 4D 65 74 61 44 61 74 61 08 00 00 ...
readen bytes are
46 4C 56 15 00 09 00 00 12 00 F4 00 00 00 02 0A 6F 6E 4D 65 74 61 44 61 74 61 80 00 B0 86 47 57 26 17 46 96 F6 E0 40 62 ...
This is FLV file begin. I don't understand the ptinciple of corruption.
How can 01 05 00 00 transform to just 15???
UPDATE 5
File opening done like following
void
StdioURLProtocolHandlerTest :: testRead()
{
StdioURLProtocolManager::registerProtocol("test");
URLProtocolHandler* handler = StdioURLProtocolManager::findHandler("test:foo", 0,0);
VS_TUT_ENSURE("", handler);
int retval = 0;
retval = handler->url_open(mSampleFile, URLProtocolHandler::URL_RDONLY_MODE);
VS_TUT_ENSURE("", retval >= 0);
int32_t totalBytes = 0;
printf("Bytes:\n");
do {
//...
url_open() function follows:
int StdioURLProtocolHandler :: url_open(const char *url, int flags)
{
if (!url || !*url)
return -1;
reset();
const char * mode;
switch(flags) {
case URLProtocolHandler::URL_RDONLY_MODE:
mode="r";
break;
case URLProtocolHandler::URL_WRONLY_MODE:
mode="w";
break;
case URLProtocolHandler::URL_RDWR_MODE:
mode="r+";
break;
default:
return -1;
}
// The URL MAY contain a protocol string. Find it now.
char proto[256];
const char* protocol = URLProtocolManager::parseProtocol(proto, sizeof(proto), url);
if (protocol)
{
size_t protoLen = strlen(protocol);
// skip past it
url = url + protoLen;
if (*url == ':' || *url == ',')
++url;
}
// fprintf(stderr, "protocol: %s; url: %s; mode: %s\n", protocol, url, mode);
mFile = fopen(url, mode);
if (!mFile)
return -1;
return 0;
}
Should be fixed in the GIT repository on the cross_compile branch as of today. I will roll this into tip of tree later this week / early next week.
Now the stdio handler opens all files as binary.