AES-GCM with the Kernel Crypto API from Userspace - c

Question

I am trying to use the kernel crypto API to communicate with a hardware accelerator from userspace.
I use af_alg for the communication with the kernel. I can use the API for hashing or "normal" symmetric encryption but I can not get it to work with an aead algorithm. The drivers are registered under /proc/crypto.
I follow the kernel documentation for the userspace interface to prepare the message but I get the same error Invalig Argument every time.
#ifndef AF_ALG
#define AF_ALG 38
#endif
#ifndef SOL_ALG
#define SOL_ALG 279
#endif
int main(void)
{
int openfd;
int tfmfd;
char key [16] = "SecretKeyforAES!";
char initVector[12] = "123456789012";
//initiaze vector, in/out vector and buffer
struct af_alg_iv *iv;
struct iovec iov;
char buf[64];
//size of the aad and the auth tag
uint32_t aadSize = 8;
uint32_t taglen = 12;
//define algorithm used for cryptography
struct sockaddr_alg sa = {
.salg_family = AF_ALG,
.salg_type = "aead",
.salg_name = "gcm(aes)"
};
//create and bind socket
tfmfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
printf("open socket\n");
int bindvalue = bind(tfmfd, (struct sockaddr *) &sa, sizeof(sa));
if(bindvalue != 0){
perror("send != 0");
}
assert(bindvalue == 0);
printf("socket bound\n");
//set socket options: key, AEAD Authentication size
setsockopt(tfmfd, SOL_ALG, ALG_SET_KEY, key, 16);
int setTag = setsockopt(tfmfd, SOL_ALG, ALG_SET_AEAD_AUTHSIZE, NULL, taglen);
assert(setTag >= 0);
printf("set socket options\n");
//accept connection
openfd = accept(tfmfd, NULL, 0);
//check errors with the connections
assert(tfmfd > 0);
assert(openfd > 0);
//Prepare Message
struct msghdr msg = { 0 };
struct cmsghdr *cmsg = NULL;
char cbuf[128] = {0};
msg.msg_control = cbuf;
msg.msg_controllen = sizeof(cbuf);
msg.msg_iovlen = 0;
msg.msg_iov = NULL;
//set the Headervalues for the Operation
cmsg = CMSG_FIRSTHDR(&msg);
assert(cmsg != NULL);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_OP;
cmsg->cmsg_len = CMSG_LEN(CMSG_SPACE(sizeof(uint32_t)));
*(__u32 *) CMSG_DATA(cmsg) = ALG_OP_ENCRYPT;
//set headervalues for IV
cmsg = CMSG_NXTHDR(&msg, cmsg);
assert(cmsg != NULL);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_IV;
cmsg->cmsg_len = CMSG_LEN(CMSG_SPACE(sizeof(initVector))); //iv_msg_size
//set value for iv
iv = (void *)CMSG_DATA(cmsg);
iv->ivlen = 12;
memcpy(iv->iv, initVector, 12);
//set headervalues for aad
uint32_t *assoclen = NULL;
cmsg = CMSG_NXTHDR(&msg, cmsg);
assert(cmsg != NULL);
cmsg->cmsg_level = SOL_ALG;
cmsg->cmsg_type = ALG_SET_AEAD_ASSOCLEN;
cmsg->cmsg_len = CMSG_LEN(sizeof(*assoclen));
assoclen = (void *) CMSG_DATA(cmsg);
*assoclen = (uint32_t)aadSize;
printf("Header values set\n");
//send initial message
int send = sendmsg(openfd, &msg, MSG_MORE);
if(send < 0){
perror("send < 0"); //This is where the error occures
}
assert(send > 0); //This assertion fails following the error above
printf("Send initial Message...\n");
Following the kernel doc I prepared the initial message with the algorithm, iv and length of aad information.
I set the taglen and mode with setsockopt as well. I read the man pages of msghdr, cmsg and sendmsg but did not find my error.
EDIT:
Added a check for binding failure but check exits with 0 -> no error

Answer 1

You are not quite correct in counting cmsg_len.
For the first value that's correct though CMSG_LEN(CMSG_SPACE(sizeof(uint32_t))) reflects ALG_SET_OP.
But not CMSG_LEN(CMSG_SPACE(sizeof(initVector))); //iv_msg_size where the data is iv = (void *)CMSG_DATA(cmsg); -> struct af_alg_iv *iv;.
Thus it should beCMSG_LEN(sizeof(struct af_alg_iv) + sizeof(initVector)).
Also, char cbuf[128] = {0}; must consist the total size of ancillary data, what is char cbuf[CMSG_SPACE(sizeof(uint32_t)) + CMSG_SPACE(sizeof(struct af_alg_iv) + sizeof(initVector))] = {0}