Linux Kernel Line Discipline copy_from_user - c

I have developed a simple line discipline using v4.0.5 of the Linux Kernel, running under Mint Linux.
The tty_ldisc_ops structure looks as follows:
static struct tty_ldisc_ops my_ldisc = {
.owner = THIS_MODULE,
.magic = TTY_LDISC_MAGIC,
.name = "my_ldisc",
.open = my_open,
.close = my_close,
.read = my_read,
.write = my_write,
.ioctl = my_ioctl,
.poll = my_poll,
.receive_buf = my_receive,
.write_wakeup = my_wakeup,
};
The module gets added via insmod my_lkm.ko. I know it's getting added correct as I've used printk to indicate it and can see the message via dmesg. Also, at startup, my userspace application uses ioctl and I have also verified that works via printk.
The problem is, in my_write, copy_from_user always returns a non-zero value indicating that it has failed somehow.
Here is my_write():
static ssize_t my_write(struct tty_struct *tty,
struct file *file,
const unsigned char *buf,
size_t nr)
{
int error = 0;
unsigned char data[MAX]; //MAX is 256
if(!my_tty) {
return -EIO;
}
if(nr > MAX) { //too big
return -ENOMEM;
}
error = copy_from_user(data,buf,nr);
printk("copy_from_user returned %d\n",error);
//here, error is always equal to nr
//(which is 12 in my example application)
if(error==0) {
printk("success\n"); //never get here
return nr;
}
return error;
}
From what I've researched, copy_from_user eventually calls pa_memcpy which does validation of the pointers being used. That validation is failing, but I can't tell why. I don't know see how *buf and data overlap or would cause a fault.
Output from uname -a: Linux mint-linux 4.0.5-040005-generic #201506061639 SMP Sat Jun 6 16:40:45 UTC 2015 UTC x86_64 x86_64 x86_64 GNU/Linux
A snippet of the userspace application is:
#define OPEN_FLAGS (O_RDWR|O_NONBLOCK)
int main(int argc, char **argv)
{
int fd=-1;
int bytes_written= 0;
char device="/dev/ttyUSB0";
unsigned char outbuffer[128]={0};
fd=open(device,OPEN_FLAGS);
//set baud rate, etc., switch to my_ldisc (using N_MOUSE)
outbuffer[0]=0x01;
outbuffer[1]=0x02;
outbuffer[2]=0x03;
outbuffer[3]=0x04;
outbuffer[4]=0x05;
outbuffer[5]=0x06;
outbuffer[6]=0x07;
outbuffer[7]=0x08;
outbuffer[8]=0x09;
outbuffer[9]=0x0A;
outbuffer[10]=0x0B;
outbuffer[11]=0x0C;
bytes_written=write(fd,outbuffer,12);
while(true) {
//...
sleep(1);
}
}
In addition, any access of buf in my_write causes instability in the VM. Even following the tty driver example in the o'reilly linux drivers book like this:
printk(KERN_DEBUG "%s - ", __FUNCTION__);
for(i=0;i<nr;i++)
{
printk("%02x ",buf[i]);
}
printk("\n");
Following Tsyvarev's advice, I printed the pointer in the user space application and the kernel module. They were different which meant I should access the incoming buffer directly. I used printf("%p\n",outbuffer); to do that in user space and the equivalent printk in kernel space.
So, slowing down and testing the module line by line helped me to fix the original problem, which it turns out was a bug in the user space application.
FWIW, the compiler never did give me a warning about the use of __user in the original code. Had it worked the way Tsyvarev suggested it would at compile time, it would have made this a lot easier to track down.

Unlike to .write method of struct file_operations, which accepts pointer to user data, .write method for struct tty_operations accepts pointer to kernel data, and these data shold be accessed via usual methods such as memcpy or even directly.
Modern kernel uses __user attribute for mark user-space data, and this attribute is checked (at compile time) when data is accessed. So having compiler warnings enabled will reveal usage of the data with incorrect origin.

Related

linux proc size limit problems

I'm trying to write a linux kernel module that can dump the contents of other modules to a /proc file (for analysis). In principle it works but it seems I run into some buffer limit or the like. I'm still rather new to Linux kernel development so I would also appreciate any suggestions not concerning the particular problem.
The memory that is used to store the module is allocated in this function:
char *get_module_dump(int module_num)
{
struct module *mod = unhiddenModules[module_num];
char *buffer;
buffer = kmalloc(mod->core_size * sizeof(char), GFP_KERNEL);
memcpy((void *)buffer, (void *)startOf(mod), mod->core_size);
return buffer;
}
'unhiddenModules' is an array of module structs
Then it is handed over to the proc creation here:
void create_module_dump_proc(int module_number)
{
struct proc_dir_entry *dump_module_proc;
dump_size = unhiddenModules[module_number]->core_size;
module_buffer = get_module_dump(module_number);
sprintf(current_dump_file_name, "%s_dump", unhiddenModules[module_number]->name);
dump_module_proc = proc_create_data(current_dump_file_name, 0, dump_proc_folder, &dump_fops, module_buffer);
}
The proc read function is as follows:
ssize_t dump_proc_read(struct file *filp, char *buf, size_t count, loff_t *offp)
{
char *data;
ssize_t ret;
data = PDE_DATA(file_inode(filp));
ret = copy_to_user(buf, data, dump_size);
*offp += dump_size - ret;
if (*offp > dump_size)
return 0;
else
return dump_size;
}
Smaller Modules are dumped correctly but if the module is larger than 126,796 bytes only the first 126,796 bytes are written and this error is displayed when reading from the proc file:
*** Error in `cat': free(): invalid next size (fast): 0x0000000001f4a040 ***
I've seem to run into some limit but I couldn't find anything on it. The error seems to be related so memory leaks but the buffer should be large enough so I don't see where this actually happens.
The procfs has a limit of PAGE_SIZE (one page) for read and write operations. Usually seq_file is used to iterate over the entries (modules in your case ?) to read and/or write smaller chunks. Since you are running into problems with only larger data, I suspect this is the case here.
Please have a look here and here if you are not familiar with seq_files.
A suspicious thing is that in dump_proc_read you are not using "count" parameter. I would have expected copy_to_user to take "count" as third argument instead of "dump_size" (and in subsequent calculations too). The way you do, always dump_size bytes are copied to user space, regardless the data size the application was expecting. The bigger dump_size is, the larger the user area that gets corrupted.

How to write a simple Linux device driver?

I need to write an SPI Linux character device driver for omap4 from scratch.
I know some basics of writing device drivers. But, I don't know how to start writing platform specific device driver from scratch.
I've written some basic char drivers, and I thought writing SPI device driver would be similar to it. Char drivers have a structure file_operations which contains the functions implemented in the driver.
struct file_operations Fops = {
.read = device_read,
.write = device_write,
.ioctl = device_ioctl,
.open = device_open,
.release = device_release, /* a.k.a. close */
};
Now, I am going through spi-omap2-mcspi.c code as a reference to get an idea to start developing SPI driver from scratch.
But, I don't see functions such as open, read, write etc.
Don't know from where the program starts.
First, start by writing a generic kernel module. There are multiple places to look up for information but I found this link to be very useful. After you have gone through all examples specified there you can start writing your own Linux Driver Module.
Please note, that you will not get away with just copy-pasting the example code and hope it will work, no. Kernel API can sometimes change and examples will not work. Examples provided there should be looked at as a guide on how to do something. Depending on the kernel version you are using you have to modify the example in order to work.
Consider using TI platform-provided functions as much as you can, because that can really do a lot of work for you, like requesting and enabling needed clocks, buses, and power supplies. If I recall correctly you can use the functions to acquire memory-mapped address ranges for direct access to registers. I have to mention that I have a bad experience with TI-provided functions because they do not properly release/clean up all acquired resources, so for some resources, I had to call other kernel services to release them during module unload.
Edit 1:
I'm not entirely familiar with Linux SPI implementation but I would start by looking at omap2_mcspi_probe() function in drivers/spi/spi-omap2-mcspi.c file. As you can see there, it registers it's methods to Linux master SPI driver using this API: Linux/include/linux/spi/spi.h. In contrast to char driver, the main functions here are *_transfer() functions. Look up the struct descriptions in spi.h file for further details. Also, have a look at this alternative device driver API, too.
I assume your OMAP4 linux uses one of arch/arm/boot/dts/{omap4.dtsi,am33xx.dtsi} device-tree, thus it compiles drivers/spi/spi-omap2-mcspi.c (if you don't know about device-tree, read this). Then:
the SPI master driver is done,
it (most probably) registers with Linux SPI core framework drivers/spi/spi.c,
it (probably) works fine on your OMAP4.
You actually don't need to care about the master driver to write your slave device driver. How do I know spi-omap2-mcspi.c is a master driver? It calls spi_register_master().
SPI master, SPI slave ?
Please refer to Documentation/spi/spi_summary. The doc refers to Controller driver (master) and Protocol drivers (slave). From your description, I understand you want to write a Protocol/Device driver.
SPI protocol ?
To understand that, you need your slave device datasheet, it shall tell you:
the SPI mode understood by your device,
the protocol it expects on the bus.
Contrary to i2c, SPI does not define a protocol or handshake, SPI chips manufacturers have to define their own. So check the datasheet.
SPI mode
From include/linux/spi/spi.h:
* #mode: The spi mode defines how data is clocked out and in.
* This may be changed by the device's driver.
* The "active low" default for chipselect mode can be overridden
* (by specifying SPI_CS_HIGH) as can the "MSB first" default for
* each word in a transfer (by specifying SPI_LSB_FIRST).
Again, check your SPI device datasheet.
An example SPI device driver?
To give you a relevant example, I need to know your SPI device type. You would understand that a SPI flash device driver is different from a SPI FPGA device driver. Unfortunately there are not so many SPI device drivers out there. To find them:
$ cd linux
$ git grep "spi_new_device\|spi_add_device"
I don't know if I understood your question correctly. As m-ric pointed out, there are master drivers and slave drivers.
Usually master drivers are more hardware bound, I mean, they usually manipulate IO registers or do some memory mapped IO.
For some architectures already supported by linux kernel (like omap3 and omap4) master drivers are already implemented (McSPI).
So I assume you want to USE those SPI facilities of omap4 to implement a slave device driver (your protocol, to communicate with your external device through SPI).
I've written the following example for BeagleBoard-xM (omap3). The full code is at https://github.com/rslemos/itrigue/blob/master/alsadriver/itrigue.c (worth a view, but have more initialisation code, for ALSA, GPIO, module parameters). I've tried to set apart code that deals with SPI (maybe I forgot something, but anyway you should get the idea):
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/spi/spi.h>
/* MODULE PARAMETERS */
static uint spi_bus = 4;
static uint spi_cs = 0;
static uint spi_speed_hz = 1500000;
static uint spi_bits_per_word = 16;
/* THIS IS WHERE YOUR DEVICE IS CREATED; THROUGH THIS YOU INTERACT WITH YOUR EXTERNAL DEVICE */
static struct spi_device *spi_device;
/* SETUP SPI */
static inline __init int spi_init(void) {
struct spi_board_info spi_device_info = {
.modalias = "module name",
.max_speed_hz = spi_speed_hz,
.bus_num = spi_bus,
.chip_select = spi_cs,
.mode = 0,
};
struct spi_master *master;
int ret;
// get the master device, given SPI the bus number
master = spi_busnum_to_master( spi_device_info.bus_num );
if( !master )
return -ENODEV;
// create a new slave device, given the master and device info
spi_device = spi_new_device( master, &spi_device_info );
if( !spi_device )
return -ENODEV;
spi_device->bits_per_word = spi_bits_per_word;
ret = spi_setup( spi_device );
if( ret )
spi_unregister_device( spi_device );
return ret;
}
static inline void spi_exit(void) {
spi_unregister_device( spi_device );
}
To write data to your device:
spi_write( spi_device, &write_data, sizeof write_data );
The above code is independent of implementation, that is, it could use McSPI, bit-banged GPIO, or any other implementation of an SPI master device. This interface is described in linux/spi/spi.h
To make it work in BeagleBoard-XM I had to add the following to the kernel command line:
omap_mux=mcbsp1_clkr.mcspi4_clk=0x0000,mcbsp1_dx.mcspi4_simo=0x0000,mcbsp1_dr.mcspi4_somi=0x0118,mcbsp1_fsx.mcspi4_cs0=0x0000
So that an McSPI master device is created for omap3 McSPI4 hardware facility.
Hope that helps.
file_operations minimal runnable example
This example does not interact with any hardware, but it illustrates the simpler file_operations kernel API with debugfs.
Kernel module fops.c:
#include <asm/uaccess.h> /* copy_from_user, copy_to_user */
#include <linux/debugfs.h>
#include <linux/errno.h> /* EFAULT */
#include <linux/fs.h> /* file_operations */
#include <linux/kernel.h> /* min */
#include <linux/module.h>
#include <linux/printk.h> /* printk */
#include <uapi/linux/stat.h> /* S_IRUSR */
static struct dentry *debugfs_file;
static char data[] = {'a', 'b', 'c', 'd'};
static int open(struct inode *inode, struct file *filp)
{
pr_info("open\n");
return 0;
}
/* #param[in,out] off: gives the initial position into the buffer.
* We must increment this by the ammount of bytes read.
* Then when userland reads the same file descriptor again,
* we start from that point instead.
* */
static ssize_t read(struct file *filp, char __user *buf, size_t len, loff_t *off)
{
ssize_t ret;
pr_info("read\n");
pr_info("len = %zu\n", len);
pr_info("off = %lld\n", (long long)*off);
if (sizeof(data) <= *off) {
ret = 0;
} else {
ret = min(len, sizeof(data) - (size_t)*off);
if (copy_to_user(buf, data + *off, ret)) {
ret = -EFAULT;
} else {
*off += ret;
}
}
pr_info("buf = %.*s\n", (int)len, buf);
pr_info("ret = %lld\n", (long long)ret);
return ret;
}
/* Similar to read, but with one notable difference:
* we must return ENOSPC if the user tries to write more
* than the size of our buffer. Otherwise, Bash > just
* keeps trying to write to it infinitely. */
static ssize_t write(struct file *filp, const char __user *buf, size_t len, loff_t *off)
{
ssize_t ret;
pr_info("write\n");
pr_info("len = %zu\n", len);
pr_info("off = %lld\n", (long long)*off);
if (sizeof(data) <= *off) {
ret = 0;
} else {
if (sizeof(data) - (size_t)*off < len) {
ret = -ENOSPC;
} else {
if (copy_from_user(data + *off, buf, len)) {
ret = -EFAULT;
} else {
ret = len;
pr_info("buf = %.*s\n", (int)len, data + *off);
*off += ret;
}
}
}
pr_info("ret = %lld\n", (long long)ret);
return ret;
}
/*
Called on the last close:
http://stackoverflow.com/questions/11393674/why-is-the-close-function-is-called-release-in-struct-file-operations-in-the-l
*/
static int release(struct inode *inode, struct file *filp)
{
pr_info("release\n");
return 0;
}
static loff_t llseek(struct file *filp, loff_t off, int whence)
{
loff_t newpos;
pr_info("llseek\n");
pr_info("off = %lld\n", (long long)off);
pr_info("whence = %lld\n", (long long)whence);
switch(whence) {
case SEEK_SET:
newpos = off;
break;
case SEEK_CUR:
newpos = filp->f_pos + off;
break;
case SEEK_END:
newpos = sizeof(data) + off;
break;
default:
return -EINVAL;
}
if (newpos < 0) return -EINVAL;
filp->f_pos = newpos;
pr_info("newpos = %lld\n", (long long)newpos);
return newpos;
}
static const struct file_operations fops = {
/* Prevents rmmod while fops are running.
* Try removing this for poll, which waits a lot. */
.owner = THIS_MODULE,
.llseek = llseek,
.open = open,
.read = read,
.release = release,
.write = write,
};
static int myinit(void)
{
debugfs_file = debugfs_create_file("lkmc_fops", S_IRUSR | S_IWUSR, NULL, NULL, &fops);
return 0;
}
static void myexit(void)
{
debugfs_remove_recursive(debugfs_file);
}
module_init(myinit)
module_exit(myexit)
MODULE_LICENSE("GPL");
Userland shell test program:
#!/bin/sh
mount -t debugfs none /sys/kernel/debug
insmod /fops.ko
cd /sys/kernel/debug/lkmc_fops
## Basic read.
cat f
# => abcd
# dmesg => open
# dmesg => read
# dmesg => len = [0-9]+
# dmesg => close
## Basic write
printf '01' >f
# dmesg => open
# dmesg => write
# dmesg => len = 1
# dmesg => buf = a
# dmesg => close
cat f
# => 01cd
# dmesg => open
# dmesg => read
# dmesg => len = [0-9]+
# dmesg => close
## ENOSPC
printf '1234' >f
printf '12345' >f
echo "$?"
# => 8
cat f
# => 1234
## seek
printf '1234' >f
printf 'z' | dd bs=1 of=f seek=2
cat f
# => 12z4
You should also write a C program that runs those tests if it is not clear to you what system calls are being called for each of those commands. (or you could also use strace and find out :-)).
The other file_operations are a bit more involved, here are some further examples:
ioctl
poll
mmap
Start with software models of simplified hardware in emulators
Actual device hardware development is "hard" because:
you can't always get your hand on a given hardware easily
hardware APIs may be complicated
it is hard to see what is the internal state of the hardware
Emulators like QEMU allow us to overcome all those difficulties, by simulating simplified hardware simulation in software.
QEMU for example, has a built-in educational PCI device called edu, which I explained further at: How to add a new device in QEMU source code? and is a good way to get started with device drivers. I've made a simple driver for it available here.
You can then put printf's or use GDB on QEMU just as for any other program, and see exactly what is going on.
There is also an OPAM SPI model for you specific use case: https://github.com/qemu/qemu/blob/v2.7.0/hw/ssi/omap_spi.c

How to set breakpoint to obtain address of a function in fork.c , in the kernel source?

Good day to all. I have this query which I hope someone is able to help me with. I forward my gratitude and thanks in advance. I had done hours of search but unable to find a solution.
My problem:
I need to obtain the address of the " security_task_create(clone_flags)" function the following code snippet (located in line 926 ,fork.c as per "/usr/src/linux-2.6.27/kernel/fork.c") -:
************************************ ************************************
static struct task_struct *copy_process(unsigned long clone_flags,
unsigned long stack_start,
struct pt_regs *regs,
unsigned long stack_size,
int __user *child_tidptr,
struct pid *pid,
int trace)
{
int retval;
struct task_struct *p;
int cgroup_callbacks_done = 0;
if ((clone_flags & (CLONE_NEWNS|CLONE_FS)) == (CLONE_NEWNS|CLONE_FS))
return ERR_PTR(-EINVAL);
/*
* Thread groups must share signals as well, and detached threads
* can only be started up within the thread group.
*/
if ((clone_flags & CLONE_THREAD) && !(clone_flags & CLONE_SIGHAND))
return ERR_PTR(-EINVAL);
/*
* Shared signal handlers imply shared VM. By way of the above,
* thread groups also imply shared VM. Blocking this case allows
* for various simplifications in other code.
*/
if ((clone_flags & CLONE_SIGHAND) && !(clone_flags & CLONE_VM))
return ERR_PTR(-EINVAL);
****retval = security_task_create(clone_flags);****
if (retval)
goto fork_out;
retval = -ENOMEM;
p = dup_task_struct(current);
if (!p)
goto fork_out;
rt_mutex_init_task(p);
************************************ ************************************
I've enabled KDB access over keyboard in my Fedora Core 16 machine with kernel 3.1.7. Upon entering KDB console i.e. " kdb[0]> , I typed security_task_create and a hex address e.g. 0x0040118e is displayed.
My Questions:
Is the displayed hex address - the address of the security_task_create upon the kernel loaded?
2.If not, how am I able to obtain the address of the security_task_create function? How do I configure KDB to obtain the address of the security_task_create function?
What I have in mind is to insert a breakpoint at line 926 in fork.c using KDB when the kernel runs security_task_create in memory. If such is indeed the proper solution, how do I obtain the address of security_task_create using such method?
For getting address of any symbol in kernel use System.map file simply.
CONFIG_KALLSYMS is needs to be enabled in kernel configuration for getting all symbols in that file.
Just grep for printk in your source directory and I'm sure you'll find tons of examples.
printk(KERN_INFO "fork(): process `%s' used deprecated "
"clone flags 0x%lx\n",
get_task_comm(comm, current),
clone_flags & CLONE_STOPPED);

copy_to_user vs memcpy

I have always been told(In books and tutorials) that while copying data from kernel space to user space, we should use copy_to_user() and using memcpy() would cause problems to the system. Recently by mistake i have used memcpy() and it worked perfectly fine with out any problems. Why is that we should use copy_to_user instead of memcpy()
My test code(Kernel module) is something like this:
static ssize_t test_read(struct file *file, char __user * buf,
size_t len, loff_t * offset)
{
char ani[100];
if (!*offset) {
memset(ani, 'A', 100);
if (memcpy(buf, ani, 100))
return -EFAULT;
*offset = 100;
return *offset;
}
return 0;
}
struct file_operations test_fops = {
.owner = THIS_MODULE,
.read = test_read,
};
static int __init my_module_init(void)
{
struct proc_dir_entry *entry;
printk("We are testing now!!\n");
entry = create_proc_entry("test", S_IFREG | S_IRUGO, NULL);
if (!entry)
printk("Failed to creats proc entry test\n");
entry->proc_fops = &test_fops;
return 0;
}
module_init(my_module_init);
From user-space app, i am reading my /proc entry and everything works fine.
A look at source code of copy_to_user() says that it is also simple memcpy() where we are just trying to check if the pointer is valid or not with access_ok and doing memcpy.
So my understanding currently is that, if we are sure about the pointer we are passing, memcpy() can always be used in place of copy_to_user.
Please correct me if my understanding is incorrect and also, any example where copy_to_user works and memcpy() fails would be very useful. Thanks.
There are a couple of reasons for this.
First, security. Because the kernel can write to any address it wants, if you just use a user-space address you got and use memcpy, an attacker could write to another process's pages, which is a huge security problem. copy_to_user checks that the target page is writable by the current process.
There are also some architecture considerations. On x86, for example, the target pages must be pinned in memory. On some architectures, you might need special instructions. And so on. The Linux kernels goal of being very portable requires this kind of abstraction.
This answer may be late but anyway copy_to_user() and it's sister copy_from_user() both do some size limits checks about user passed size parameter and buffer sizes so a read method of:
char name[] = "This message is from kernel space";
ssize_t read(struct file *f, char __user *to, size_t size, loff_t *loff){
int ret = copy_to_user(to, name, size);
if(ret){
pr_info("[+] Error while copying data to user space");
return ret;
}
pr_info("[+] Finished copying data to user space");
return 0;
}
and a user space app read as read(ret, buffer, 10); is OK but replace 10 with 35 or more and kernel will emit this error:
Buffer overflow detected (34 < 35)!
and cause the copy to fail to prevent memory leaks. Same goes for copy_from_user() which will also make some kernel buffer size checks.
That's why you have to use char name[] and not char *name since using pointer(not array) makes determining size not possible which will make kernel emit this error:
BUG: unable to handle page fault for address: ffffffffc106f280
#PF: supervisor write access in kernel mode
#PF: error_code(0x0003) - permissions violation
Hope this answer is helpful somehow.

Trouble Calling ioctl from user-space C

I'm trying to implement a program to access memory on an embedded system. I need to access some control register so I think that ioctl is the best way to do it. I have added the ioctl to the fops:
struct file_operations aes_fops = {
read: aes_read,
write: aes_write,
unlocked_ioctl: aes_ioctl,
open: aes_open,
release: aes_release
};
And have set up the function:
int aes_ioctl(struct inode *inode,
struct file *file,
unsigned int ioctl_num,
unsigned long ioctl_param){
printk(KERN_INFO "in ioctl\n");
....
}
But I am not getting inside of this function. Here is my user space code. Please help me understand if I am doing this totally wrong.
int main(int argc, char* argv[]){
int fd = fopen("/dev/aes", "r+");
ioctl(fd, 0, 1);
fclose(fd);
}
Some of the code is apparently for older kernels, because I am compiling for an embedded system where an older version of Linux has been modified.
The problem with your code is the request number you are using - 0. The kernel has some request number reserved for internal use. The kernel regards the request number as a struct, separates it to fields and calls the right subsystem for it.
See Documentation/ioctl/ioctl-number.txt (from Linux 3.4.6):
Code Seq#(hex) Include File Comments
========================================================
0x00 00-1F linux/fs.h conflict!
0x00 00-1F scsi/scsi_ioctl.h conflict!
0x00 00-1F linux/fb.h conflict!
0x00 00-1F linux/wavefront.h conflict!
0x02 all linux/fd.h
0x03 all linux/hdreg.h
...
Depending on what you are during, you'd have to follow the kernel guidelines for adding new ioctls():
If you are adding new ioctl's to the kernel, you should use the _IO
macros defined in <linux/ioctl.h>:
_IO an ioctl with no parameters
_IOW an ioctl with write parameters (copy_from_user)
_IOR an ioctl with read parameters (copy_to_user)
_IOWR an ioctl with both write and read parameters.
See the kernel's own Documentation/ioctl/ioctl-decoding.txt document for further details on how those numbers are structured.
In practice, if you pick Code 1, which means starting from 0x100 up until 0x1ff, you'd be fine.
Is your setup of the aes_fops structure correct? I've never seen it done that way. All the code I have is:
.unlocked_ioctl = aes_ioctl,
rather than:
unlocked_ioctl: aes_ioctl,
Colons within a structure (as you have in your setup) fields are used for bit fields as far as I'm aware (and during definition), not initialising the individual fields.
In other words, try:
struct file_operations aes_fops = {
.read = aes_read,
.write = aes_write,
.unlocked_ioctl = aes_ioctl,
.open = aes_open,
.release = aes_release
};
Note: It appears that gcc once did allow that variant of structure field initialisation but it's been obsolete since gcc 2.5 (see here, straight from the GCC documentation). You really should be using the proper method (ie, the one blessed by the ISO standard) to do this.
Without knowing the error returned, it's hard to say... My first though is your permissions on your file descriptor. I've seen similar issues before. First, you can get some more information about the failure if you take a look at the return of the ioctl:
#include <errno.h>
int main(int argc, char* argv[])
{
long ret;
int fd = fopen("/dev/aes", "r+");
ret = ioctl(fd, 0, 1);
if (ret < 0)
printf("ioctl failed. Return code: %d, meaning: %s\n", ret, strerror(errno));
fclose(fd);
}
Check the return values and this should help give you something to search on. Why check? See the bottom of the post...
Next in order to check if it is permissions issue, run the command:
ls -l /dev/aes
if you see something like:
crw------- 1 root root 10, 57 Aug 21 10:24 /dev/aes
Then just issue a:
sudo chmod 777 /dev/aes
And give it a try again. I bet it will work for you. (Note I ran that with root permissions since root is the owner of my version of your mod)
If the permissions are already OK, then I have a few more suggestions:
1) To me, the use of fopen/fclose is strange. You really only need to do:
int fd = open("/dev/aes");
close(fd);
My system doesn't even let your code compile as is.
2) Your IOCTL parameter list is old, I don't know what kernel version your compiling on, but recent kernels use this format:
long aes_ioctl(struct file *file, unsigned int ioctl_num, unsigned long ioctl_param){
Note the removal of the inode and the change of the return type. When I ran your code on my system, I made these changes.
Best of luck!
Note: Why should we check the return when we're "not getting into the ioctl"? Let me give you an example:
//Kernel Code:
//assume include files, other fops, exit, miscdev struct, etc. are present
long hello_ioctl(struct file *file, unsigned long ioctl_num, unsigned long ioctl_param) {
long ret = 0;
printk("in ioctl");
return ret;
}
static const struct file_operations hello_fops = {
owner: THIS_MODULE,
read: hello_read,
unlocked_ioctl: hello_ioctl,
};
static int __init hello_init(void) {
int ret;
printk("hello!\n");
ret = misc_register(&hello_dev); //assume it worked...
return ret;
}
User space code:
//assume includes
void main() {
int fd;
long ret;
fd = open("/dev/hello");
if(fd) {
c = ioctl(fd, 0, 1);
if (c < 0)
printf("error: %d, errno: %d, meaning: %s\n", c, errno, strerror(errno));
close(fd);
}
return;
}
So what's the output? Lets assume bad file permissions on /dev/hello (meaning our user space program can't access /dev/hello).
The dmesg | tail shows:
[ 2388.051660] Hello!
So it looks like we didn't "get in to" the ioctl. What's the output from the program?
error: -1, errno: 9, meaning: Bad file descriptor
Lots of useful output. Clearly the ioctl call did something, just not what we wanted. Now changing the permissions and re-running we can see the new dmesg:
[ 2388.051660] Hello!
[ 2625.025339] in ioctl

Resources