Is there a way to increase the size of ebpf stack getting "error looks like the bpf stack limit of 512 bytes is exeeded", so is there a ebpf helper function or command that I can use to increase the sizeof stack so won't get this error
On this page it says this, but it was posted some time ago; I wonder if there is any work around this limitation:
Currently, no. The stack size is limited to 512 bytes, and there is no kmalloc style dynamic allocation inside the bpf program either. One way you could try is with per-cpu map with value size of 4k and fill in the 4k map value and submit it with the map value. But I never tried this before.
This is my ebpf function
struct {
__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
__type(key, long);
__type(value, long);
__uint(max_entries, 1024);
} tcp_map SEC(".maps");
struct buffer2
{
char buffer[1024];
};
SEC("kprobe/__sys_sendto")
int bpf_prog3(struct pt_regs *ctx)
{
char buf[300];
struct buffer2 buf1;
int fd = (int)PT_REGS_PARM1(ctx);
//cha ptr=(char *)PT_REGS_PARM2(ctx);
//char *buffer=(char *)ptr;
bpf_probe_read(buf1.buffer,1023,(void *)PT_REGS_PARM2(ctx));
bpf_trace_printk("[fd = %d]\n",sizeof("[fd = %d]\n"),fd);
bpf_trace_printk("[buffer = %x]\n",sizeof("[buffer = %x]\n"),buf);
long ptr3=PT_REGS_PARM3(ctx);
if(ptr3>0)
bpf_trace_printk("***********************************************\n",sizeof("***********************************************\n"));
bpf_trace_printk("[count = %ld]\n",sizeof("[count = %d]\n"),ptr3);
long *value;
value = bpf_map_lookup_elem(&tcp_map, buf1.buffer);
bpf_map_update_elem(&tcp_map, 0, &buf1, BPF_ANY);
}
I like to read 1024 bytes from function parameter 2 using bpf_probe_read(buf1.buffer,1023,(void *)PT_REGS_PARM2(ctx)) and then share the buf1 struct object to userspace in this trace.
I'm trying to send a struct from user-space to my module in kernel space using netlink, my struct in the user-space is:
struct test{
unsigned int length;
char name[MAX_NAME_LENGTH];
};
and in the kernel space is:
struct test{
__u32 length;
char name[MAX_NAME_LENGTH];
};
where MAX_NAME_LENGTH is a macro defined to be equal 50.
In the user-space, I've the function main which send my struct to the kernel with the following code:
int main(){
struct iovec iov[2];
int sock_fd;
struct sockaddr_nl src_add;
struct sockaddr_nl dest_add;
struct nlmsghdr * nl_hdr = NULL;
struct msghdr msg;
struct test message;
memset(&message, 0, sizeof(struct test));
message.length = 18;
strcpy(message.name, "Just a test\0");
sock_fd = socket(PF_NETLINK, SOCK_RAW, NETLINK_USER);
if (sock_fd < 0){
printf("Netlink socket creation failed\n");
return -1;
}
memset(&src_add, 0, sizeof(src_add));
src_add.nl_family = AF_NETLINK;
src_add.nl_pid = getpid();
memset(&dest_add, 0, sizeof(dest_add));
dest_add.nl_family = AF_NETLINK;
dest_add.nl_pid = 0; // Send to linux kernel
dest_add.nl_groups = 0; // Unicast
bind(sock_fd,(struct sockaddr *)&src_add,sizeof(src_add));
nl_hdr = (struct nlmsghdr *) malloc(NLMSG_SPACE(sizeof(struct test)));
memset(nl_hdr, 0, NLMSG_SPACE(sizeof (struct test)));
nl_hdr->nlmsg_len = NLMSG_SPACE(sizeof(struct test));
nl_hdr->nlmsg_pid = getpid();
nl_hdr->nlmsg_flags = 0;
iov[0].iov_base = (void *)nl_hdr;
iov[0].iov_len = nl_hdr->nlmsg_len;
iov[1].iov_base = &message;
iov[1].iov_len = sizeof(struct test);
memset(&msg,0, sizeof(msg));
msg.msg_name = (void *)&dest_add;
msg.msg_namelen = sizeof(dest_add);
msg.msg_iov = &iov[0];
msg.msg_iovlen = 2;
sendmsg(sock_fd,&msg,0);
close(sock_fd);
return 0;
}
And in the kernel side I've registered a function called callback to be called every time that a message is received, this is the callback function:
static void callback(struct sk_buff *skb){
struct nlmsghdr *nl_hdr;
struct test * msg_rcv;
nl_hdr = (struct nlmsghdr*)skb->data;
msg_rcv = (struct test*) nlmsg_data(nl_hdr);
printk(KERN_INFO "Priting the length and name in the struct:%u, %s\n",msg_rcv->length, msg_rcv->name);
}
When I run these codes and see the dmesg output I receive the following message: Priting the length and name in the struct:0,, so why the fields of the struct filled in the user-space side aren't being sent to the kernel?
Btw, NETLINK_USER is defined as 31.
DON'T DO THAT. YOUR CODE HAS BUGS BY DESIGN.
I'm going to first explain the one superfluous issue that prevents your code from doing what you want, then explain why what you want is a bad idea, then explain the right solution.
1. Doing what you want
You "want" to send a packet consisting of a netlink header followed by a struct. In other words, this:
+-----------------+-------------+
| struct nlmsghdr | struct test |
| (16 bytes) | (54 bytes) |
+-----------------+-------------+
The problem is that's not what you're telling your iovec. According to your iovec code, the packet looks like this:
+-----------------+--------------+-------------+
| struct nlmsghdr | struct test | struct test |
| (16 bytes) | (54 bytes) | (54 bytes) |
| (data) | (all zeroes) | (data) |
+-----------------+--------------+-------------+
This line:
iov[0].iov_len = nl_hdr->nlmsg_len;
Should be this:
iov[0].iov_len = NLMSG_HDRLEN;
Because your first iovec slot is just the Netlink header; not the whole packet.
2. Why what you want is bad
C has a gotcha called "data structure padding." Don't skip this lecture; I'd argue that anyone who deals with the C language MUST read it ASAP: http://www.catb.org/esr/structure-packing/
The gist of it is that C compilers are allowed to introduce garbage between the members of any structure. Thus, when you declare this:
struct test {
unsigned int length;
char name[MAX_NAME_LENGTH];
};
The compiler is technically allowed to mutate that during implementation into something like
struct test {
unsigned int length;
unsigned char garbage[4];
char name[MAX_NAME_LENGTH];
};
See the problem? If your kernel module and your userspace client were generated by different compilers, or by the same compiler but with slightly different flags, or even by slightly different versions of the same compiler, the structures might differ and the kernel will receive garbage, no matter how correct your code looks.
Update: Someone asked me to elaborate on that, so here it goes:
Suppose you have the following structure:
struct example {
__u8 value8;
__u16 value16;
};
In userspace, the compiler decides to leave it as is. However, in kernelspace the compiler "randomly" decides to convert it to:
struct example {
__u8 value8;
__u8 garbage;
__u16 value16;
};
In your userspace client, you then write this code:
struct example x;
x.value8 = 0x01;
x.value16 = 0x0203;
In memory, the structure will look like this:
01 <- value8
02 <- First byte of value16
03 <- Second byte of value16
When you send that to the kernel, the kernel will, of course, receive the same thing:
01
02
03
But it will interpret it differently:
01 <- value8
02 <- garbage
03 <- First byte of value16
junk <- Second byte of value16
(End of Update)
In your case the problem is aggravated by the fact that you define test.length as unsigned int in userspace, yet for some reason you change it into __u32 in kernelspace. Your code is problematic even before structure padding; if your userspace defines basic integers as 64-bit, the bug will also inevitably trigger.
And there's another problem: "Btw, NETLINK_USER is defined as 31" tells me you're following tutorials or code samples long obsolete or written by people who don't know what they are doing. Do you know where that 31 comes from? It's the identifier of your "Netlink family." They define it as 31 because that's the highest possible value it can have (0-31), and therefore, it's the most unlikely one to collide with other Netlink families defined by the kernel. (Because they are numbered monotonically.) But most careless Netlink users are following the tutorials, and therefore most of their Netlink families identify as 31. Therefore, your kernel module will be unable to coexist with any of them. netlink_kernel_create() will kick you out because 31 is already claimed.
And you might be wondering, "well shit. There are only 32 available slots, 23 of them are already taken by the kernel and there's an unknown but likely large number of additional people wanting to register different Netlink families. What do I do?!"
3. The proper way
It's 2020. We don't use Netlink anymore. We use better-Netlink: Generic Netlink.
Generic Netlink uses strings and dynamic integers as family identifiers, and drives you to use Netlink's "attribute" framework by default. (The latter encourages you to serialize and deserialize structures in a portable way, which is the real solution to your original problem.)
This code needs to be visible to both your userspace client and kernel module:
#define SAMPLE_FAMILY "Sample Family"
enum sample_operations {
SO_TEST, /* from your "struct test" */
/* List more here for different request types. */
};
enum sample_attribute_ids {
/* Numbering must start from 1 */
SAI_LENGTH = 1, /* From your test.length */
SAI_NAME, /* From your test.name */
/* This is a special one; don't list any more after this. */
SAI_COUNT,
#define SAI_MAX (SAI_COUNT - 1)
};
This is the kernel module:
#include <linux/module.h>
#include <linux/version.h>
#include <net/genetlink.h>
#include "../include/protocol.h"
/*
* A "policy" is a bunch of rules. The kernel will validate the request's fields
* match these data types (and other defined constraints) for us.
*/
struct nla_policy const sample_policy[SAI_COUNT] = {
[SAI_LENGTH] = { .type = NLA_U32 },
[SAI_NAME] = { .type = NLA_STRING },
};
/*
* This is the function the kernel calls whenever the client sends SO_TEST
* requests.
*/
static int handle_test_operation(struct sk_buff *skb, struct genl_info *info)
{
if (!info->attrs[SAI_LENGTH]) {
pr_err("Invalid request: Missing length attribute.\n");
return -EINVAL;
}
if (!info->attrs[SAI_NAME]) {
pr_err("Invalid request: Missing name attribute.\n");
return -EINVAL;
}
pr_info("Printing the length and name: %u, '%s'\n",
nla_get_u32(info->attrs[SAI_LENGTH]),
(unsigned char *)nla_data(info->attrs[SAI_NAME]));
return 0;
}
static const struct genl_ops ops[] = {
/*
* This is what tells the kernel to use the function above whenever
* userspace sends SO_TEST requests.
* Add more array entries if you define more sample_operations.
*/
{
.cmd = SO_TEST,
.doit = handle_test_operation,
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 2, 0)
/* Before kernel 5.2, each op had its own policy. */
.policy = sample_policy,
#endif
},
};
/* Descriptor of our Generic Netlink family */
static struct genl_family sample_family = {
.name = SAMPLE_FAMILY,
.version = 1,
.maxattr = SAI_MAX,
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 2, 0)
/* Since kernel 5.2, the policy is family-wide. */
.policy = sample_policy,
#endif
.module = THIS_MODULE,
.ops = ops,
.n_ops = ARRAY_SIZE(ops),
};
/* Called by the kernel when the kernel module is inserted */
static int test_init(void)
{
return genl_register_family(&sample_family);
}
/* Called by the kernel when the kernel module is removed */
static void test_exit(void)
{
genl_unregister_family(&sample_family);
}
module_init(test_init);
module_exit(test_exit);
And here's the userspace client (You need to install libnl-genl-3 --sudo apt install libnl-genl-3-dev on Debian/Ubuntu):
#include <errno.h>
#include <netlink/genl/ctrl.h>
#include <netlink/genl/genl.h>
#include "../include/protocol.h"
static struct nl_sock *sk;
static int genl_family;
static void prepare_socket(void)
{
sk = nl_socket_alloc();
genl_connect(sk);
genl_family = genl_ctrl_resolve(sk, SAMPLE_FAMILY);
}
static struct nl_msg *prepare_message(void)
{
struct nl_msg *msg;
msg = nlmsg_alloc();
genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, genl_family, 0, 0, SO_TEST, 1);
/*
* The nla_put* functions ensure that your data will be stored in a
* portable way.
*/
nla_put_u32(msg, SAI_LENGTH, 18);
nla_put_string(msg, SAI_NAME, "Just a test");
return msg;
}
int main(int argc, char **argv)
{
struct nl_msg *msg;
prepare_socket();
msg = prepare_message();
nl_send_auto(sk, msg); /* Send message */
nlmsg_free(msg);
nl_socket_free(sk);
return 0;
}
This code should work starting from kernel 4.10. (I tested it in 4.15.) The kernel API was somewhat different before that.
I left a pocket version of my test environment (with makefiles and proper error handling and everything) in my Dropbox, so you can run it easily.
I'm at exercise 17 of Learn C the hard way,which require building a database and store it in a FILE.I have initialized a database as expected, but when I increased the number of rows (particularly over the threshold of 100) in the database, it returned
Exception: STATUS_ACCESS_VIOLATION at rip=0010040132C.
I used GDB to search for the error, and here's the result.
Thread 1 "ex17" received signal SIGSEGV, Segmentation fault.
0x000000010040132c in Database_create (conn=0x600049490, max_data=12, >max_rows=200) at ex17_1.c:87
87 (conn->db->rows +isizeof(struct Address)) =addr;
Here's the code I have used.
struct Address{
int id;
int set;
char *name;
char *email;
};
struct Database{
int MAX_ROWS;
int MAX_DATA;
struct Address *rows;
};
struct Connection{
FILE *file;
struct Database *db;
};
void Database_create(struct Connection *conn,int max_data,int max_rows){
conn->db->MAX_DATA =max_data;
conn->db->MAX_ROWS = max_rows;
conn->db->rows =malloc(max_rows*sizeof(struct Address));
for(int i=0;i<max_rows;i++){
struct Address addr = {.id =i,.set = 0};
*(conn->db->rows +i*sizeof(struct Address)) =addr;
}
}
I have done some research and I think that STATUS_ACCESS_VIOLATION occurs when
you access a part of memory you are not supposed to. But I have not seen the error in my code yet.
Can someone check what is the possible reason here?
The error is on this line
+i*sizeof(struct Address)
The compiler already multiplies by the size when adding to pointers so don't do it again. You want only
+i
I am trying to send an ethernet packet using RAW socket in C Linux. I have following struct definition in my code:
typedef struct vlink_header_s
{
uint8_t verCmd;
uint8_t reverseVerCmd;
}vlink_header_t;
typedef struct vlink_reg_rd_s
{
vlink_header_t header;
uint32_t address;
uint16_t length;
}vlink_reg_rd_t;
In main i created a struct:
vlink_reg_rd_t g_pkt;
g_pkt.header.verCmd = 0x10|VLINK_CMD_REG_RD;
g_pkt.header.reverseVerCmd = ~(g_pkt.header.verCmd);
g_pkt.address = 0x0007 .....
and message:
char sendbuf[1024];
struct ether_header *eh = (struct ether_header *) sendbuf;
how do I add all the info from the struct g_pkt to this sendbuf after ether_header so I can send a complete packet using:
sendto(sockfd, sendbuf, txLen, 0, (struct sockaddr*)&socket_address, sizeof(struct sockaddr_ll));
Everything else in my code is working, I tried other ways by adding info to sendbuf one by one and it works fine and my machine receive the packets on the other side too. I just want to make it more versatile because there are a bunch of commands and structs for each commands will work best. Thanks.
Try to use the memcpy function:
#include<iostream>
typedef struct vlink_header_s
{
uint8_t verCmd;
uint8_t reverseVerCmd;
}vlink_header_t;
typedef struct vlink_reg_rd_s
{
vlink_header_s header;
uint32_t address;
uint16_t length;
}vlink_reg_rd_t;
using namespace std;
int main()
{
vlink_reg_rd_t data;
//TODO: Set values into typedef data.
int size = sizeof(vlink_reg_rd_t); //get
char* buffer = new char[size];
memset(buffer, 0x00, size);
memcpy(buffer, &data, size); //Copy data from vlink_reg_rd_t to char*
//TODO: Send the buffer.
delete[] buffer; //free memory
return 0;
}
IMPORTANT: be aware of the order in which data types number are written into the buffer. Also it is necessary check the align the data into the structure to avoid extra bytes at the moment of use memcpy. Here you can check this topic:
for Microsoft:
https://msdn.microsoft.com/en-us/library/xh3e3fd0.aspx
https://msdn.microsoft.com/en-us/library/83ythb65.aspx
For Gcc:
https://gcc.gnu.org/onlinedocs/gcc-3.3/gcc/Type-Attributes.html
I have done this way with protocol buffer , you can take a look at : https://www.google.com.vn/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=protocol+buffer&* .
Edit : this way called serialize data , as you serialize your data into a proto file then compile it to the packet file that you use on both server and client .
I'm having problems with a client-server communication made with writev()/readv().
I have two struct, header and data defined like this:
typedef struct {
int op;
int key;
} message_hdr_t;
typedef struct {
int len;
char *data;
} message_data_t;
The server does (in short):
message_hdr_t h = {1, 11};
message_data_t d;
d.len = 3;
strcpy(d.data, "msg");
struct iovec tosend[2];
tosend[0].iov_base = &h;
tosend[0].iov_len = sizeof(message_hdr_t);
tosend[1].iov_base = &d;
tosend[1].iov_len = sizeof(message_data_t);
writev(socket, tosend, 2);
close(socket);
The client (in short):
struct iovec received[2];
readv(socket, received, 2);
message_hdr_t header;
header.op = ((message_hdr_t *) received[0].iov_base)->op;
header.key = ((message_hdr_t *) received[0].iov_base)->key;
printf("Received op: %i, key: %i\n",header.op,header.key;
close(socket);
But the client gets a segfault because received[0].iov_base is NULL. Why?
The socket is correctly opened and the client is correctly connected to the server. It's an AF_UNIX socket.
First, in your server code, you are writing a pointer. This makes no sense. You don't want to transmit pointers over the wire. To transmit a string, you have to do something like this:
char* message = ...;
message_hdr_t h = {1, 11};
uint32_t message_length = strlen(message);
struct iovec tosend[3];
tosend[0].iov_base = &h;
tosend[0].iov_len = sizeof(message_hdr_t);
tosend[1].iov_base = &message_length;
tosend[1].iov_len = sizeof(message_length);
tosend[2].iov_base = message;
tosend[2].iov_len = message_length;
(You may want to move the string length to the message header and save one element of the vector, and make the protocol more readable).
Second, readv won't allocate memory for you, or divine out how many bytes you want to read. It's your job to correctly initialize iov_base and iov_len in the IO vector passed to readv. In order to read a dynamically-allocated variable-size string, you probably want to read twice. First, read a part of the message that contains the length of the string, then allocate the string, and read the rest of the message.