I've been running some code under UBSan, and found an error which I've never seen before:
/usr/include/c++/7/bits/stl_algobase.h:324:8: runtime error: store to misaligned address 0x611000001383 for type 'struct complex', which requires 4 byte alignment
0x611000001383: note: pointer points here
66 46 40 02 00 00 00 00 00 00 00 00 04 01 18 00 08 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00
^
(g++-7.3.0, Ubuntu 18.04, flags -fsanitize=address -fsanitize=undefined)
What does this error mean? Is it truly an error (it is in the standard library, so it can't be too bad, right?), and should I care about it?
You probably use a pointer cast which casts a block of raw memory to a complex*.
Example:
void* raw = getBuffer(); // Made up function which returns a buffer
auto size = *static_cast<uint16_t>*(raw); // Maybe your format says that you got a 2 Byte size in front
auto* array = static_cast<complex*>(raw+sizeof(uint16_t)); // ... and complex numbers after
std::transform(array, array+size, ...); // Pass this into STL
Boom! You got UB.
Why?
The behavior is undefined in the following circumstances: [...]
Conversion between two pointer types produces a result that is incorrectly aligned
[...]
If the resulting pointer is not correctly aligned [68] for the referenced type, the behavior is undefined.
See https://stackoverflow.com/a/46790815/1930508 (where I got these from)
What does it mean?
Every pointer must be aligned to the type it is pointing to. For complex this means an alignment of 4. In short this means that array (from above) must be evenly divisible by 4 (aka array % 4 == 0) Assuming that raw is aligned to 4 bytes you can easily see that array cannot as (raw + 2) % 4 == 2 (because of raw % 4 == 2)
If the size would be a 4-Byte value, then array would have been aligned if (and only if) raw was aligned. Whether this is guaranteed depends on where it comes from.
So yes this is truly an error and may lead to a real bug although not always (depending on moon phase etc. as it is always with UB, see the answer above for details)
And no it is NOT in the STL, it just happens to be detected there because UBSAN watches memory dereferences. So while the actual UB is the static_cast<complex*> it is only detected when reading from that pointer.
You can use export UBSAN_OPTIONS=print_stacktrace=1 prior to executing the program to get a stacktrace and find out where your wrong cast is.
Tip: You only need to check casts. Any struct/type allocated via new is always aligned (and every member inside), unless tricks like "packed structs" are used.
Related
tl;dr : I'm trying to execute dynamically some code from another snippet. But I am stuck with handling memory reference (e.g. mov 40200b, %rdi): can I patch my code or the snippet running code so that 0x40200b is resolved correctly (as the offset 200b from the code)?
To generate the code to be executed dynamically I start from a (kernel) object and I resolve the references using ld.
#!/usr/bin/python
import os, subprocess
if os.geteuid() != 0:
print('Run this as root')
exit(-1)
with open("/proc/kallsyms","r") as f:
out=f.read()
sym= subprocess.Popen( ['nm', 'ebbchar.ko', '-u' ,'--demangle', '-fposix'],stdout=subprocess.PIPE)
v=''
for sym in sym.stdout:
s = " "+ sym.split()[0]+ "\n"
off = out.find(s)
v += "--defsym "+s.strip() + "=0x" +out[off-18:off -2]+" "
print(v)
os.system("ld ebbchar.ko "+ v +"-o ebbchar.bin");
I then transmit the code to be executed with through a mmaped file
int fd = open(argv[1], O_RDWR | O_SYNC);
address1 = mmap(NULL, page_size, PROT_WRITE|PROT_READ , MAP_SHARED, fd, 0);
int in=open(argv[2],O_RDONLY);
sz= read(in, buf+8,BUFFER_SIZE-8);
uint64_t entrypoint=atol(argv[3]);
*((uint64_t*)buf)=entrypoint;
write(fd, buf, min(sz+8, (size_t) BUFFER_SIZE));
I execute code dynamycally with this code
struct mmap_info *info;
copy_from_user((void*)(&info->offset),buf,8);
copy_from_user(info->data, buf+8, sz-8);
unsigned long (*func)(void) func= (void*) (info->data + info->offset);
int ret= func();
This approch work for code that don't access memory such as "\x55\x48\x89\xe5\xc7\x45\xf8\x02\x00\x00\x00\xc7\x45\xfc\x03\x00\x00\x00\x8b\x55\xf8\x8b\x45\xfc\x01\xd0\x5d\xc3" but I have problems when memory is involved.
See example below.
Let's assume i wan't execute dynamically the function vm_close. Objdump -d -S returns:
0000000000401017 <vm_close>:
{
401017: e8 e4 07 40 81 callq ffffffff81801800 <__fentry__>
printk(KERN_INFO "vm_close");
40101c: 48 c7 c7 0b 20 40 00 mov $0x40200b,%rdi
401023: e9 b6 63 ce 80 jmpq ffffffff810e73de <printk>
At execution, my function pointer points to the right code:
(gdb) x/12x $rip
0xffffc90000c0601c: 0x48 0xc7 0xc7 0x0b 0x20 0x40 0x00 0xe9
0xffffc90000c06024: 0xb6 0x63 0xce 0x80
(gdb) x/2i $rip
=> 0xffffc90000c0601c: mov $0x40200b,%rdi
0xffffc90000c06023: jmpq 0xffffc8ff818ec3de
BUT, this code will fail since:
1) In my context $0x40200b points at the physical address $0x40200b, and not offset 200b from the beginning of the code.
2) I don't understand why but the address displayed there is actually different from the correct one (0xffffc8ff818ec3de != ffffffff810e73de) so it won't point on my symbol and will crash.
Is there a way to solve my 2 issues?
Also, I had trouble to find good documentation related to my issue (low-level memory resolution), if you could give me some, that would really help me.
Edit: Since I run the code in the kernel I cannot simply compile the code with -fPIC or -fpie which is not allowed by gcc (cc1: error: code model kernel does not support PIC mode)
Edit 24/09:
According to #Peter Cordes comment, I recompiled it adding mcmodel=small -fpie -mno-red-zone -mnosse to the Makefile (/lib/modules/$(uname -r)fixed/build/Makefile)
This is better than in the original version since the generated code before linking is now:
0000000000000018 <vm_close>:
{
18: ff 15 00 00 00 00 callq *0x0(%rip) # 1e <vm_close+0x6>
printk(KERN_INFO "vm_close");
1e: 48 8d 3d 00 00 00 00 lea 0x0(%rip),%rdi # 25 <vm_close+0xd>
25: e8 00 00 00 00 callq 2a <vm_close+0x12>
}
2a: c3 retq So thanks to rip-relative addressing
Thus I’m now able to access the other variables on my script…
Thus, after linking I can successfully access my variable embedded within the buffer.
40101e: 48 8d 3d e6 0f 00 00 lea 0xfe6(%rip),%rdi # 40200b
Still, one problem remains:
The symbol I want to access (printk) and my executable buffer are in different address spaces, for exemple:
printk=0xffffffff810e73de:
Executable_buffer=0xffffc9000099d000
But in my callq to printk, I have only 32 bits to write the address to call as an offset from $rip since there is no .got section in the kernel. This means that printk has to be located within [$rip-2GO, $rip+2GO]. But this is not the case there.
Do I have a way to access the printk address although they are located more than 2GO away from my buffer (I tried to used mcmodel=medium but I haven't seen any difference in the generated code), for instance by modifying gcc options so that the binary actually have a .got section?
Or is there a reliable way to force my executable and potentially-too large-for-kmalloc buffer to be allocated in the [0xffffffff00000000 ; 0xffffffffffffffff] range? (I currently use __vmalloc(BUFFER_SIZE, GFP_KERNEL, PAGE_KERNEL_EXEC); )
Edit 27/09:
I succedded in allocationg my buffer in the [0xffffffff00000000 ; 0xffffffffffffffff] range using the non exported __vmalloc_node_range function as a (dirty) hack.
IMPORTED(__vmalloc_node_range)(BUFFER_SIZE, MODULE_ALIGN,
MODULES_VADDR + get_module_load_offset(),
MODULES_END, GFP_KERNEL,
PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE,
__builtin_return_address(0));
Then, when I know the address of my executable buffer and the address of the kernel symbols (by parsing /proc/kallsyms), I can patch my binary using ld’s option --defsym symbol=relative_address where relative_address = symbol_address - buffer_offset .
Despite being extremely dirt, this approach actually works.
But I need to relink my binary each time I execute it since the buffer may (and will) be allocated at a different address. To solve this issue, I think the best way would be to build my executable as a real position independent executable so that I can just patch the global offset table and not fully relink the module.
But with the options provided there I got a rip-relative address but no got/plt. So I'd like to find a way to build my module as a proper PIE.
This post is getting huge, messy and we are deviating from the original question. Thus, I opened a new simplified post there. If I get interesting answers, I'll edit this post to explain them.
Note: For the sake of simplicity, safety tests are not displayed there
Note 2: I am perfectly aware that my PoC is very unusual and can be a bad practice but I'd like to do it anyway.
I have an f77 unformatted binary file.
I know that the file contains 2 floats and a long integer as well as data.
The size of the file is 536870940 bytes which should include 512^3 float data values together with the 2 floats and the long integer.
The 512^3 float data values make up 536870912 bytes leaving a further 28 bytes.
My problem is that I need to work out where the 28 bytes begins and how to skip this amount of storage so that I can directly access the data.
I prefer to use C to access the file.
Unfortunately, there is no standard what unformatted means. But some methods are more common than others.
In many Fortran versions I have used, every write command writes a header (often unsigned int 32) of how many bytes the data is, then the data, then repeats the header value in case you're reading from the rear.
From the values you have provided, it might be that you have something like this:
uint32(record1 header), probably 12.
float32, float32, int32 (the three 'other values' you talked about)
uint32(record1 header, same as first value)
uint32(record2 header, probably 512^3*4)
float32*512^3
uint32(record2 header, same as before)
You might have to check endianness.
So I suggest you open the file in a hexdump program, and check whether bytes 0-3 are identical to bytes 16-19, and whether bytes 20-23 are repeated at the end of the data again.
If that is the case, I'll try to check the endianness to see whether the values are little or big endian, and with a little luck you'll have your data.
Note: I assume that these three other values are metadata about the data, and therefore would be at the beginning of the file. If that's not the case, you might have them at the end.
Update:
In your comment, you write that your data begins with something like this:
0C 00 00 00 XX XX XX XX XX XX XX XX XX XX XX XX 0C 00 00 00
^- header-^ ^-header -^
E8 09 FF 1F (many, many values) E8 09 FF 1F
^- header-^ ^--- your data ---^ ^-header -^
Now I don't know how to read data in C. I leave this up to you. What you need to do is skip the first 24 bytes, then read the data as (probably little endian) 4-byte floating values. You will have 4 bytes left that you don't need any more.
Important note:
Fortran stores arrays column-major, C afaik stores them row-major. So keep in mind that the order of the indices will be reversed.
I know how to read this in Python:
from scipy.io import FortranFile
ff = FortranFile('data.dat', 'r', '<u4')
# read the three values you are not interested in
threevals = ff.read_record('<u4')
# read the data
data = ff.read_record('<f4')
ff.close()
I am getting error when i am trying to dump a packet in pcap file.
{
unsigned char *ofilename = "packet.pcap";
pcap_t *fp;
pcap_dumper_t *dumpfile;
const struct pcap_pkthdr *header;
fp = pcap_open_dead(DLT_RAW,256);
if(fp != NULL)
{
dumpfile = pcap_dump_open(fp, ofilename);
if(dumpfile == NULL)
{
printf("\nError opening output file\n");
return;
}
pcap_dump((u_char *)dumpfile,header,data);
pcap_close(fp);
pcap_dump_close(dumpfile);
}
}
HERE data is a u8 data[256].. its 256 byte data.. which has the packet bytes like this
FF FF FF FF FF FF 00 50 56 A8 11 39 81 00 0F FC 81 00 1F FC 08 06 00 01 08 00 06 04 00 01 00 50 56 A8 11 39 65 2B 01 0A 00 00 00 00 00 00 65 2B
But when i open packet.pcap i am getting "The capture file appears to be damaged or corrupt. (pcap: File has 1847605831-byte packet, bigger than maximum of 65535)"
Could someone pls help me on this whats going wrong
Kindly install "pcapfix" on Linux and run it on the corrupt file as follows
$ pcapfix -d 'file / file path here'
This will fix it.
Try something such as
{
unsigned char *ofilename = "packet.pcap";
pcap_t *fp;
pcap_dumper_t *dumpfile;
struct pcap_pkthdr header;
fp = pcap_open_dead(DLT_RAW,256);
if(fp != NULL)
{
dumpfile = pcap_dump_open(fp, ofilename);
if(dumpfile == NULL)
{
printf("\nError opening output file\n");
return;
}
header.caplen = 256; /* or however many bytes actually contain packet data */
header.len = 256; /* or however many bytes actually contain packet data */
gettimefoday(&header.ts); /* I'm assuming this is on some flavor of UN*X */
pcap_dump((u_char *)dumpfile,&header,data);
pcap_close(fp);
pcap_dump_close(dumpfile);
}
}
For one thing, just because a function takes an argument of type "{something} *", that doesn't mean you should pass to it a variable of type "{something} *". You must pass it a value of type "{something} *", but it must be a valid value, i.e. it must point to something.
An uninitialized variable of type "{something} ``*", which is what you have in your code, doesn't point to anywhere valid.
However, if you declare a variable of type "{something}", rather than "{something} *", you can use the & operator on that variable to get a value of type "{something} *" that points to the variable.
Then, as indicated, you have to give that variable a value if you're passing it to pcap_dump(). You have to set the len and caplen members of a struct pcap_pkthdr; the caplen member must be equal to the actual number bytes of packet data (which might be less than the size of the array if the packet isn't, in your case, exactly 256 bytes long), and the len member must be at least that value; len would only be bigger than caplen if the packet came from a capture done with a "snapshot length" value that discarded everything in the packet past a certain point, which isn't the case here, so len should be equal to caplen.
You probably also want to set the time stamp of the packet; I'm assuming you're running on some form of UN*X here, so you can use gettimeofday() to get the current time. If this is Windows with WinPcap, you'll probably have to do something else.
(header must not be const here, as you have to set it. It doesn't have to be const; it's const in the declaration of pcap_dump(), but that just means that pcap_dump() won't change it, so you can pass it a pointer to something that's const; you don't have to pass it something that'sconst`.)
I am missing a fundamental point on passing values.
In my code, I wrote this prototype/function:
void drawFont (char A[],unsigned char length, char x1, char y1, uint16 FGcolor);
I call the function using a call like this:
drawFont ("William",7,15,25,YEL,0);
or
drawFont ("W",1,15,25,YEL,0);
Both of these work fine in the code. If I examine A[0] in the function, I will see a value of '57' representing an ASCII 'W'. All fine and good.
My question/problem is: When I attempt to replicate the 'W' using the ASCII value instead of the string representation, my code fails Example:
drawFont (57,1,15,25,YEL,0);
The value of A[0] in the code is: 0, but the address of A is 57. So somehow the compiler assumes that I want to pass a pointer? I'm confused, why did it pass the string fine, but not the value?
Thanks in advance from a novice C programmer.
If you pass
drawFont (57,1,15,25,YEL,0);
you will be doming the same as
drawFont ('W',1,15,25,YEL,0);
which is just one character. The function requires an array of characters, that's why it's not correct.
The prototype for your function declares that the first parameter is a pointer to a character:
void drawFont (char A[],...);
If you pass an integer, you are breaking the rules. The compiler should warn about it. Mine gives a warning about the following code on the line with func(57):
void func(char a[])
{
}
char arr[] = {87,84,70};
int main()
{
func(57);
func("W");
func(arr);
}
x.c(9) : warning C4047: 'function' : 'char *' differs in levels of indirection from 'int'
x.c(9) : warning C4024: 'func' : different types for formal and actual parameter 1
If you look at the assembly output from the compiler, you can see the problem more clearly (I cut out the uninteresting parts):
PUBLIC _arr
_DATA SEGMENT
_arr DB 057H ;Here's the char array at an address in memory.
DB 054H
DB 046H
ORG $+1
$SG1297 DB 'W', 00H ;Here's the "W" at an address in memory (nul-terminated)
_DATA ENDS
; 9 : func(57);
00003 6a 39 push 57 ; Push 57 as the address, WRONG!
00005 e8 00 00 00 00 call _func
0000a 83 c4 04 add esp, 4
; 10 : func("W");
0000d 68 00 00 00 00 push OFFSET $SG1297 ; Push the address of the string.
00012 e8 00 00 00 00 call _func
00017 83 c4 04 add esp, 4
; 11 : func(arr);
0001a 68 00 00 00 00 push OFFSET _arr ; Push the address of the array.
0001f e8 00 00 00 00 call _func
00024 83 c4 04 add esp, 4
The code is expecting to go to an address to read characters. Who knows what is in address 57.
The value of A[] is actually a pointer, such as &A[0] (you could rewrite the function using char *A instead of char A[]
That is the address of the first slot, not what is stored in it. In the memory location pointed to by the pointer, there exists a value 57. When you write a function with a char array or char pointer parameter, you tell the compiler: I will pass you a memory address. Go to that address and start reading each char (number) until you arrive at the null byte, then stop.
But when you pass literal integer 57 as the argument, that is not a valid pointer to an array of characters. You are missing the level of indirection.
So lets assume your char A[] resides at 0xfeaa0011
In RAM at 0xfeaa0011 -> "William" (or 52, ...)
The correct call to drawFont() actually looks like drawFont(0xfeaa0011, 1, 15, 25, YEL, 0)
So you see replacing the pointer with 57 isn't the same thing.
Your function requires a string ( char[] ) as an argument.
When you pass "William" it is interpreted as a string, that's normal.
When you pass "W" it is also interpreted as a string:
string[2] = { 'W' + 0 } //notice the null byte following your char
That's because you used double-quotes " instead of single-quotes '.
Therefore passing 57 or 'W' is viewed as a single char and not a char[]/string which then gives you an error since your function is expecting a char[] and you gave it a char.
I have a 256-bit value in Verilog:
reg [255:0] val;
I want to define a system task $foo that calls out to external C using the VPI, so I can call $foo like this:
$foo(val);
Now, in the C definition for the function 'foo', I cannot simply read the argument as an integer (PLI_INT32), because I have too many bits to fit in one of those. But, I can read the argument as a string, which is the same thing as an array of bytes. Here is what I wrote:
static int foo(char *userdata) {
vpiHandle systfref, args_iter, argh;
struct t_vpi_value argval;
PLI_BYTE8 *value;
systfref = vpi_handle(vpiSysTfCall, NULL);
args_iter = vpi_iterate(vpiArgument, systfref);
argval.format = vpiStringVal;
argh = vpi_scan(args_iter);
vpi_get_value(argh, &argval);
value = argval.value.str;
int i;
for (i = 0; i < 32; i++) {
vpi_printf("%.2x ", value[i]);
}
vpi_printf("\n");
vpi_free_object(args_iter);
return 0;
}
As you can see, this code reads the argument as a string and then prints out each character (aka byte) in the string. This works almost perfectly. However, the byte 00 always gets read as 20. For example, if I assign the Verilog reg as follows:
val = 256'h000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f;
And call it using $foo(val), then the C function prints this at simulation time:
VPI: 20 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f
I have tested this with many different values and have found that the byte 00 always gets mapped to 20, no matter where or how many times it appears in val.
Also, note that if I read the value in as a vpiHexStrVal, and print the string, it looks fine.
So, two questions:
Is there a better way to read in my 256-bit value from the Verilog?
What's going on with the 20? Is this a bug? Am I missing something?
Note: I am using Aldec for simulation.
vpiStringVal is used when the value is expected to be ASCII text, in order to get the value as a pointer to a C string. This is useful if you want to use it with C functions that expect a C string, such as printf() with the %s format, fopen(), etc. However, C strings cannot contain the null character (since null is used to terminate C strings), and also cannot represent x or z bits, so this is not a format that should be used if you need to distinguish any possible vector value. It looks like the simulator you are using formats the null character as a space (0x20); other simulators just skip them, but that doesn't help you either. To distinguish any possible vector value use either vpiVectorVal (the most compact representation) or vpiBinStrVal (a binary string with one 0/1/x/z character for each bit).