I am wondering how to properly double buffer the framebuffer to avoid tearing. I've done lots of research on this topic and I can't seem to find anything.
I've tried FBIO_WAITFORVSYNC. But per this thread: How to query Vsync phase in Linux it seems that this won't work.
I've also tried using FBIOGET_VSCREENINFO
and FBIOPAN_DISPLAY per this thread: Linux framebuffer graphics and VSync. But it failed due to an error discussed in this thread: invalid argument error when setting yres_virtual in fb_var_screeninfo
That thread suggested using a different driver (vesafb) to resolve the error. I managed to install uvesafb on my machine but the "Invalid Argument" error didn't go away.
I've also tried just mmaping a larger buffer per this person's suggestion: http://betteros.org/tut/graphics1.php#doublebuffer but mmap keeps returning -1.
I've also tried to implement the solution discussed here: https://pyra-handheld.com/boards/threads/my-frustrating-experiences-with-dev-fb.21062/. However the thread died without the actual solution being posted and I am doubtful of the efficiency of swapping hardware addresses (or if it can even be done).
Any help would be so greatly appreciated on this topic!
Due to a request here is the code I would ideally like to get to work:
fb0 = open("/dev/fb0", O_RDWR);
if(fb0 == 0)
error("Could not open framebuffer located in /dev/fb0!");
if (ioctl(fb0, FBIOGET_FSCREENINFO, &screeninfo_fixed) == -1)
error("Could not retrive fixed screen info!");
if (ioctl(fb0, FBIOGET_VSCREENINFO, &screeninfo_var) == -1)
error("Could not retrive variable screen info!");
screeninfo_var.xres_virtual = screeninfo_var.xres;
screeninfo_var.yres_virtual = screeninfo_var.yres * 2;
screeninfo_var.width = screeninfo_var.xres;
screeninfo_var.height = screeninfo_var.yres;
screeninfo_var.xoffset = 0;
screeninfo_var.yoffset = 0;
if (ioctl(fb0, FBIOPUT_VSCREENINFO, &screeninfo_var) == -1)
error("Could not set variable screen info!");
This will always print "Could not set variable screen info!" due to some issue extending the virtual framebuffer size.
The way I have solved the double buffering issue for framebuffer on linux is by using a separate back buffer, allocated using mmap for the same screensize:
bbp = mmap(0, screensize, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
(where screensize = vinfo.yres_virtual * finfo.line_length;)
Then write all changes to this back buffer, and after all writes are over, copy the entire back buffer to the main framebuffer pointer location using say, an update() function:
memcpy(fbp, bbp, screensize);
You could, also have an updateRect() function that copies only a given area. This logic is working fine on my x64 PC gnu/linux platform.
Related
I've implemented a char device for my kernel module and implemented a read function for it. The read function calls copy_to_user to return data to the caller. I've originally implemented the read function in a blocking manner (with wait_event_interruptible) but the problem reproduces even when I implement read in a non-blocking manner. My code is running on a MIPS procesor.
The user space program opens the char device and reads into a buffer allocated on the stack.
What I've found is that occasionally copy_to_user will fail to copy any bytes. Moreover, even if I replace copy_to_user with a call to memcpy (only for the purposes of checking... I know this isn't the right thing to do), and print out the destination buffer immediately afterwards, I see that memcpy has failed to copy any bytes.
I'm not really sure how to further debug this - how can I determine why memory is not being copied? Is it possible that the process context is wrong?
EDIT: Here's some pseudo-code outlining what the code currently looks like:
User mode (runs repeatedly):
char buf[BUF_LEN];
FILE *f = fopen(char_device_file, "rb");
fread(buf, 1, BUF_LEN, f);
fclose(f);
Kernel mode:
char_device =
create_char_device(char_device_name,
NULL,
read_func,
NULL,
NULL);
int read_func(char *output_buffer, int output_buffer_length, loff_t *offset)
{
int rc;
if (*offset == 0)
{
spin_lock_irqsave(&lock, flags);
while (get_available_bytes_to_read() == 0)
{
spin_unlock_irqrestore(&lock, flags);
if (wait_event_interruptible(self->wait_queue, get_available_bytes_to_read() != 0))
{
// Got a signal; retry the read
return -ERESTARTSYS;
}
spin_lock_irqsave(&lock, flags);
}
rc = copy_to_user(output_buffer, internal_buffer, bytes_to_copy);
spin_unlock_irqrestore(&lock, flags);
}
else rc = 0;
return rc;
}
It took quite a bit of debugging, but in the end Tsyvarev's hint (the comment about not calling copy_to_user with a spinlock taken) seems to have been the cause.
Our process had a background thread which occasionally launched a new process (fork + exec). When we disabled this thread, everything worked well. The best theory we have is that the fork made all of our memory pages copy-on-write, so when we tried to copy to them, the kernel had to do some work which could not be done with the spinlock taken. Hopefully it at least makes some sense (although I'd have guessed that this would apply only to the child process, and the parent's process pages would simply remain writable, but who knows...).
We rewrote our code to be lockless and the problem disappeared.
Now we just need to verify that our lockless code is indeed safe on different architectures. Easy as pie.
I want to get RGB values of screen pixels in the most efficient way, using Linux. So I decided to use the framebuffer library in C (fb.h) to access the framebuffer device (/dev/fb0) and read from it directly.
This is the code:
#include <stdint.h>
#include <linux/fb.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
int main() {
int fb_fd;
struct fb_fix_screeninfo finfo;
struct fb_var_screeninfo vinfo;
uint8_t *fb_p;
/* Open the frame buffer device */
fb_fd = open("/dev/fb0", O_RDWR);
if (fb_fd < 0) {
perror("Can't open /dev/fb0\n");
exit(EXIT_FAILURE);
}
/* Get fixed info */
if (ioctl(fb_fd, FBIOGET_FSCREENINFO, &finfo) < 0) {
perror("Can't get fixed info\n");
exit(EXIT_FAILURE);
}
/* Get variable info */
if (ioctl(fb_fd, FBIOGET_VSCREENINFO, &vinfo) < 0) {
perror("Can't get variable info\n");
exit(EXIT_FAILURE);
}
/* To access to the memory, it can be mapped*/
fb_p = (uint8_t *) mmap(0, finfo.smem_len, PROT_READ | PROT_WRITE, MAP_SHARED, fb_fd, 0);
if (fb_p == MAP_FAILED) {
perror("Can't map memory\n");
exit(EXIT_FAILURE);
}
/* Print each byte of the frame buffer */
for (int i = 0; i < finfo.smem_len; i++) {
printf("%d\n", *(fb_p + i));
// for (int j = 0; j < 500000000; j++); /* Delay */
}
munmap(fb_p, 0);
close(fb_fd);
return 0;
}
But when I print the values, I don't get what I was expecting...
If I pick the RGB value of the pixel (0, 0) with a tool like grabc, I get this:
#85377e
133,55,126
But the first printings with my code are:
126
145
198
...
It looks like I am obtaining well the first value of the first pixel, corresponding to blue, but the rest is wrong.
If you look at the grabc source code (https://www.muquit.com/muquit/software/grabc/grabc.html), you'll see that he's querying X Windows (and NOT the hardware frame buffer per se).
root_window=XRootWindow(display,XDefaultScreen(display));
target_window=selectWindow(display,&x,&y);
...
ximage=XGetImage(display,target_window,x,y,1,1,AllPlanes,ZPixmap);
...
color->pixel=XGetPixel(ximage,0,0);
XDestroyImage(ximage);
For whatever it's worth - and for several different reasons - I'd strongly encourage you to consider doing the same.
You might also be interested in:
Intro to Low-Level Graphics on Linux
DirectFB
In some rare buggy device using fbdev you may need to align the mmap returned pointer to the next system page in order to get the first pixel correctly, and by doing so you may also need to mmap one more page.
I don't think that's your case, it seems like you're trying to get the xserver desktop pixels by using linux fbdev, unless your xserver is configured to use the fbdev driver (and even so I'm not sure would work) isn't gonna work.
If reading the desktop pixels is your target you should look into some screenshot tool.
Beside, fbdev drivers are usually really basic, is a low-level access but there's nothing efficient about mmapping and reading pixels, the Xlib or any higher level graphic system may be aware and support your graphic card accelerations and be far more efficient (perhaps using DMA to read the whole framebuffer to your system memory without loading the cpu).
Working on iOS there was a private framework iOMobileFrameBuffer and created a lovely recording software for the screen. I've dabbled in this before so I will give you what I know. iOS is obviously different than Linux FrameBuffer (lack of documentation is pretty similar...) but I would personally take a look at this answer. Although the question doesn't fit what you ask, that answer explains the PixelBuffer along with a small example of exactly what you asked. Give it a read and reply with any questions, I will answer to the best of my ability.
I have 2 applications running on my linux box, a server and a client. My server and client examples I am working with is from
Dave Marshalls examples.
Everything works well, but when I try this in my background process and I want to extend my original segment (perhaps due to an application upgrade in the future) I either have to change my key or somehow pass the shmctl(shmid, IPC_RMID, 0) call in my app. Since my app cannot exit graciously and I cannot set this right at the beginning after allocation (since no other app will be able to use this shared mem space once it has been marked for deletion) I have no way of cleaning this space up.
The best I have come up with so far is to shmget my old section, check if it exists, clear it if it does then allocate it to a high value. This would look like:
void init_shared_mem(void)
{
int shmid;
key_t key = 0x1235; //key to be passed to shmget()
int oldSize = 27;
int newSize = 28;
char * shm;
//check to see if an allocation exists
if ((shmid = shmget(key, oldSize, IPC_CREAT | 0666)) < 0)
{
perror("shmget: shmget failed");
}
//unallocate it if it does
else if (shmctl(shmid , IPC_RMID , 0) == -1)
{
perror("shmid");
}
//reallocate new section
if ((shmid = shmget(key, newSize, IPC_CREAT | 0666)) < 0)
{
perror("shmget: shmget failed");
exit(1);
}
if ((shm = shmat(shmid, NULL, 0)) == (char *) -1)
{
perror("shmat");
exit(1);
}
}
Other SO questions either don't seem to touch on this or bring it up as a possible concern. Is there a better way to do this that I am missing or perhaps I can simply reallocate it to a higher value instead of clearing it?
You don't say which operating system you are running on. If you're on Linux, OpenBSD 5.1 or later or other operating systems configured a particular way you may well be able to attach after having done IPC_RMID so long as something is still attached to it but bear in mind this behaviour is not portable (here's an older examination of IPC_RMID behaviour on different OSes). Without this behaviour it's going to be difficult to avoid stale segments being left behind if your program crashes (aka "doesn't exit graciously") when it's the last thing attached to the segment.
I should also note that your question sounds similar to the problem discussed over in Options for robust process cleanup over on the UNIX Socket FAQ forum where suggestions included: use Linux's IPC_RMID behaviour, have a monitoring parent process check for process death and do cleanup.
I have implemented a linux kernel driver which uses deferred IO mechanism to track the changes in framebuffer node.
static struct fb_deferred_io fb_defio = {
.delay = HZ/2,
.deferred_io = fb_dpy_deferred_io,
};
Per say the registered framebuffer node is /dev/graphics/fb1.
The sample application code to access this node is:
fbfd = open("/dev/graphics/fb1", O_RDWR);
if (!fbfd) {
printf("error\n");
exit(0);
}
screensize = 540*960*4;
/* Map the device to memory */
fbp = (unsigned char *)mmap(0, screensize, PROT_READ | PROT_WRITE, MAP_SHARED,
fbfd, 0);
if ((int)fbp == -1) {
printf("Error: failed to start framebuffer device to memory.");
}
int grey = 0x1;
for(cnt = 0; cnt < screensize; cnt++)
*(fbp + cnt) = grey<<4|grey;
This would fill up entire fb1 node with 1's.
The issue now is at the kernel driver when i try to read the entire buffer I find data mismatch at different locations.
The buffer in kernel is mapped as:
par->buffer = dma_alloc_coherent(dev, roundup((dpyw*dpyh*BPP/8), PAGE_SIZE),(dma_addr_t *) &DmaPhysBuf, GFP_KERNEL);
if (!par->buffer) {
printk(KERN_WARNING "probe: dma_alloc_coherent failed.\n");
goto err_vfree;
}
and finally the buffer is registered through register_framebuffer function.
On reading the source buffer I find that at random locations the data is not been written instead the old data is reflected.
For example:
At buffer location 3964 i was expecting 11111111 but i found FF00FF00.
On running the same application program with value of grey changed to 22222222
At buffer location 3964 i was expecting 22222222 but i found 11111111
It looks like there is some delayed write in the buffer. Is there any solution to this effect, because of partially wrong data my image is getting corrupted.
Please let me know if any more information is required.
Note: Looks like an issue of mapped buffer being cacheable or not. Its a lazy write to copy the data from cache to ram. Need to make sure that the data is copied properly but how still no idea.. :-(
"Deferred io" means that frame buffer memory is not really mapped to a display device. Rather, it's an ordinary memory area shared between user process and kernel driver. Thus it needs to be "synced" for kernel to actually do anything about it:
msync(fbp, screensize, MS_SYNC);
Calling fsync(fbfd) may also work.
You may also try calling ioctl(fbfd, FBIO_WAITFORVSYNC, 0) if your driver supports it. The call will make your application wait until vsync happens and the frame buffer data was definitely transferred to the device.
I was having a similar issue where I was having random artifacts displaying on the screen. Originally, the framebuffer driver was not using dma at all.
I tried the suggestion of using msync(), which improved the situation (artifacts happened less frequently), but it did not completely solve the issue.
After doing some research I came to the conclusion that I need to use dma memory because it is not cached. There is still the issue with mmap because it is mapping the kernel memory to userspace. However, I found that there is already a function in the kernel to handle this.
So, my solution was in my framebuffer driver, set the mmap function:
static int my_fb_mmap(struct fb_info *info, struct vm_area_struct *vma)
{
return dma_mmap_coherent(info->dev, vma, info->screen_base,
info->fix.smem_start, info->fix.smem_len);
}
static struct fb_ops my_fb_ops = {
...
.fb_mmap = my_fb_mmap,
};
And then in the probe function:
struct fb_info *info;
struct my_fb_par *par;
dma_addr_t dma_addr;
char *buf
info = framebuffer_alloc(sizeof(struct my_fb_par), &my_parent->dev);
...
buf = dma_alloc_coherent(info->dev, MY_FB_SIZE, dma_addr, GFP_KERNEL);
...
info->screen_base = buf;
info->fbops = &my_fb_ops;
info->fix = my_fb_fix;
info->fix.smem_start = dma_addr;
info->fix.smem_len = MY_FB_SIZE;
...
par = info->par
...
par->buffer = buf;
Obviously, I've left out the error checking and unwinding, but hopefully I have touched on all of the important parts.
Note: Comments in the kernel source say that dmac_flush_range() is for private use only.
Well eventually i found a better way to solve the issue. The data written through app at mmaped device node is first written in cache which is later written in RAM through delayed write policy. In order to make sure that the data is flushed properly we need to call the flush function in kernel. I used
dmac_flush_range((void *)pSrc, (void *)pSrc + bufSize);
to flush the data completely so that the kernel receives a clean data.
For my bachelor thesis i want to visualize the data remanence of memory and how it persists after rebooting a system.
I had the simple idea to mmap a picture to memory, shut down my computer, wait x seconds, boot the computer and see if the picture is still there.
int mmap_lena(void)
{
FILE *fd = NULL;
size_t lena_size;
void *addr = NULL;
fd = fopen("lena.png", "r");
fseek(fd, 0, SEEK_END);
lena_size = ftell(fd);
addr = mmap((void *) 0x12345678, (size_t) lena_size, (int) PROT_READ, (int) MAP_SHARED, (int) fileno(fd), (off_t) 0);
fprintf(stdout, "Addr = %p\n", addr);
munmap((void *) addr, (size_t) lena_size);
fclose(fd);
fclose(fd_log);
return EXIT_SUCCESS;
}
I ommitted checking return values for clarities sake.
So after the mmap i tried to somehow get the address, but i usually end up with a segmentation fault as to my understanding the memory is protected by my operating system.
int fetch_lena(void)
{
FILE *fd = NULL;
FILE *fd_out = NULL;
size_t lenna_size;
FILE *addr = (FILE *) 0x12346000;
fd = fopen("lena.png", "r");
fd_out = fopen("lena_out.png", "rw");
fseek(fd, 0, SEEK_END);
lenna_size = ftell(fd);
// Segfault
fwrite((FILE *) addr, (size_t) 1, (size_t) lenna_size, (FILE *) fd_out);
fclose(fd);
fclose(fd_out);
return 0;
}
Please also note that i hard coded the adresses in this example, so whenever you run mmap_lena the value i use in fetch_lena could be wrong as the operating system takes the first parameter to mmap only as a hint (on my system it always defaults to 0x12346000 somehow).
If there is any trivial coding error i am sorry as my C skills have not fully developed.
I would like to now if there is any way to get to the data i want without implementing any malloc hooks or memory allocator hacks.
Thanks in advance,
David
One issue you have is that you are getting back a virtual address, not the physical address where the memory resides. Next time you boot, the mapping probably won't be the same.
This can definitly be done within a kernel module in Linux, but I don't think there is any sort of API in userspace you can use.
If you have permission ( and I assume you could be root on this machine if you are rebooting it ), then you can peek at /dev/mem to see the actual phyiscal layout. Maybe you should try sampling values, reboot, and see how many of those values persisted.
There is a similar project where a cold boot attack is demonstrated. The source code is available, maybe you can get some inspiration there.
However, AFAIR they read out the memory without loading an OS first and therefore do not have to mess with the OSs memory protection. Maybe you should try this too to avoid memory being overwritten or cleared by the OS after boot.
(Also check the video on the site, it's pretty impressive ;)
In the question Direct Memory Access in Linux we worked out most of the fundamentals needed to accomplish this. Note, mmap() is not the answer to this for exactly the reasons that were stated by others .. you need a real address, not virtual, which you can only get inside the kernel (or by writing a driver to relay one to userspace).
The simplest method would be to write a character device driver that can be read or written to, with an ioctl to give you a valid start or ending address. Again, if you want pointers on the memory management functions to use in the kernel, see the question that I've linked to .. most of it was worked out in the comments in the first (and accepted) answer.
Your test code looks odd
FILE *addr = (FILE *) 0x12346000;
fwrite((FILE *) fd_out, (size_t) 1,
(size_t) lenna_size, (FILE *) addr);
You can't just cast an integer to a FILE pointer and expect to get something sane.
Did you also switch the first and last argument to fwrite ? The last argument is supposed to be the FILE* to write to.
You probably want as little OS as possible for this purpose; the more software you load, the more chances of overwriting something you want to examine.
DOS might be a good bet; it uses < 640k of memory. If you don't load HIMEM and instead write your own (assembly required) routine to jump into pmode, copy a block of high memory into low memory, then jump back into real mode, you could write a mostly-real-mode program which can dump out the physical ram (minus however much the BIOS, DOS and your app use). It could dump it to a flash disc or something.
Of course the real problem may be that the BIOS clears the memory during POST.
I'm not familiar with Linux, but you'll likely need to write a device driver. Device drivers must have some way to convert virtual memory addresses to physical memory addresses for DMA purposes (DMA controllers only deal with physical memory addresses). You should be able to use those interfaces to deal directly with physical memory.
I don't say it's the lowest effort, but for completeness sake,
You can Compile a MMU-less Linux Kernel
And then you can have a blast as all addresses are real.
Note 1: You might still get errors if you access few hardware/bios mapped address spaces.
Note 2: You don't access memory using files in this case, you just assign an address to a pointer and read it's content.
int* pStart = (int*)0x600000;
int* pEnd = (int*)0x800000;
for(int* p = pStart; p < pEnd; ++p)
{
// do what you want with *p
}