I'm trying to mimic what arducam does https://www.arducam.com/product/arducam-1mp4-quadrascopic-camera-bundle-kit-for-raspberry-pi-nvidia-jetson-nano-xavier-nx-four-ov9782-global-shutter-color-camera-modules-and-camarray-camera-hat-b0331/
So I have got an FPGA that does combine 4 camera streams and output them into one big frame buffer CSI MIPI.
The problem right now is with the platform either Linux or TDA4 from texas instruments which don't support video4linux.
But that's not an issue, I would like to know actually how would one write a V4L driver that separates that big frame into 4 virtual cameras in linux ?
Vision Components supplies open-source linux device drivers for their line of MIPI-CSI cameras. If you need to write device drivers, you at least can look at a reference. They interface to a Lattice FPGA that they configure with i2c over a variety of platforms.
Lattice sells very small form factor FPGAs and provides a MIPI-CSI IP core you can use if you're interested in developing your own camera.
This may be a template you can explore..
https://www.vision-components.com/fileadmin/external/documentation/software/vc_mipi_driver_list/index.html
https://www.latticesemi.com/csi2rx
Related
I was wandering, why there are no implementations of the devices written in CMSIS-Driver?
I mean that I have few peripherals: LCD, temperature and pressure sensor, current meter etc. - all of them very popular used in arduino and learning sets.
Each of these devices uses some protocol to communicate with the uC. some are for i2C, some communicate by SPI, some by UART. I was wondering if there are drivers that handle those devices, and as a backend use CMSIS-Driver API.
I think that it is a decent api, and after all standard develop by ARM, so why I can not find any drivers using it?
For example when I was looking for s18b20 (temperature sensor for 1-wire), I was easily able to find driver for this device written in RUST language, but I was not able to find any implementation for C that would use CMSIS. (in this case compare to rust is quite solid, because Rust has nice embedded API, and you can easily use the driver on multiple targets, just like CMSIS-Driver is spouse to work)
I was able to find some projects using this peripheral, but they all operated on HAL that is different for every uC, so the implementation is not portable ( unlike RUST, and potentially CMSIS-Driver)
So my main questions are:
Why there are so little implementations based on CMSIS-Driver? Maybe there is some hidden implementation repository that I do not know about?
Am I missing something? Is the CMSIS-Driver not designed for the casual developers? Who is it designed for then ?
CMSIS is not concerned with external devices, it deals primarily with interface drivers for interfaces on the microcontroller die. So if you have an SPI device, you might use the CMSIS. SPI driver for that part, but it is then your responsibility as a developer to write the higher-level driver for the external device.
Higher-level software platforms such as ARM's embed, or ST's CubeMX use CMSIS interface drivers, and include drivers for common higher level devices. They tend to be for more complex devices related to networking, filesystems and displays. I would not expect much support for such trivially simple devices such as a temperature sensor.
I am developing my own OS and for days I’ve been wondering how to control video card, audio card to build a delicate GUI. It seems that the procedure is a bit different from that of controlling mouse or keyboard.
By the way,
1.How does an OS recognize hardware drivers?
2.Are all hardware drivers written for specific platforms (such as Windows,Linux,etc.)?
Still freshman in the university and failed to find relevant information
thanks for help =)
how to control video card
Graphics cards typically support a VGA mode and the VESA 2.0 standard. So this could be a good starting point for your first video driver.
How does an OS recognize hardware drivers?
Linux uses a device tree describing the hardware which has a compatible field for every item that needs a driver. All hardware drivers are kept in linker generated lists. If a driver has the same compatible value it can be used to drive the hardware. The linker list item is generated from a macro in the driver code using a specialized linker script.
Are all hardware drivers written for specific platforms (such as Windows, Linux, etc.)?
If you provide the right wrappers you may be able to reuse existing drivers. E.g. U-Boot reuses Linux drivers for USB devices. And the NDISwrapper can be used to run Windows WLAN drivers on Linux.
I want to make an (extremely simple) operating system. I am currently learning about graphics cards.
This is what I know so far (please correct me if I am wrong):
A graphics card has two modes: a text mode, and a graphics mode.
You can write data to a graphics cards using BIOS (instead of accessing the graphics card directly).
What I want to do is to write directly to the graphics card's video memory without using BIOS (because I want to understand how things work). So I have the following questions:
How do I know what is the base address of the video memory of the graphics card, is this
done by probing the PCI bus to get the base address, or is the base address fixed (just like the COM ports base addresses is fixed for example)?
Are all graphics cards accessed in the same way, or do I have to create device drivers for all available graphics cards?
Edit: I am using x86.
Introduction
Graphics cards are a very complex topic, I'm confident in saying that they are the most complex subsystem you'll find on a PC.
If you ever found yourself lost programming an XHCI (USB 3.0) or an old RTL8239A network interface card then be prepared because this is much more complex.
Graphics controllers are the products of a very competitive marketing - rarely a vendor opens the specifications and when it does, it gives an intentionally poor support.
If you add that the hardware itself deals with: codecs, audio (yes, audio streams too), 3D programmable pipelines, video signals and video outputs, surface formats, media formats, DMA and memory remapping then you can see that it is not an easy task to program a video card.
The better approach, in my opinion, is to "retrace the history" of the video cards.
Start from the MDA then move to CGA then EGA and finally to VGA.
The VGA legacy is still supported, the specifications can be found here or in the first part of this PDF from Intel.
You can program the VGA without the BIOS "easily" - meaning that it is an already well-known and documented hardware architecture (but not necessarily easy to configure).
I don't remember if the previous adapters were subsets of the VGA or not, if not they aren't supported anymore probably.
You can try with a virtual machine or an emulator.
When you are satisfied with the VGA you can move to the SVGA.
Here come the troubles: as Wikipedia confirms, the VGA was the last truly standardised video card/adapter interface:
Unlike VGA—a purely IBM-defined standard—Super VGA was never formally defined.
The organisation VESA standardised a BIOS API called Video BIOS Extensions to allow the use of SVGA cards to driverless OSes but that's not what you were looking for.
You can try reverse engineering a VBE BIOS but I think it will be a nightmare - a senseless stream of writes to IO ports and MMIOs.
Making sense of tenths of configuration registers without any reference is almost impossible.
Note that we are still talking about 1998 technology up to this point.
After the VESA VBE effort, no more standard interfaces have been published - the only reliable way to program a video card with less than 20 years is by signing an NDA with its vendor.
Luckily, recently (actually, not anymore), Intel entered the market with its Intel GFX (a.k.a. Intel HD Graphics) cards.
Intel never aimed to manufacture top-of-the-notch video cards, not even closely - so they can be open about their architecture since that's not their core business.
The result is this marvellous set of Programming Reference Manuals that describe the functionality of their video cards.
Complete with (traditionalistic) minimal information to program them.
In general, hobbyists stops before this point (at the SVGA checkpoint), because the hardware has become very complex and the efforts very huge.
For example, my Haswell integrated video card is documented with 17 PDFs of about 250 pages each (on average).
The display part is documented in a PDF on its own, the framebuffer has disappeared in favour of Display surface and the display part alone of the hardware is this:
While this may not be very comprehensible, it should suffice to get an idea of the numerous technology that a programmer must understand before programming a modern video card.
You can surely take a look at the Linux source code but beware that the Linux kernel is no usually of immediate understanding even for simple controllers - it is not a toy OS, it is a real OS with its own API and interface that must fit the hardware interface (actually the other way around).
Furthermore, only the Intel and AMD video drivers are really open source, the others are either proprietary or just a bunch of undocumented code.
Brief outline of common VGA modes programming
If you just want to program the VGA (a very respectable task indeed!) you can start by setting the video modes 03h (text mode) or 13h (graphics mode).
Video mode 03h
The frame buffer is at 0b8000h (physical address), usually accessed as 0b800h:0000h as it is handy to have a zero offset.
The screen is made up of 80x25 characters, each characters occupy a word (16-bit) in the frame buffer.
The low byte is the character code - the character map used will associate a glyph to a code (e.g. 41h to A).
The high order byte is the attribute byte - the low nibble is the foreground colour, the high nibble is the background colour.
More information can be found in the EGA/CGA/VGA links above.
Video mode 13h
It is a graphical mode with 320x200 pixel, the frame buffer is at 0a0000h (physical address) usually accessed as 0a000h:0000h for the same reason of above.
Each pixel is a single byte, the value of the byte selects the colour of the pixel.
The default palette can be changed by programming the DAC registers (3c7h, 3c8h, 3c9h for the VGA adapter).
Answers
A graphics card has two modes: a text mode, and a graphics mode.
Not necessarily, today this distinction may not exist anymore.
The MDA had only a text mode.
EGA, CGA and VGA and SVGA had both.
The modern approach is to draw the text, however during boot or during particular situations (e.g. BSOD) a basic video driver in text mode is used.
This driver probably uses a BIOS service since the video driver may not be available/reliable.
You can write data to a graphics cards using BIOS
Up to the SVGA era, then BIOS support was discontinued.
How do I know what is the base address of the video memory of the graphics card, is this done by probing the PCI bus to get the base address, or is the base address fixed (just like the COM ports base addresses is fixed for example)?
Video cards have been connected through the history to the ISA, PCI, AGP and PCIe buses.
Only the ISA bus wasn't configurable (at least not from the beginning), the others had configurable BARs (Base Address Registers) per function (the smallest addressable entity in the PCI bus).
In order to get the base address of the MMIO registers of a video card the PCI or PCIe bus must be enumerated and the standard registers in the configuration space must be read/set.
Dealing with PCIe is not as easy as dealing with PCI.
Note that not even the UARTs have a fixed address, they are configured by default to map to the legacy (3f8h, 2f8h, 3e8h and 2e8h) addresses but the hardware was (is?) in a SuperIO chip behind a PCI-to-LPC bridge that emulated a PCI-to-ISA bridge.
With the advent of the Intel platform hub architecture (i.e. the death of the north and south bridge) the SuperIO chip eventually made it into the PCH or moved behind the SPI controller.
Are all graphics cards accessed in the same way, or do I have to create device drivers for all available graphics cards?
Each graphic card is a beautiful vicious creature on its own.
A device driver is needed for each model.
Some driver can be reused for a whole family of models but this is not true in general.
I am post-newbie to Embedded Linux driver development, have developed Character, UART & simple USB drivers, and have worked with SBC (Raspberry Pi 2). My main learning resources are: Essential Linux Device Drivers, Embedded Linux System Design & Development, youtube tutorials and LDD3. Because of lack of real life driver experience, I have some questions that I would like to share:
Templates: are there some sort of driver templates for speeding up the coding process e.g. have the regularly used structures & functions for being modified & customized for the specific drivers.
What’s suitable minimum distribution:
What I am doing now is either building a kernel image using Buildroot then inserting the developed USB driver into it OR inserting the developed USB driver into a Ubuntu. And for proper testing I have to unload the default USB driver (usb_storage); otherwise can’t test my module. So, my question is regarding real-life development & how-to get minimum kernel that has no drivers, & how-to include my module into it (to be part of it) while building with Buildroot?
What Embedded Linux Driver Development community do you suggest for detailed Q&A?
i.e. In future if I want to ask about low level detail such as what is this x structure, or I am getting this error while using this y function .... is there a specialized community? because all I found are for Linux in general, for Kernel hacking, or don’t have experiences.
What is the missed link in the chain?
Developing applications for Microcontrollers relies on Datasheet, User Guide, Registers, C, etc. Developing Drivers for Embedded Linux relies on C, Linux API, Subsystems, etc. In most learning resources, I never found a relationship between both. So, are there really separated? OR there is a missed link between them??
e.g. for this I2C sensor http://www.robot-electronics.co.uk/htm/srf10tech.htm we have to manipulate it (send hexa to & receive values from its registers). If we imagine that we want to develop an I2C driver for it, then for sure we would need to do the same from within the driver, which is explained in Essential Linux Device Drivers (page#278). The main point behind it is that I2C core hides many complexes & simplifies driver development process.
However, I expect the case in real life is not for a sensor, instead it might be a Microcontroller or a complicated device. So, how would we manipulate the Registers & Peripherals of this Microcontroller? Also, I didn’t find tutorial for such a case!
Can I detect insertion and removal of headset using alsa API ? Which API should I use ? My kernel is 3.0 Linux running on ARM.
Having wanted precisely this functionality for a embedded project, I did some investigations and came to the conclusion (about 6 months ago) that there isn't any generic support in ALSA for jack insertion detection.
Interestingly, I did find headphone (and microphone) detection support in the codec driver I was using (tlv310aic3xxx), but is didn't seem to be plumbed into any upper layers. I suspect that the reason this exists is is Android.
There are essentially two ways of adding this support:
Add support to the codec driver - probably exposing a sysfs node that something in user-space can then block on.
Force access to the I2C bus on which the codec is hung (the codec driver usually 'owns' the device) and programme the relevant registers from user-space.
You may face an additional architectural problem in that whilst the codec can detect insertion events, it needs some way of interrupting the CPU. The tlv310aic3xxx devices have programmable GPIO pins, which can be connecting to a interrupt line on the main CPU (if on an embedded platform, this will be another GPIO line). Without this, you'll need to poll it.