Exposing a custom filesystem over SMB or other network protocol - filesystems

We've written a library that implements a filesystem-like API on top of a custom database system.
We'd like to be able to mount this filesystem as an ordinary filesytem from other machines across a network.
Are there any libraries, that can run in user space, that lets other machines on the network mount this and treat it like an ordinary file system? (Preferably in Python or C++)

One of the options is to use our Callback File System and expose the filesystem as a virtual disk, which can be shared using regular Windows sharing mechanisms. Callback File System includes a kernel-mode driver, needed to do the job, and offers user-mode C++ API which you use to expose your data as a filesystem.
Once there was .NET implementation of SMB server called WinFUSE, but it's long gone and almost no traces are left.
Update: On linux you can use FUSE to implement a filesystem and mount it, and then use some mechanisms (not necessarily a library) to expose the mounted filesystem as an SMB share.

Related

What's the relationship between AT commands and device vendor's C API?

I'm doing an embedded-system project now. From my view, AT commands can be sent to a device to retrieve 4G information, dial and so on. On the other hand, we can also do that by calling APIs provided by the 4G vendor.
My question is what's the relationship between them? Is the API a wrapper for the AT commands?
TL;DR
Vendor's API (not only C, but also C++, Java or Python depending to the vendor and the modem model) can both be wrappers for AT commands and a wider, more powerful set of API were the user can port complex applications.
It depends on the vendor and on the model.
A jungle of modems produced by different vendors
It is impossible to define a general "rule" about API provided by a Cellular Module (not necessarily a 4G module).
First of all every vendor usually implements standard AT Commands (both Hayes commands and extended standard commands for cellular devices). In the same way every vendor has it's own implementation of the user application area where the customers can store their own application to control the modem's functionalities and to use them according to their own application requirements.
AT commands remain the interface to be used when the modem needs to be connected (and driven) by an external host. When the user application area is used, instead, a wider set of API is usually provided. They may include:
A library exporting a subset of the OS capabilities (threads management, events, semaphores, mutexes, SW timers, FS access and so on)
A library offering the capability to manage the specific HW of the device (GPIOs, SPI, I2C, ADC, DAC and so on)
A library offering a programmatical way to perform action, related to connectivity, that would normally be executed through AT commands (registration status check, PIN code insertion, PDP context activation, SMS management, TCP/UDP/TLS sockets)
The latter usually access a base layer involving all the functionalities provided by the modem. Usually this is the same layer invoked by the AT commands sent through modem's serial interface.
Sending AT commands... from the vendor's API?
Of course it often happens that the library mentioned above provides just a subset of the functionalities usually exported with the AT commands set so, in order to "fill the gap", a further set of APIs is usually exported as well:
A set of functions that allow the simulation of AT commands sent to the modem's serial port. Sending them and parsering the responses they send in the vitual internal serial/USB interface allow the user to port in the internal user application area the the application they previously run on an external host processor (with obvious BOM benefits).
As an example, please check Telit Appzone here and here. It was the inspiration of my answer because I know it very well.
I don't know why you name the title that there's a relationship between AT command and Linux-C API.
Regarding AT command, you can take a look at this wiki article for general information.
Each module has a specified AT command sets. Normally, the module manufacture just offers AT command set and what return values are.
Is API a wrapper of AT command?
If you can use the API provided by the manufacturer, then yes, it's a wrapper of the AT command handler.
My question is what's the relationship between them? Is the API a wrapper for the AT commands?
It is impossible to be sure without having any details of the device, but probably any C API for it wraps the AT command set, either by communicating with the device directly over an internal serial interface or by going through a device driver that uses AT commands to communicate with it.
However, it is at least conceivable, albeit unlikely, that the 4G device offers an alternative control path that the C API uses (definitely via a driver in this case).
I'm not quite sure what the point of the question is, though. If you are programming the device and its 4G component in C, and the manufacturer has provided a C API, then use it! If you are programming in some other language then at least consider using the C API, which you should be able to access from most other languages in some language-specific way. You should not expend effort on rolling your own without a compelling reason to reject the API already provided to you.

How to pass a variable-size array from Windows kernel driver to a user-mode process?

I'm learning Windows kernel programming, and I'm wondering how do I pass a byte array from a kernel driver to my user-mode application, where the kernel driver initiates the call?
If I were to do this among user-mode processes (say, from a service to a GUI app) I'd use a named pipe or a shared memory with a named event and a named mutex for synchronization. But I'm lost what to do on the kernel driver side.
Here's my actual example: I have a kernel callback function that can be called any time with a STRING. I then need to pass the string from within it to my currently running user-mode process and to alert it.
There are tons of ways for kernel-mode to user-mode Inter-Process Communication, and different requirements can suit different techniques.
For starters, you have the option of named pipes (even in kernel-mode). However, there's something you should know... It isn't officially documented for a normal kernel-mode device driver (although there is a documented interface for Filesystem Mini-Filter device drivers).
If you want to use a named pipe from a normal kernel-mode device driver, you'll have to locate the address to NtCreateNamedPipeFile or rely on IoCreateFile (which NtCreateNamedPipeFile relies on internally, using an undocumented structure).
For using a named pipe from a Filesystem Mini-Filter device driver, you have FltCreateNamedPipeFile.
Moving on from the named pipes idea, you have the option of Local Procedure Calls! However, once again, another dead-end in terms of documentation. It is relatively straight forward to do it as a client in kernel-mode though. There's a documented interface for Ports with a Filesystem Mini-Filter device driver though: FltCreateCommunicationPort.
Moving on again, you could attach to the user-mode client and write directly to its memory.
If you really wanted, you could rely on something simple like a shared event to notify the user-mode client that you've just attached to it and written into its virtual memory.

Is accessing a device driver in Linux only possible using its device file?

Let's say I have a webcam, and I installed the device driver for this webcam in my Linux OS, now a device file will be created for the device driver (for example: /dev/video0).
Now say I want to create a program in C that wants to access this webcam. How can my program access the device driver for the webcam, should my program use the device file (/dev/video0) to access the device driver, or is there another way?
You asked a general question, and then gave a specific example. I'll try to address both.
When you load a driver, the way to communicate with it from user space is by whatever means this driver defined. Typically, this is through a /dev device created for the driver. If that's the case, yes, that's the only way to communicate with it.
This is not universally true. Many drivers also have entries under the /sys sysfs pseudo file system, and some aspects can be modified through there. In fact, there are whole classes of drivers that are only accessible through the /sys fs. Prominent examples are GPIO and Led devices, that can be turned on and off via access to /sys/class/gpio and similar paths.
Another option, considered deprecated but still sometimes used, is to use the /proc pseudo file system. Again, this is up to the driver to define its communication method. As the user, you will have to follow whatever protocol the driver defined.
Also, some drivers don't have any file system presence at all. The most obvious standard example are network interfaces. The only way to communicate with them is via the networking system calls.
In the particular example you provided, you talked about a video camera that appears as /dev/video0. Such a camera is, usually, a Video4Linux (or v4l) camera, and those are accessed via their character devices.
With that said, the protocol for communicating with the camera might have wrappers that makes life easier. If you open the actual device, you might have to implement a rather complicated handshake with it. Instead, you can use the v4l library to wrap the details of the access.
Make no mistake. You're still talking to the character device in /dev. It's just that it's not your code that does it, but the library's.

Intercepting file access

I have a program (a game, really), which uses a .wad file to store its resources. Is it possible to somehow intercept access to this file, and emulate it?
For instance, I want to dynamically replace some sprites. Instead of creating a new file, is it possible to make this game think it is accessing the .wad, but actually we process its requests?
Under Windows the File System Filter Driver provides a low level I/O hook that a program can register to be passed the I/O requests to the file system.
Filter drivers can also alter the data passed via filters or deny filesystem requests.
Implementation, maintenance and support of your such kernel-mode code is non-trivial.
Anyway you could also take a look at:
Winpooch: an antivirus has to intercept file accesses so an open source antivirus can be a good starting point to study filters.
EasyHook: for Windows API hooking

requirement of root privileges for libpcap functions

The pcap_lookupdev() fills in the errbuf variable when run as non-root user, while the same functions returns the value of the first available network interface when run as root.
Is this access disabled by the OS or the library. I think it is the OS. What is the right answer?
This is not a homework question
In general, when it comes to accessing files, devices and other services provided by the OS, access models in Unix (and, thus, Linux) are implemented in the OS.
Userspace programs are expected to just try whatever they want to do and gracefully handle any error condition by e.g. informing the user with a message.
This has several advantages:
Maintainability: Access policy enforcement remains with the OS and can be configured uniformly. The administrator that wants to restrict access to a resource does so once, rather than having to configure this library here, than that library there, then...
Configurability: The administrator can configure as simple or complex an access policy they need without being limited by each userspace implementation.
Security: Userspace programs should not in general be trusted with enforcing access policy. It would be like having a wolf guard the sheep.
EDIT:
In your case, pcap needs low-level access to the network interface. Due to the security implications (capturing network traffic, generating arbitrary network packets etc), such access is limited to privileged users only. On Linux, for example, pcap needs the CAP_NET_RAW capability to be available to the user.
Many of the pcap functions require root privileges in order to work correctly. Might this be the problem?
It mostly depends on OS. Not all pcap functions require root privilege on all OS.
Ref to Reference Manual Pages, all special privilege requirements are listed respectively.

Resources