I am trying to do some testing of several thousand NFSv3 fileserver exports across hundreds of servers. Lots of things can go wrong, from configurations on the server to network connectivity. The most complete test I can do is to actually try to mount it on a client.
I can do that, but actually mounting everything is more than I need, takes state and resources beyond the program's execution, and tends to stress client a bit. I've more than once seen problems that seem to indicate something on the client was unhappy and preventing a mount from happening. (With no changes other than a client reboot, mounts worked again).
I was hoping to instead code something lighter weight that would simply act as a NFS client and see if the NFS MOUNT call successfully returned a filehandle. If it did, my server is running and my client is authorized. But I've not found any simple code to do so.
When I look at the Linux Source, it looks like at least some of the code is involved with it being a linux module, which is confusing.
Is there some user-space code that just requests a NFS filehandle via a mount call that I might be able to strip down? (Or is there any reason that my idea wouldn't work)? This is all AUTH_SYS, so I don't have to get kerberos tickets or anything.
Without knowing more, I will speculate a little based on my knowledge of NFS/Linux file systems.
I assume your client is linux (but the same logic could apply with windows if it had an nfs client present).
It sounds like when you do the mounts you are reaching a point where resources are being consumed to the point that the client cannot mount any more nfs mounts. Makes sense as when you reboot it starts working again, and a reboot will drop the nfs mounts (assuming that you are explicitly/programmatically mounting) thus allowing mounts to occur again. I bet you are just mounting the nfs mounts and are never unmounting them. So I would suggest the following:
mount
access a file or directory in the just mounted nfs filesystem
unmount the nfs file system
well, then just get the rpc protocol definition files for the version of nfs you are running, probably version 3 or 4, run them through the rpcget xdr protocol compiler you will get client functions in c that you can compile that you can make calls to the server with. But they will execute several system system calls, no way to do network communication linux without that happening and it will go throuth the tcp/ip stack (you will probably use udp) in the linux kernel. U can probably find the nfs protocol definition files on the SUN/Oracle website, or you can get find them in the source for a linux distribution -- you will be making application layer calls but the client calls rpc library functions which in turn will call linux system calls which go to the kernel
Related
I know this should be obvious, but I have found far too many DIFFERENT answers and the ones I've tried all fail (sometimes or all the time), so...
We are working on a service and some applications that run at startup on a Windows 10 computer that performs an automatic login. The service and applications require Windows sockets for TCP, UDP and Multicast. Most of the time, our programs fail because they get errors about the network not being ready and such. Currently, we work around this by just adding a dumb, fixed length delay time before attempting to start, but we would prefer to start as soon at the network is ready to be used.
Our most recent attempt was to wait on the LanmanWorkstation (Workstation) service, but that generally reports it is running/ready before the sockets functions will succeed. I have also seen suggestions to use LanmanServer (Server) or Netman (Network Connections) or maybe even Tcpip (TCP/IP Protocol Driver), but I cannot find anything definitive. One would think this is a common requirement, so why would Microsoft make the info so difficult to find?
Ahem. Does any know a definitive method for a service or application to wait until winsock functions will succeed before using them? Short of a spin wait on a failing winsock function, of course!
I am writing a C program for Linux that reads and writes to files on an NFS server. The share is mounted hard; attempts to access it will block indefinitely until they work. Having my program block indefinitely is bad; it is still capable of doing useful work even if the files are unavailable. Remounting the share soft is not an option.
Having two processes, one of which does work and won't block, and another that handles file IO and might block is an option, but would constitute a Major Change. I'd like to avoid that. Really, I want to say, "I know that you're hard mounted so that naive programs can pretend you're a highly reliable local disk. But I know better and I am prepared to cope with any access failing, similar to the behavior if you were soft mounted." So:
In C, how can I access files on a hard-mounted NFS share, getting errors if the server is unavailable instead of blocking indefinitely?
I can run as root if necessary, but would prefer not. Using root to remount the share is right out. I can potentially rely on new features, but the further back support goes the better.
My research suggests the answer is that it's just not possible, but perhaps I've missed something.
You've not missed anything, you will never receive server unavailable errors because the kernel will never deliver them on hard mounted nfs mount points.
Because the hard option is a property of the mount point, you can't have applications that pick and choose because the kernel isn't set out to behave in that manner.
However, you do mention that you can run the application as root. Why not mount the file system somewhere else soft, and then get your anticipated behaviour?
I have remote disks mounted on my system using NFS and I am trying to write to the files on the mounted remote disks using pwrite() API.
It doesn't happen every time but in some cases while doing I/O pwrite() fails and set the error number to EIO(Input/Output error).
Can some one please explain why this error occur on the first place and is there any way I can correct it?
Thanks
From (bad) experiences with reading and writing to NFS based files I learned that you have a good chance to work around this EIO by simply retrying the failed I/O operation (read(), write()).
Also on NFS one can not assume that read()/write() do transfer the amount of data specified, so it is good idea to always check the return value of the function in question on how many byte were transfered.
I see the issue in the underlying NFS functionality or in the way the NFS driver's results are handled by the kernel, so I strongly assume pread()/pwrite() show the same effects as I witnessed when using read()/write().
here is the situation:
I have written a C program doing some wireless measurements on a WRT54GL Router (OpenWRT White Russian, Busybox 1.00, Dropbear client v0.49). Please note that i can not use a more up to date version of the operating system on the router or install additional packages (just scripts or small programs are allowed).
Up to now, i log my measurements results every 15 minutes from the router to the server via a
cat localfile | ssh target_address cat ">" remotefile
which i call from my C program (system()) for every logfile which is created or present at the moment the log starts. What i don't like is, that the system call opens a new shell for every single call, causing some overhead. The good thing is that in this way the data is encrypted and because i do a connection for every file, i can directly get per file feedback from the server, so that i can remove the logs from the router. (Other approaches calling scripts from the router on the server, which then return values for the logging did not work, as the dropbear ssh client does not support this return).
So what i'm asking for: what could be a more elegant way to do so and to reduce this overhead ? By now, i've read a few tutorials about how to use TLS / TCP Sockets (so i can send the data encrypted to the server). Another possibility could be a HTTP PUT or POST, but there i am not sure how i could get feedback for the data being send. So i would just like to hear your oppions and how you guys would try to tackle this.
Best regards
Since you're talking about log files, this sounds like a job for the syslog protocol.
I am pretty sure OpenWRT supports it out of the box.
I'm trying to find proper way to handle stale data on NFS client. Consider following scenario:
Two servers mount same NFS shared storage with number of files
Client application on 1 server deletes some files
Client application on 2 server tries to access deleted files and fails with: Stale NFS file handle (nothing strange, error is expected)
(Also it may be useful to know, that cache mount options are pretty high on both servers for performance reasons).
What I'm trying to understand is:
Is there reliable method to check, that file is present? In the scenario given above lstat on the file returns success and application fails only after trying to move file.
How can I manually sync contents of directory on the client with server?
Some general advise on how to write reliable file management code in case of NFS?
Thanks.
Is there reliable method to check, that file is present? In the scenario given above lstat on the file returns success and application fails only after trying to move file.
That's it normal NFS behavior.
How can I manually sync contents of directory on the client with server?
That is impossible to do manually, since NFS pretends to be a normal POSIX-compliant file system.
I have tried once to code close()/open() in an attempt to somehow mitigate the effects of the NFS client-side caching. In my case I needed to read the info written to the file on other server. But even the reopen trick had close to zero effect. And I can't add fdatasync() to the writing side, since that slows whole application down.
My experience with NFS to date is that nothing you can do. In critical code paths I simply coded to retry the file operations which return ESTALE.
Some general advise on how to write reliable file management code in case of NFS?
Mod me down all you want, but if your customers want reliability then they shouldn't use NFS.
My company for example advertises use of proper distributed file system (I intentionally omit the brand) if customer wants reliability. Our core software is not guaranteed to run on NFS and we do not support such configurations. But in our case we really need the guarantees that as soon as the data are written to FS they become accessible on all other nodes.
Coherency in NFS can be achieved, but at the cost of performance, making NFS barely usable. (Check its mount options.) NFS is caching like crazy to hide the fact that it is a server file system. To make all operations coherent, NFS client would have to go to the NFS server synchronously for every little operation, bypassing the local cache. And that would never be fast.
But since we are talking Linux here, one can advise customers of the software to evaluate available cluster file systems. E.g. RedHat now officially support GFS. I have heard about people using CodaFS, but have no hard info on it.
i have had success with doing ls -l on the directory which contains the file.
You could try the ''noac'' mount option
from man nfs:
In addition to preventing the client
from caching file attributes, the noac
option forces application writes to
become synchronous so that local
changes to a file become visible on
the server immediately. That way,
other clients can quickly detect
recent writes when they check the
file's attributes.
Using the noac option provides
greater cache coherence among NFS
clients accessing the same files, but
it extracts a significant performance
penalty. As such, judicious use of
file locking is encouraged instead.
You could have two mounts, one for critical fast changing data that you need synchronized and another mount for other data.
Also, look into NFS locking and its limitations.
As for general advice:
One way to truncate a file that is concurrently read from multiple hosts is to write the content into a temporary file and then rename that file to the final location.
On the same filesystem this operation should be atomic.