is it possible to handle processes that are long running and may not keep an application running in the cli. for example zfs scrub /tank completes in a few seconds yet the process of the scrub continues in the background. there is a utility called zed that can run a script or other actions when the scrub completes to get the details of the scrub. is there a way that ansible can handle situations like this to receive results and act on them or is this something that a tool like jenkins would be needed
Use the wait_for module to make Ansible wait for the process to complete.
For example, you could configure the zed utility to create a file when the scrub process completes.
- name: Wait until the file /tmp/scrub-results is present before continuing
wait_for:
path: /tmp/scrub-results
The wait_for module can test many other conditions such as the presence of an open port or specific content to appear in a file. It can also watch a specific process ID until it terminates.
It is best to read the documentation to determine which is most suitable for your use case.
Related
I'm having trouble finding the answer to this question, maybe I'm just not asking the question properly. I have to put a file that is relatively large (~500MB at least) in an ftp server and then run a process that takes it in as a parameter. My question is as follows. If i'm using ftp.exe to do this, does the put command lock the process until the file is finished being copied?
I was planning on using a .bat file to execute the commands needed but I don't know if the file is going to be completely copied before the other process starts reading it.
edit: for clarity's sake, here is a sample of the .bat that I would be executing.
ftp -s:commands.txt ftpserver
and the contents of the commands.txt would be
user
password
put fileName newFileName
quote cmd_to_execute
quit
The Windows ftp.exe (as probably all similar scriptable clients) executes the commands one-by-one.
No parallel processing takes place.
FTP as a protocol doesn't specify placing a lock on the files before writing it. However this doesn't prevent anyone from implementing this feature as it is a great value add.
Some FileSystems NTFS) may provide locking mechanism to prevent concurrent access. See this File locking - Wikipedia
See this thread as a reference: How do filesystems handle concurrent read/write?
I'm sure my question has probably been answered previously but I didn't find anything specific to my situation after searching for a while.
Background:
I have written a suite of data acquisition tools in C that run on an embedded system running Debian Wheezy. There is a main module, called Dispatch, whose job is to launch the rest of the modules and pass messages between them. I put a trivial bash script in /etc/init.d that executes Dispatch when the system boots since this system runs unattended. This system runs without any local user interaction so Dispatch should really be written to function as a daemon but it is not. The startup script simply executes /opt/bcdispatch &.
There's a bug in one of the other modules that causes it to crash every few days. I'm trying to hunt down that bug but in the meantime I am trying to write a watchdog program that will detect the crash, kill off all of my processes, then relaunch Dispatch. For reasons I won't go into it is not sufficient to just relaunch the crashed process, the whole suite of tools needs to be restarted.
What I'm trying to do:
I wrote a simple watchdog program that periodically executes popen("ps aux | grep bc") (all of my process names start with "bc" which makes it easy to find them with grep), finds that one of the modules has crashed by looking for anything with a "zombie" status in any of the lines read from popen(), kills all of my processes by calling system("kill <PID>"), then executes the startup script in /etc/init.d and exits. I modified the startup script so that it launches the watchdog after launching Dispatch. The startup script now looks like:
/opt/bcdispatch &
/opt/mywatchdog &
Everything is being run as root. There are no other user accounts on the system.
Problem
The watchdog process works fine if I run it from the command line. It kills off all of the processes it's supposed to, launches the startup script, then exits. However, when the watchdog is launched by the startup script at boot time it doesn't do its thing. It's running, one of the processes it's monitoring has crashed, but it doesn't kill the rest of them off. It just sits there like a giant turd. I can start another instance of it from the command line and that one works just fine.
Question
So my question (finally!) is: why can't my program kill other processes when launched via a startup script? I suspect it has something to do with the fact that the watchdog process no longer has a terminal associated with it? I tried substituting the call to system("kill <PID>") with kill(PID) but that didn't change anything.
EDIT
It just occurred to me that it's not the kill()ing part that doesn't work (well, that might be broken as well), the call to popen("ps aux | grep bc") must not be working since the watchdog should exit after it finds the zombie process but it isn't. Its PID is still the same as it was when the system booted. I guess this means the title of this question isn't very good.
Found the problem. The output of my watchdog's call to popen("ps aux | grep bc") was being truncated to 80 columns, presumably because it was no longer attached to a terminal and that's the default terminal width. That truncation was causing problems for the way the program was parsing the results of the ps command so it never found the crashed process. Changing the command to popen("ps -w aux | grep bc") was all that was needed to fix it.
I was hesitant to post this question because I assumed someone somewhere had asked it already but after much scouring, I've come up empty, so here it is.
BACKGROUND: I'm running a local agent (written in C, listening via TCP) which allows for execution of a small number of scripts/commands remotely. (Via a web interface, to be specific.) The scripts themselves are a mixture of binaries, bash, or perl scripts and the agent itself doesn't really care, as long as they are allowed in the list.
(This is on a corporate, internal network and this is in the very early stages, so please don't debate the merits of security at this time.)
The C agent code to launch processes is this:
sprintf(mrun, "%s %s 2>&1", file, args);
mexec = popen(mrun, "r");
[read some returned buffer]
pclose(mexec);
This approach works well for both external bash and perl scripts, provided the scripts just execute commands (or do things in the foreground). However, I recently had a need to expand a script to include a restart of a daemon, in this case, named.
The script itself (bash) is simple:
#!/bin/bash
pkill -9 named
/local/mnt/named/sbin/named -c /local/mnt/named/var/named.conf &
echo "restarted"
The problem I am running into is that the script never finishes (i.e. restarted is never echo'd) when run via the C agent, so the control is never returned and the TCP socket never gets free'd up. As far as the agent is concerned, the process is still running. If I run the script from a terminal, it works fine and control is returned back to me.
Am I missing something that would allow the script to execute normally when being forked off from a C daemon versus just being called from the bash terminal?
I know of nohup and I guess could use that if all else fails but I was curious if there is some other kind of workaround for doing this.
Based on feedback from the comments above, I was able to get the script to continue working after launching the daemon process, thanks to some additional redirects:
/local/mnt/named/sbin/named -c /local/mnt/named/var/named.conf </dev/null &> /dev/null &
So, thanks to fork0 for that bit of knowledge.
Afterward, I noticed that the TCP socket connection wouldn't close properly, even though the script was done working. After some more info below and doing a lot of research, it turns out that child processes will inherit (and keep open) file descriptors from the parent process (which includes sockets).
I looked all over for methods to disown the child process but didn't really find any that would work for me (or didn't constitute an entire rewrite of the agent).
Finally, I stumbled upon this question, which is related but not in a programming language I use:
os.execute without inheriting parent's fds
This basically involved the child process closing any open file descriptors inside the code, thus freeing them to be closed by the parent. (I think?)
I added a few lines to the bash script to do this prior to starting named and it does work.
for i in `nawk 'BEGIN{ for(i=1;i<=255;i++) print i}'`
do
eval exec `echo $i | sed -e 's/.*/&<\&-/'`
done
(I would up using nawk instead of seq because I need it to run on Solaris and Linux.)
Some basic testing shows that this has solved the major issue of the socket not being able to close but I'll need to do some more research on whether this will have any other ramifications that I am not aware of. There may also be a better, safer way to achieve this but at least I'm on the right track.
I want to build an odometer to keep track of how long I've been on a server since I last reset the counter.
Recently I've been logging quite a bit of time working on one of my school's unix servers and began wondering just how much time I had racked up in the last couple days. I started trying to think of how I could go about writing either a Bash script or C program to run when my .bash_profile was loaded (ie. when I ssh into the server), background itself, and save the time to a file when I closed the session.
I know how to make a program run when I login (through the .bash_profile) and how to background a C program (by way of forking?), but am unsure how to detect that the ssh session has been terminated (perhaps by watching the sshd process?)
I hope this is the right stack exchange to ask how you would go about something like this and appreciate any input.
Depending on your shell, you may be able to just spawn a process in the background when you log in, and then handle the kill signal when the parent process (the shell) exits. It wouldn't consume resources, you wouldn't need root privileges, and it should give a fairly accurate report of your logged in time.
You may need to use POSIX semaphores to handle the case of multiple shells logged in simultaneously.
Have you considered writing a script that can be run by cron every minute, running "who", looking at its output for lines with your uid in them, and bumping a counter if it finds any? (Use "crontab -e" to edit your crontab.)
Even just a line in crontab like this:
* * * * * (date; who | grep $LOGNAME)>>$HOME/.whodata
...would create a log you could process later at your leisure.
I'm writing a TFTP server program for university, which needs exclusive access to the files it opens for reading. Thus it can be configured that if a file is locked by another process that it waits for the file to become unlocked.
Is there any way on Win32 to wait for a file become unlocked without creating a handle for it first?
The reason I ask, is that if another process calls CreateFile() with a dwShareMode that is incompatible to the one my process uses, I won't even be able to get a file handle to use for waiting on the lock using LockFileEx().
Thanks for your help in advance!
If you take a look at the Stack Overflow questions What Win32 API can be used to find the process that has a given file open? and SYSTEM_HANDLE_INFORMATION structure, you will find links to code that can be used to enumerate processes and all open handles of each running process. This information can be used to obtain a HANDLE to the process that has the file open as well as its HANDLE for the file. You would then use DuplicateHandle() to create a copy of the file HANDLE, but in the TFTP process' handle table. The duplicated HANDLE could then be used by the TFTP process with LockFileEx().
This solution relies on an internal function, NtQuerySystemInformation(), and an undocumented system information class value that can be used to enumerate open handles. Note that this feature of NtQuerySystemInformation() "may be altered or unavailable in future versions of Windows". You might want to use an SEH handler to guard against access violations were that to happen.
As tools from MS like OH and Process Explorer do it, it is definitely possible to get all the handles opened by a process. From there to wait on what you'd like the road is still long, but it is a beginning :)
If you have no success with the Win32 API, one place to look at is for sure the NT Native API http://en.wikipedia.org/wiki/Native_API
You can start from here http://msdn.microsoft.com/en-us/library/windows/desktop/ms724509%28v=vs.85%29.aspx and see if it works with the SystemProcessInformation flag.
Look also here for a start http://nsylvain.blogspot.com/2007/09/how-list-all-open-handles.html
The native API is poorly documented, but you can find resources online (like here http://www.osronline.com/article.cfm?id=91)
As a disclaimer, I should add that the Native API is somehow "internal", and therefore subject to change on future versions. Some functions, however, are exposed also publicly in the DDK, at kernel level, so the likelihood of these functions to change is low.
Good luck!