Did anyone encounter this behavior also and knows a solution? so_timeout seems the parameter to enlarge but I had no success with this.
In the log files I found caused by pipe closed.
A manual sftp and 'ls *' command took more as 20 minutes to get a listing back. So I guess it is a Camel timeout. Can this set per route?
2020-02-07T15:54:29,624 WARN [com.bank.fuse.filetransfer.config.bankFileTransferManagerLoggingNotifier] (Camel (rabobank-file-transfer-manager-core) thread #4494 - sftp://server.eu/outgoing/attachments) ExchangeFailedEvent | RouteName: SAPSF-ONE-TIME-MIGRATION-18 | OriginatingUri: sftp://server.eu/outgoing/attachments?antInclude=*.pgp&consumer.bridgeErrorHandler=true&delay=20000&inProgressRepository=%23inProgressRepo-SAPSF-ONE-TIME-MIGRATION&knownHostsFile=%2Fhome%2Fjboss%2F.ssh%2Fknown_hosts&move=sent&onCompletionExceptionHandler=%23errorStatusOnCompletionExceptionHandler&password=xxxxxx&privateKeyFile=%2Fhome%2Fjboss%2F.ssh%2Fid_rsa&readLock=none&soTimeout=1800000&streamDownload=true&throwExceptionOnConnectFailed=true&username=account | completedWithoutException: false | toEndpoint: | owner: [SAP] | sm9CI: [CI5328990] | priority: [Low] | BreadcrumbId: ID-system-linux-bank-com-42289-1580217016920-0-5929700 | exception: Cannot change directory to: ..
Maybe soTimeout=1800000 was too short. A manual sftp and ls * command took about 20 minutes.
Since this was a one time action. I resolved it with a manual sftp.
I'm working on an security oriented project based on Erlang. This application needs to access some parts of the system restricted to root or other privileged users. Currently, this project will only work on Unix/Linux/BSD systems and should not alter files (read-only access).
I've thought (and tested) some of these solutions, but, I don't know what should I take with Erlang. What is the worst? What is the best? What is the easiest to maintain?
Thanks!
1 node (as root)
This solution is the worst, and, I want to avoid it even on testing servers.
_____________________________
| |
| (root) |
| ___________ _______ |
| | | | | |
| | Erlang VM |<---| Files | |
| |___________| |_______| |
|_____________________________|
You can see here a big picture of what I don't currently want.
#!/usr/bin/env escript
main([]) ->
ok;
main([H|T]) ->
{ok, Data} = file:read_file(H),
io:format("~p: ~p~n", [H,Data]),
main(T).
Run it as root, and voilà.
su - root
${script_path}/readfile.escript /etc/shadow
1 node (as root) + 1 node (as restricted user)
I need to start 2 nodes, one running as root or with another privileged user and one other running node with restricted users, easily accessible from outside world. This method work pretty well but has many issue. Firstly, I can't connect to privileged user node with standard Erlang distributed protocol due to remote procedure call between connected nodes (restricted node can execute arbitrary commands on privileged node). I don't know if Erlang can actually filter RPC before executing them. Secondly, I need to manage two nodes on one host.
________________ ____________________________
| | | |
| (r_user) | | (root) |
| ___________ | | ___________ _______ |
| | | | | | | | | |
| | Erlang VM |<===[socket]===>| Erlang VM |<---| Files | |
| |___________| | | |___________| |_______| |
|________________| |____________________________|
In following examples, I will start two Erlang shell. The first shell will be in restricted mode:
su - myuser
erl -sname restricted -cookie ${mycookie}
The second one will run with a privileged user:
su - root
erl -sname privileged -cookie ${mycookie}
Standard Erlang RPC (not enough security)
Finally, on restricted node (via shell for this example), I can have access to all files:
{ok, Data} = rpc:call(privileged, file, read_file, ["/etc/shadow"]).
With "Firewall" Method
I'm using local unix socket in this example, supported only until R19/R20.
Restricted user need to have access to this socket, stored somewhere in
/var/run.
1 node (as restricted user) + external commands (with sudo)
I give the right to Erlang VM process to execute commands with sudo. I just need to execute specific program, get stdout and parse it. In this case, I need to use existing available programs from my system or create a new one.
________________ _______________________
| | | |
| (r_user) | | (root) |
| ___________ | | ______ _______ |
| | | | | | | | | |
| | Erlang VM |<===[stdin]===>| sudo |<---| Files | |
| |___________| | | |______| |_______| |
|________________| |_______________________|
1 node (as restricted user) + ports (setuid)
I create a ports set with setuid flag. This program has now right to read from files but, I need to place it in secure place on the server. If I want to make it dynamic, I should also define a strict protocol between Erlang VM and this ports. IMO, setuid is rarely a good answer.
________________ ________________________
| | | |
| (r_user) | | (root) [setuid] |
| ___________ | | _______/ _______ |
| | | | | | | | | |
| | Erlang VM |<===[stdin]===>| ports |<---| Files | |
| |___________| | | |_______| |_______| |
|________________| |________________________|
1 node (as restricted user) + NIF
I don't think I can give specific right to a NIF inside Erlang VM, maybe with capsicum or other non-portable/OS-specific kernel features.
_______________
| |
| (r_user) |
| ___________ |
| | | |
| | Erlang VM | |
| |___________| |
| | | |
| | NIF | |
| |___________| | _______
| | | | | |
| | ??? |<---| Files |
| |___________| | |_______|
|_______________|
1 node (as restricted user) + 1 daemon (as root)
I can create a daemon running as root, connected to Erlang VM with an Unix Socket or another methods. This solution is a bit like ports or external command with sudo, except I need to manage a long living daemon with privilege.
________________ _________________________
| | | |
| (r_user) | | (root) |
| ___________ | | ________ _______ |
| | | | | | | | | |
| | Erlang VM |<===[socket]==>| daemon |<---| Files | |
| |___________| | | |________| |_______| |
|________________| |_________________________|
Custom Erlang VM
OpenSSH and lot of other secure software runs as root and create 2 interconnected process with pipes. When starting Erlang VM as root, 2 processes are spawned, one as root, and another in restricted user. When some action require root privilege, restricted process send a request to root process and wait for its answer. I guess its the more complex solution currently, and I don't master enough C and Erlang VM to make this thing working well.
______________ _______________
| | | |
| (root) | | (r_user) |
| __________ | | ___________ |
| | | | | | | |
| | PrivProc |<===[pipe]===>| Erlang VM | |
| |__________| | | |___________| |
|______________| |_______________|
From security perspective your best option is to minimise the amount and complexity of code running with root privileges. So I would rule out all the options when you run a whole Erlang VM as root - there's simply too much code there to lock it down safely.
As long as you only need to read some files, the best option would be to write a small C program that you run from the Erlang VM with sudo. All this program has to do is to open the file for you and hand over the file descriptor to the Erlang process via a Unix socket. I used to work on a project that relied on this technique to open privileged TCP ports, and it worked like a charm. Unfortunately that wasn't an open source project, but with some googling I found this library that does exactly the same thing: https://github.com/msantos/procket
I'd advise you to fork procket and strip down it a bit (you don't seem to need icmp support, only regular files opened in read-only mode).
Once you have the file descriptor in the Erlang VM, you can read from it in different ways:
Using a NIF like procket:read/2 does.
Access the file descriptor as an Erlang port, see the network sniffing example in the procket docs.
I am trying to add a file to selenium IDE from a shared network location. When I type in the location of the file then click on another command in the case it adds extra \ to the file location which results in the file not being uploaded.
\\qtc-nas.citicus.com\backups\CB\test.txt
Any help would be appreciated
http://www.w3schools.com/js/js_strings.asp
You need to escape the slashes:
[info] Executing: |store | \\xyz\abc\ | c |
[info] Executing: |echo | ${c} | |
[info] echo: \\xyz\abc\
I'm trying to run a Camel (Spring based) project on tanuki service wrapper as a windows service.
I tried both integration methods #1 and #2: WrapperSimpleApp, WrapperStartStopApp and both resulted in the following behavior:
Camel started OK and IMMEDIATELY wrapper shuts it down!
At first I suspected this has to do with Camel non daemon threads, but after playing with the integration methods and org.tanukisoftware.wrapper.WrapperStartStopApp.systemThreadCount system property, I realized there must be another reason why wrapper shuts down the JVM.
Would anyone know why?
Here is wrapper and Camel log - note 2nd and 3rd lines:
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | 2012-09-01 20:39:45,596 INFO spring.SpringCamelContext Apache Camel 2.6.0 (CamelContext: main-camel-context) started in 1.450 seconds
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | WrapperSimpleApp Debug: main method completed
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | WrapperManager Debug: ShutdownHook started
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | WrapperManager Debug: WrapperManager.stop(0) called by thread: Wrapper-Shutdown-Hook
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | WrapperManager Debug: Send a packet STOP : 0
INFO | jvm 1 | 2012/09/01 20:39:45.643 | srvmain | WrapperManager Debug: Pausing for 1,000ms to allow a clean shutdown...
DEBUG | wrapperp | 2012/09/01 20:39:45.643 | srvmain | read a packet STOP : 0
DEBUG | wrapper | 2012/09/01 20:39:45.643 | srvmain | JVM requested a shutdown. (0)
DEBUG | wrapper | 2012/09/01 20:39:45.643 | srvmain | wrapperStopProcess(0, FALSE) called.
DEBUG | wrapper | 2012/09/01 20:39:45.643 | srvmain | Sending stop signal to JVM
DEBUG | wrapperp | 2012/09/01 20:39:45.643 | srvmain | send a packet STOP : NULL
INFO | jvm 1 | 2012/09/01 20:39:45.752 | srvmain | WrapperManager Debug: Received a packet STOP :
INFO | jvm 1 | 2012/09/01 20:39:45.752 | srvmain | WrapperManager Debug: Stopped checking for control events.
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: Thread, Wrapper-Shutdown-Hook, handling the shutdown process.
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: calling listener.stop()
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperSimpleApp Debug: stop(0)
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: returned from listener.stop() -> 0
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: shutdownJVM(0) Thread: Wrapper-Shutdown-Hook
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: wait for 0 shutdown locks to be released.
INFO | jvm 1 | 2012/09/01 20:39:46.735 | srvmain | WrapperManager Debug: Send a packet STOPPED : 0
DEBUG | wrapperp | 2012/09/01 20:39:46.735 | srvmain | read a packet STOPPED : 0
DEBUG | wrapper | 2012/09/01 20:39:46.735 | srvmain | JVM signaled that it was stopped.
INFO | jvm 1 | 2012/09/01 20:39:46.954 | srvmain | WrapperManager Debug: Closing backend connection.
INFO | jvm 1 | 2012/09/01 20:39:46.954 | srvmain | WrapperManager Debug: Closed backend socket (Normal): java.net.SocketException: socket closed
INFO | jvm 1 | 2012/09/01 20:39:46.954 | srvmain | WrapperManager Debug: Returned from backend handler.
DEBUG | wrapperp | 2012/09/01 20:39:46.954 | srvmain | socket read no code (closed?).
DEBUG | wrapperp | 2012/09/01 20:39:46.954 | srvmain | Closing backend socket.
INFO | jvm 1 | 2012/09/01 20:39:47.500 | srvmain | WrapperManager Debug: Server daemon shut down
INFO | jvm 1 | 2012/09/01 20:39:47.500 | srvmain | WrapperManager Debug: ShutdownHook complete
ERROR | wrapper | 2012/09/01 20:40:07.608 | srvmain | Shutdown failed: Timed out waiting for the JVM to terminate.
ERROR | wrapper | 2012/09/01 20:40:07.827 | srvmain | JVM did not exit on request, terminated
STATUS | wrapper | 2012/09/01 20:40:08.778 | srvmain | <-- Wrapper Stopped
Here is a thread dump I ran on Camel (standalone):
"RMI TCP Connection(idle)" daemon prio=6 tid=0x000000000f194800 nid=0x1a24 waiting on condition [0x000000001062e000]
"RMI TCP Connection(6)-10.244.123.73" daemon prio=6 tid=0x000000000f194000 nid=0x1218 runnable [0x000000001038e000]
"RMI TCP Connection(4)-10.244.123.73" daemon prio=6 tid=0x000000000f193000 nid=0x188c in Object.wait() [0x00000000104ad000]
"RMI TCP Connection(idle)" daemon prio=6 tid=0x000000000eb1f000 nid=0xbf8 waiting on condition [0x000000001020f000]
"JMX server connection timeout 20" daemon prio=6 tid=0x000000000e4e1800 nid=0x1804 in Object.wait() [0x00000000100af000]
"RMI TCP Connection(idle)" daemon prio=6 tid=0x000000000eba3000 nid=0x10e8 waiting on condition [0x000000000fe7e000]
"RMI Scheduler(0)" daemon prio=6 tid=0x000000000f5c3000 nid=0x103c waiting on condition [0x000000000ff8f000]
"RMI TCP Connection(idle)" daemon prio=6 tid=0x000000000e0d0800 nid=0xd5c waiting on condition [0x000000000fd1e000]
"Camel (main-camel-context) thread #1 - file://./toAggregator" daemon prio=6 tid=0x000000000e0bf800 nid=0x18e4 runnable [0x000000000dcaf000]
"Camel (main-camel-context) thread #0 - AggregateTimeoutChecker" daemon prio=6 tid=0x000000000ecad800 nid=0x1b54 waiting on condition [0x000000000fb6f
"RMI TCP Accept-0" daemon prio=6 tid=0x000000000d4e6800 nid=0x1ba0 runnable [0x000000000de0e000]
"RMI TCP Accept-7780" daemon prio=6 tid=0x000000000d586800 nid=0xff0 runnable [0x000000000e8fe000]
"RMI TCP Accept-0" daemon prio=6 tid=0x000000000d55e000 nid=0x38c runnable [0x000000000e7df000]
"Service Thread" daemon prio=6 tid=0x000000000b7d3800 nid=0x1980 runnable [0x0000000000000000]
"C2 CompilerThread1" daemon prio=10 tid=0x000000000b7d0800 nid=0x1be4 waiting on condition [0x0000000000000000]
"C2 CompilerThread0" daemon prio=10 tid=0x000000000b7c3800 nid=0x594 waiting on condition [0x0000000000000000]
"Attach Listener" daemon prio=10 tid=0x000000000b7c2800 nid=0x10bc runnable [0x0000000000000000]
"Signal Dispatcher" daemon prio=10 tid=0x000000000b7bb000 nid=0x1b2c waiting on condition [0x0000000000000000]
"Finalizer" daemon prio=8 tid=0x0000000002416000 nid=0x1b94 in Object.wait() [0x000000000cd5e000]
"Reference Handler" daemon prio=10 tid=0x000000000240f000 nid=0x9a0 in Object.wait() [0x000000000caae000]
"Thread-1" prio=6 tid=0x000000000e530800 nid=0xc50 runnable [0x000000000da7f000]
"DestroyJavaVM" prio=6 tid=0x000000000231f000 nid=0x199c in Object.wait() [0x00000000026ff000]
"VM Thread" prio=10 tid=0x000000000b732800 nid=0x17a0 runnable
"GC task thread#0 (ParallelGC)" prio=6 tid=0x0000000002365000 nid=0x1a94 runnable
"GC task thread#1 (ParallelGC)" prio=6 tid=0x0000000002367000 nid=0x4e8 runnable
"GC task thread#2 (ParallelGC)" prio=6 tid=0x0000000002368800 nid=0xf44 runnable
"GC task thread#3 (ParallelGC)" prio=6 tid=0x000000000236a000 nid=0xd30 runnable
"VM Periodic Task Thread" prio=10 tid=0x000000000d4fb800 nid=0x384 waiting on condition
Read this FAQ from Apache Camel:
http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
Well, the issue was indeed lake of non daemon threads, due to the way Camel was started.
Camel was started using oeg.apache.camel.spring.Main.start() rather than run().
start() just starts Camel counting on the main thread to somehow keep running. this is suitable when running in an application.
run(), on the other hand, blocks the main thread till Camel or the JVM exit.
It is also usefull to call enableHangupSupport() to register a shutdown hook to allow Camel to gracefully shutdown on JVM exit.
I must give credit where credit is due :-) thanks to the super professional team of tanuki java service wrapper for figuring this out for me.
I often have multiple viewports opened in vim, using the :vsp and :sp commands. After I've been editing for a while, I'll often run the :make command from within vim. When I get errors, vim will then show me the lines that gcc says caused my errors. However, vim will often open the file with errors in another viewport, even if that file is already open. An Example:
Before Make
--------------------
| | |
| file 1 | file 2 |
| | |
| | |
--------------------
Ok, assume there are errors in file 2
--------------------
| | |
| file 2 | file 2 |
| | |
| | |
--------------------
vim now jumps to the error line in the left viewport, even though the right viewport already had that file open.
Is there some way to tell vim not to use the file one viewport if the file that the error is in is already open in vim?
Try setting the option switchbuf=useopen.