I'm trying to automate the process of doing
git add .
git commit -m "some message"
git push origin master
In C.
I've already done a bash script with Expect
#!/bin/bash
HOME="/home/prxvvy"
filename="$HOME/Documents/github_creds.txt"
username=`awk 'NR==1' $filename`
password=`awk 'NR==3' $filename`
echo "Enter a message for commit:"
read message
expect <<EOS
spawn git add .
expect
spawn git commit -m "$message"
expect
spawn git push origin master
expect "Username for 'https://github.com':"
send "$username\r"
expect "Password for 'https://$username#github.com':"
send "$password\r"
expect eof
EOS
So how would I "send" the requested data when the "Username for 'https://github.com':" prompt comes up, with C?
PS: I don't want to use any git api or something as I did in the bash script
Answer #1: you can't. The problem here has nothing to do with Git per se. Programs that read passwords from humans make sure they're not talking to other programs, by making sure they're using the human /dev/tty interface. So your program can't push a password into another program.
Answer #2: you can, but you must re-implement expect. The expect program tricks other programs into thinking they are talking to a human. This is fairly complicated and involves opening a "pseudo tty" from the control side, initializing it properly on the dependent side, and doing any OS-level operations required to make it become the /dev/tty that other programs will get when they open it. Then, having done all this song and dance (some of which is available in some libraries via openpty or similar), you can then run a program. You then must implement the rest of expect to read data coming through the control side of the pty pair, interpret it to figure out what the program is doing, and feed it data.
(Note that if all you want to do is run git push without having to enter a password, Git can invoke authentication methods that do this. Git does not have its own built in authentication: it relies on credential helpers when using https and ssh's credentialing and authentication systems when using ssh. So if you're using https:// for your URL, you want to configure a credential helper to do the job, and if you're using ssh:// for your URL, you want to configure ssh. You said that's not what you're interested in, though.)
Related
I'm working on a C project that makes connections to remote servers. Commonly, this involves using some small terminal macros I've added to my makefile to scp an executable to that remote server. While convenient, the only part of this I've not been able to readily streamline is the part where I need to enter the password.
Additionally, in my code, I'm already using system() calls to accomplish some minor terminal commands (like sort). I'd ALSO like to be able to enter a password if necessary here. For instance, if I wanted to build a string in my code to scp a local file to my remote server, it'd be really nice to have my code pull (and use) a password from somewhere so it can actually access that server.
Does anyone a little more experienced with Make know a way to build passwords into a makefile and/or a system() call in C? Bonus points if I can do it without any third-party software/libraries. I'm trying to keep this as self-contained as possible.
Edit: In reading responses, it's looking like the best strategy is to establish a preexisting ssh key relationship with the server to avoid the login process via something more secure. More work up front for less work in the future, by the sound of it, with additional security.
Thanks for the suggestions, all.
The solution is to not use a password. SSH, and thus SCP, has, among many many others, public key authentication, which is described all over the internet. Use that.
Generally, the problem you're trying to solve is called secret management, and the takeaway is that your authentication tokens (passwords, public keys, API keys…) should not be owned by your application software, but by something instructing the authenticating layer. In other words, the way forward really is that you enable SSH to connect on its own without you entering a password by choosing something that happens to not be an interactive authentication method. So, using a password here is less elegant than just using the generally favorable method of using a public key to authenticate with your server.
Passing passwords as command line option is generally a bad idea – that leaks these passwords into things like process listings, potentially log entries and so on. Don't do it.
Running ssh-keygen to create the keys. Then, adding/appending the local system's (e.g) .ssh/id_rsa.pub file to the remote's .ssh/authorized_keys file is the best way to go.
But, I had remote systems to access without passwords but the file was not installed on the remote (needing ssh-keygen to be run on the remote). Or, the remote .ssh/authorized_keys files did not have the public key from my local system in it.
I wanted a one-time automated/unattended script to add it. A chicken-and-the-egg problem.
I found sshpass
It will work like ssh and provide the password (similar to what expect does).
I installed it once on the local system.
Using this, the script would:
run ssh-keygen on the remote [if necessary]
Append the local .ssh/id_rsa.pub public key file to the remote's .ssh/authorized_keys
Copy back the remote's .ssh/id_rsa.pub file to the local system's .ssh/authorized_keys file [if desired]
Then, ssh etc. worked without any passwords.
UPDATE:
ssh_copy_id is your fried, too.
I had forgotten about that. But, when I was doing this, I had more complex requirements.
The aforementioned script would merge/combine all the public keys and update all the authorized_keys files on all the systems. This would be repeated anytime any new system was added to the mix.
you never need to run ssh-keygen on a remote host, especially not to generate an authorized_keys file. –
Marcus Müller
I think that was inferred but not implied as a requirement [particularly in context]. I hope the answer wasn't -1 for that.
Note that (1) ssh-keygen is needed for (3) copy back the public key.
Ironically, one of the tutorial pages for ssh-copy-id says run ssh-keygen first ...
It's been my exerience when setting up certain types of systems/clusters (e.g. a development host/PC and several remote/target/test ones), if one wants to do local-to-remote actions, invariably one wants to do:
remote-to-local actions -- (e.g.) I'm ssh'ed into a remote system and want to do rcp back to the development system.
The remote system needs to do a git clone/pull from [and, sometimes, git push to] the local git server.
remote-to-remote -- copying/streaming data between target systems.
This requires that each system have a private/public key pair and all systems have an authorized_keys file that has the public keys of all the other systems.
When I've not set up the systems that way it usually comes back to haunt me [usually late at night when I'm tired]. So, I just [axiomatically] set it up that way at the outset.
One of the reasons that I developed the script in the first place. Also, since we didn't want to have to maintain a fork of a given system/distro installer for production systems, we would:
Use the stock/standard distro installer CD/USB
Use the script to add the extra/custom config, S/W, drivers, etc.
I am trying to automate SSH login by using password. I can't use expect command or sshpass etc. So I am left with only option to use password directly.
Did lot of research in google and didn't get any solution... :(
Please help me with this.
The code I tried is.
#!/bin/bash
USERNAME=user1
PASSWORD=abcd1234
HOSTS="server01.mat.us"
ssh ${HOSTS} -l ${USERNAME} -p ${PASSWORD}
The OpenSSH ssh utility doesn't accept a password on the command line or on its standard input. This also applies to the scp and sftp file transfer utilities, which invoke ssh to make the SSH connection. I believe this is a deliberate decision on the part of the OpenSSH developers. You have these options available to you:
Use an SSH key for authentication, instead of a password.
Use sshpass, expect, or a similar tool to invoke ssh through a TTY and automate responding to the password prompt.
Use (abuse) the SSH_ASKPASS feature to get ssh to get the password by running another program, described here or here, or in some of the answers here.
Get the SSH server administrator to enable host-based authentication and use that. Note that host-based authentication is only suitable for certain network environments. See additional notes here and here.
Write your own ssh client using perl, python, java, or your favorite language. There are ssh client libraries available for most modern programming languages, and you'd have full control over how the client gets the password.
Download the ssh source code and build a modified version of ssh that works the way you want.
Use a different ssh client. There are other ssh clients available, both free and commercial. One of them might suit your needs better than the OpenSSH client.
I'd recommend to use ssh keys instead of password if it's possible.
This script can help you to upload your public key to desired remote machine:
https://github.com/aprey10/ssh-authorizer
I have a use case to deploy a compiled Native C executable as a Microservice on PCF:
The compiled executable is called like so:
"mycbinary inputfile outputfile"
and terminates after the operation. The Microservice is thus not an LRP.
It is possibly a Task in PCF palance, but it does not rely on the existence of other Microservices.
It must be a standalone Microservice but not a long-running one.
How can I achieve this use case with PCF please, i.e what possibilities do I have?
The Microservice terminates when the binary is done with its work until it is needed again.
To test the feasibility of what I could do, I tried pushing some compiled C code to PCF-DEV.
I am using cf push since that's my understanding of a standalone App on PCF
cf push HelloServiceAgain -c './helloworld' -b https://github.com/cloudfoundry/binary-buildpack.git -u process --no-route
The push crashed with the following message:
Waiting for app to start...
Start unsuccessful
TIP: use 'cf.exe logs HelloService --recent' for more information
In the log file there was this entry:
OUT Process has crashed with type: "web"
Then I pushed another command which expects parameters. This started without a problem, but the same message in the log file
cf push HelloServiceGCC -c 'gcc -o ./hellogcc ./hello1.c' -b https://github.com/cloudfoundry/binary-buildpack.git -u process --no-route
I have the following additional questions please:
1) Is the message "Process has crashed with type: "web" an ERROR? And why is the command called multiple times?
2) The second push which succeeded is supposed to create a compiled hellogcc executable which I expect to see in the same root directory. Where is the output file created and how can I access it from the local file system?
Sorry for asking so many questions but I am a newbie in the PCF business.
The Microservice is thus not an LRP. It is possibly a Task in PCF palance, but it does not rely on the existence of other Microservices. It must be a standalone Microservice but not a long-running one.
It's definitely a task. It's fine that it does not rely on other services. A task is just simple a process that runs for a finite amount of time and exits 0 for success or something else for error.
cf push HelloServiceAgain -c './helloworld' -b https://github.com/cloudfoundry/binary-buildpack.git -u process --no-route
I would recommend using this slight variation: cf push HelloServiceAgain -c 'sleep 5000' -b binary_buildpack -u process --no-route.
This will...
Assume that the compiled binary is in the current directory.
Uses the system provided binary buildpack which should be fine.
Sets the health check to be based on the process & sets no route.
Runs sleep, which is merely to pass the health check.
The purpose of this is so that your application will upload & stage. We set the command to sleep because we just need the app to stage and start (this is a workaround to make sure that staging occurs, you have to run once to trigger staging at least at the moment). Once the app starts, just run cf stop and stop the app. Since all you have is the task, you don't need the app to continue running. If you update your app, you do need to follow this process again though, as that will restage your changes.
At this point, you can cf run-task and execute your program. Ex: cf run-task app-name './helloworld'.
Hope that helps!
I have created an application to run on an Olinuxino Maxi board which is presently running an Arch Linux ARM distribution. My somewhat simple application can be considered to be in two parts:
A program that performs communication between RS232 and TCP/IP, and initiates / accepts VOIP calls via the Linphone library. How this program behaves is configured through a .conf file. This program starts up on boot. I achieved the start up boot by creating a .service file for it and then enabling it using systemctl / systemd.
A simple web page that is accessed via Lighttpd. The CGI page is written in C. This page provides means for a user to edit the .conf file through a simple form, and therefore configure the operation of the main program.
All of the above now works. The specific problem I have relates to how I can cause my service program to restart (so that it configures itself again from the .conf file) when the user submits new settings via the web page. I'm stuck on this areas because, while I'm a fairly experienced C programmer, doing development on Linux and general Linux administration is fairly new ground to me.
In case it's relevant, I'll discuss a bit about how I've set this up, including how I've set up users and so forth:
I've set up a new user of the name of the application. Call it user application-name.
The RS232/TCP/IP/VOIP program resides in the folder /home/application-name/. The .conf file also resides in here.
systemd starts the program on boot. I understand that the program is being run as root.
The web / CGI code is located in /home/application-name/web/. I've set up an alias in the Lighttpd configuration is that /cgi-bin/ points to here, and that works.
The Lighttpd server, which I understand is run as user 'http', happily accesses the web page and, on submitting of POST data, edits the ../.conf file accordingly. To allow the web server to edit the .conf file I did have to chmod that file to allow write access to others, but I am guessing that a better way to do this would be to put users application-name and http into a new user group (though I'd appreciate advice on this also).
After processing of the POST data, my C CGI program also uses system() to call a bash script, restart_application.sh.
Inside restart_application.sh, I'm making a call to systemctl to restart my main program. But it doesn't work, and I gather it doesn't work because no user except root can invoke systemctl.
So the main question is:
How should I make my program restart?
And also:
If I'm doing any absolute horrors here in terms of my setup and Linux system administration, please by all means shout angrily.
Edit 1: Unless anyone has a better approach, I'm thinking of trying the idea suggested here which is to basically 'sudo' within the bash file.
I have implemented automation through batch file to login into a remote server and run a series of commands therin. My batch file contents are as follows:
#ECHO OFF
ECHO 1 -^> Connect 2 Server
ECHO 2 -^> Quit
SET /p ID=Enter Choice :
IF "%ID%"=="1" start "" C:\work\putty.exe -load SessionName -l UserName -pw Password -m cmd.txt
IF "%ID%"=="2" exit
My cmd.txt file contains the following commands:
su superuser
password
ssh server
password
su MyID
password
The batch successfully connects to the server; but it fails to retain the session. Any tweak would be appreciated.
For my own use I verified what -m switch does for putty
The -m option performs a similar function to the ‘Remote command’ box in the SSH panel of the PuTTY configuration box (see section 4.18.1). However, the -m option expects to be given a local file name, and it will read a command from that file.
With some servers (particularly Unix systems), you can even put multiple lines in this file and execute more than one command in sequence, or a whole shell script; but this is arguably an abuse, and cannot be expected to work on all servers. In particular, it is known not to work with certain ‘embedded’ servers, such as Cisco routers.
Roughly what I guessed. I think what's happening is that your commands might be running ok, but it terminates the session when done (which is also default behaviour for ssh [command] on unix. To get around this you could try running 'bash' at the end to start a terminal as that user, which should stay running until you exit. If this doesn't work you could try running 'sleep 10' to see if the session stays running longer - should verify that's the problem we're dealing with.
Much as stackoverflow demands strictly the answer to the question, which I hope I've provided above. I feel in this case it would be remiss to not flag the serious security concerns I'd have on what you're doing. I would maintain that you shouldn't answer a question about doing something securely (with ssh and root passwords involved) without covering potential security pitfalls.
What you're doing seems like a bad idea because:
it makes a file with two unencrypted root-capable passwords to anyone who finds that file
this script is also giving you root access by default, which isn't a safe way to operate. Ever accidentally type rm -r in the wrong window?
Also, someone messing with your second hop (by say, replacing the ssh server with a script to run ssh but log passwords) would get the password to the third hop with root privileges.
SSH keys should do away with using passwords to log on. Use them.
SSH port forwarding lets you also skip sshing from the second hop - instead just tunnel through it (I'm guessing the third box isn't internet accessible), again using ssh keys lets you skip passwords.
the first 'su superuser' shouldn't be necessary unless your permissions don't allow you ssh (does an ssh_allow group exist on that box?), which might be the problem you should actually fix - not being a member of the group allowed to use SSH.