I have written scripts for Windows and Linux to essentially set up a new users workspace with all the git repositories from our server.
I would like the user to enter the password for our server once, store it in a local variable, pass that variable to each git pull command, then erase the password variable and exit.
How can I input the password when the git pull command requests it? Both for Windows batch file and a Linux shell script.
Here is code from the Linux script:
#!/bin/bash
echo "Enter password: "
read pswd
clear #No screen peaking
#This is repeated for each repo
location=folderName
mkdir $location
cd $location
git init
git remote add origin git#<server>:$location.git
git pull origin master
#Above prompts for password & is where I want to automatically input $pswd
I've tried various things recommended on SO and elsewhere, such as piping, reading from .txt file, etc. I would prefer to not need anything more than plain old windows cmd and Linux terminal commands. And as this script is just for set up purposes, I do not need to securely store the password permanently with something like ssh agent.
I'm running Windows 7 and Ubuntu 12.10, but this script is meant for setting up new users, so it should ideally work on most distributions.
Synopsis:
git pull "https://<username>:<password>#github.com/<github_account>/<repository_name>.git" <branch_name>
Example:
git pull "https://admin:12345#github.com/Jet/myProject.git" master
Note: This works for me on a bash script
I would really recommend to not try and manage that password step, and delegate that (both on Linux and Windows) to git credential helper.
See:
"Git http - securely remember credentials"
"How to use git with gnome-keyring integration"
The user will enter the password only once per session.
Read the remote url from git and then insert the ID and password (PW) to the url might work.
For example try the following:
cd ${REPOSITORY_DIR}
origin=$(git remote get-url origin)
origin_with_pass=${origin/"//"/"//${USER_ID}:${USER_PW}#"}
git pull ${origin_with_pass} master
Related
I am using Jenkins on a Windows7 system. I would like to use it to execute a batch script on a remote Windows system. The batch script will be used to flash a development board and run some tests. I came across psexec.exe. That works well through a command prompt window--I can connect and run the script without any issues, but when I try to have Jenkins do it, I get the following output:
PsExec v2.11 - Execute processes remotely
Copyright (C) 2001-2014 Mark Russinovich
Sysinternals - www.sysinternals.com
The handle is invalid.
Connecting to ABCDEFG...
Couldn't access ABCDEFG:
Connecting to ABCDEFG...
Build step 'Execute Windows batch command' marked build as failure
The command I am using in both cases is:
psexec.exe \\ABCDEFG -u "DOMAIN\username" -p "password" "C:\test.bat"
The user associated with username has administrator privileges on the remote system (ABCDEFG is not the real name of the system).
Can anyone help me figure out why it is not working through Jenkins? Or, is there an easier/better way to execute a batch script on a remote Windows system through Jenkins?
Thanks to all your help, especially Technext, I have a solution.
I needed run "services.msc", find "Jenkins", right click on it, and go to "Properties". Once the properties windows appeared, I had to click the "Stop" button to stop Jenkins, open the "Log On" tab, enter in my username and password (the username I used when running through command prompt), and start Jenkins again. That got rid of the "handle is invalid" message in Jenkins.
Update:
A better solution was to go onto the system I was using psexec.exe to get onto, go to Control Panel > User Accounts > Give other users access to this computer. Click on "Add..." and type in the username and domain Jenkins uses to run its commands (to find this, open your Jenkins in a browser window, go to Manage Jenkins > System Information and look for USERNAME and USERDOMAIN under Environment Variables). Make sure you give it Administrator rights. Then click ok. Now psexec.exe shouldn't have the "handle is invalid" issue.
Sorry, I don't have enough reputation for comments, but is the single \ a typo? Since
The handle is invalid.
probably means that the computer address is invalid. Try
psexec.exe \\ABCDEFG -u "DOMAIN\username" -p "password" "C:\test.bat"
Notice the two backslashes to access a locally mapped computer.
otherwise if that does not work i recommend the # tag
psexec.exe #servername.txt -u "DOMAIN\username" -p "password" "C:\test.bat"
where #servername.txt is a textfile containing only the servernames, one per line. The file parameter handles the formatting of \
ex servername.txt
ABCDEFG
COMPUTER2
EDIT: also found after some quick googling that it can be related to windows security.
Check out that a simple restart of the remote machine doesn't solve the problem. Also, adding parameters -h and -accepteula may help. Modified command:
psexec.exe \\ABCDEFG -u "DOMAIN\username" -p "password" -h -accepteula "C:\test.bat"
I execute below code from Jenkins pipeline groovy script to connect dynamically created VM as a resource on Jenkins master. Below code connect dynamically created VM as resource on Jenkins master with 4 executors. You can change the number of executors based on your requirement.
bat label: 'ConnectResource', script: """
#echo OFF
C:\\apps\\tools\\psexec \\\\${machine_ip} -u ${machine_ip}\\${machine_username} -p ${machine_password} -accepteula -d -h -i 1 cmd.exe /c "cd C:\\apps\\jenkins\\ & java -jar C:\\apps\\jenkins\\swarm.jar -master http://pnlv6s540:8080 -username ${jenkins_user_name} -password ${jenkins_user_password} -name ${machine_ip}_${BUILD_NUMBER} -labels ${machine_ip}_${BUILD_NUMBER} -deleteExistingClients -disableClientsUniqueId -executors 4" & ping 127.0.0.1 -n 60 > nul
"""
I have the requirement as i need to connect to the rtc and automatically checkout the files from the stream to the repository workspace.
I am writing the following commands in the bat file.
lscm login -r https://rtc.usaa.com/ccm -u uname -P password -n nickname -c
scm create workspace (workspacename) -r nickname -s (streamname)
lscm load workspace name -r nickname -d directorypath(c:codebase/rtc)
lscm logout -r nickname
while i am executing the above batch file for the first time it is creating the workspace and loading the project into the workspace path.
while i am executing the above batch file for the second time again it is creating the duplicate workspace with the same name and getting exception while loading.
I want to override the already existing workspace every time while loading but I didn't find a command for that.
can you please provide me any other way of doing it or any command that solves my problem
It will be good to delete existing local workspace sandbox before loading the new one. In my setup, we execute the following steps:
1. Delete local sandbox (if it makes sense delete existing repository workspace too)
2. Create new repository workspace
3. Load the new repository workspace to local sandbox
Either create a uniquely named workspace (perhaps by sticking a time stamp into the name?) and then delete it when you're done, or use the workspace's UUID from the creation step.
Instead of deleting and again writing the files into workspace, you can try accept incoming changes before load and then using "--force" attribute you can overwrite only the changes made files.
Accept using - SCM accept --flow-components -r <> -u <> -p <> --target
Use force at the end of the load command which you using.
this should work fine.
We are using Jenkins server for our daily build process and executes some bash scripts on remote hosts over SSH. This scripts are generating html log files on remote hosts.
We are using Copy to slave plugin to copy files on slave machines and Publish over ssh plugin to manage SSH sessions in build process.
Now the question is, We want to copy some files (log files of Scripts) from remote ssh host to Jenkins Server.
Which will be possible and better option for the same (plugin will be better if any).
EDIT :
sshpass is an option, but looking for any plugin or better way to do the job.
use sshpass command to send file in
Build Environment -> Execute Shell script on remote host using ssh ->
Post build script
sample command :
sshpass -p "password" scp path/of/file <new_server_ip>:/path/of/file
This will skip password prompt for scp command and will provide password to scp.
I think you can generate ssh keypair and pass it to the slave as a parameter with, for example, Config File Provider Plugin
Then just use scp to retrieve files using this keypair for authentication process.
Obviously way too late, but in case you're already using publish-over-ssh, want to avoid duplicating the credentials and have a shared library you can use this piece of groovy to obtain the host configuration:
import jenkins.plugins.publish_over_ssh.*
#NonCPS
def getSSHHost(name) {
def found = null
Jenkins.instance.getDescriptorByType(BapSshPublisherPlugin.Descriptor.class).each{
it.hostConfigurations.each{host ->
if (host.name == name) {
found = host
}
}
}
found
}
As mentioned, this either requires a Global Shared Library (so that your code is trusted) or (probably) a number of admin approvals, sorry for that.
This returns a BapSshHostConfiguration.
For a password connection you can do:
def sshHost = getSSHHost('Configuration Name')
def host = [host: sshHost.hostname, user: sshHost.username, password: sshHost.password]
sshHost = null
sh("""
set +x
sshpass -p "${host.password}" scp -o StrictHostKeyChecking=no ${host.user}#${host.host}:filename.extension .
set -x
""")
This copies the file to your local work directory.
Probably not the best code ever, but I'm not a groovy specialist. It works and that is enough for me. (the set +x is to avoid it echoing the command in the log, showing the password). Getting rid of anything Non-CPS (sshHost = null) before you perform a CPS call saves you a lot of headaches :)
Since it took me quite a while to figure out I wanted to share this for whoever comes next.
I'm trying to download a file from sftp site using batch script. I'm getting the following error:
Permission denied (publickey,password,keyboard-interactive).
Couldn't read packet: Connection reset by peer
When running the command:
sftp -b /home/batchfile.sftp <user>#<server ip>:<folder>
the batchfile.sftp includes these data:
password
lcd [local folder]
cd [sftp server folder]
get *
bye
Note: It's working when running at the prompt as
sftp <user>#<server ip>:<folder>
But I need the ability to enter the password automatically.
You'll want to install the sshpass program. Then:
sshpass -p YOUR_PASSWORD sftp -oBatchMode=no -b YOUR_COMMAND_FILE_PATH USER#HOST
Obviously, it's better to setup public key authentication. Only use this if that's impossible to do, for whatever reason.
If you are generating a heap of commands to be run, then call that script from a terminal, you can try the following.
sftp login#host < /path/to/command/list
You will then be asked to enter your password (as per normal) however all the commands in the script run after that.
This is clearly not a completely automated option that can be used in a cron job, but it can be used from a terminal.
I advise you to run sftp with -v option. It becomes much easier to fathom what is happening.
The manual clearly states:
The final usage format allows for automated sessions using the -b
option.
In such cases, it is necessary to configure non-interactive authentication
to obviate the need to enter a password at connection time (see
sshd(8) and ssh-keygen(1) for details).
In other words you have to establish a publickey authentication. Then you'll be able to run a batch script.
P.S.
It is wrong to put your password in your batch file.
You mention batch files, am I correct then assuming that you're talking about a Windows system? If so you cannot use sshpass, and you will have to switch to a different option.
Two of such options, that follow diametrically opposite philosophies are:
psftp: command-line tool that you can call from within your batch scripts; psftp is part of the PuTTY package and you can find it here http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
Syncplify.me FTP Script: a scriptable FTP/S and SFTP client for Windows that allows you to store your password in encrypted "profile files"; check it out here http://www.syncplify.me/products/ftp-script/
Either way, switching from password to PKI authentication is strongly recommended.
PSFTP -b path/file_name.sftp user#IP_server -hostkey 1e:52:b1... -pw password
the file content is:
lcd "path_file for send"
cd path_destination
mput file_name_to_send
quit
to have the hostkey run:
psftp user#IP_SERVER
You need to use the command pscp and forcing it to pass through sftp protocol. pscp is automatically installed when you install PuttY, a software to connect to a linux server through ssh.
When you have your pscp command here is the command line:
pscp -sftp -pw <yourPassword> "<pathToYourFile(s)>" <username>#<serverIP>:<PathInTheServerFromTheHomeDirectory>
These parameters (-sftp and -pw) are only available with pscp and not scp. You can also add -r if you want to upload everything in a folder in a recursive way.
This command will help you
sshpass -p MYPASSWORD sftp MYUSERNAME#HOST
I want to install postgresql for use as the backend to a Windows application.
This seems to be no problem if postgresql is not already installed on the system.
If postgresql is already installed then unless the command line parameters contain the superpassword etc of the existing installation then the install fails.
As I will likely never know the superpassword or other account details of any pre-existing postgresql instances and the machine owners may not either it seems that this will frustrate any attempt to install postgresql in such a situation.
I believe it is possible to install completely independent instances of sql server but is this possible for postgresql?
BTW:
If the command line does contain the correct superpassword then the install just seems to overwrite the existing install and ignores parameters like --prefix etc .
I used init db to create a new database cluster before doing a second install but this new cluster was ignored?
In general you can have multiple independent instances of PostgreSQL. Strictly speaking it's database cluster with separate:
data directory
configuration (e.g. postgresql.conf, pg_hba.conf)
listening TCP/UDP port (default 5432+)
owner user and superuser role
locale and default encoding
log file
postmaster server process (on Windows postgres.exe)
Perfect well-done example is Debian with easy to use postgresql-common infrastructure (pg_ctlcluster, pg_lsclusters, pg_createcluster, pg_dropcluster, included SSL, log rotation and so on).
EDIT:
I found it's rather easy to install second, third, etc. instance of same versioned PostgreSQL under Windows with EnterpriseDB's installer, no need to use initdb and pg_ctl (assuming 64-bit installation, probably you need to use Program Files (x86) for 32-bit installation):
Open cmd with admin privileges (Run as Administrator)
Execute: cd "C:\Program Files\PostgreSQL\9.0\installer\server"
Create new database cluster (press Enter on every step): initcluster.vbs postgres postgres 12345 "C:\Program Files\PostgreSQL\9.0" "C:\Program Files\PostgreSQL\9.0\data2" 5433 DEFAULT
Register as Windows Service: startupcfg.vbs 9.0 postgres 12345 "C:\Program Files\PostgreSQL\9.0" "C:\Program Files\PostgreSQL\9.0\data2" postgresql-x64-9.0-2
Run newly created service postgresql-x64-9.0-2 using services.msc and you have second server
Change 12345 to your password specified during PostgreSQL installation. You don't have to use data2 directory, use whatever you like (but of course not existing data directory).
On Windows 7 I had success following these steps. You'll need the PsExec.exe utility available in the Sysinternals Suite. I assume here that the path to the Sysinternals Suite and the path to the bin folder of your existing PostgreSQL installation are in your PATH environment variable.
Open a cmd.exe window and enter the following command to open a prompt as the Network Service account.
psexec -i -u "nt authority\network service" cmd.exe
The Network Service account won't have access to your PATH, so cd 'C:\PostgreSQL\9.3\bin' and then enter the following command to initialize a data directory for your new instance. I've called mine "data2". It doesn't have to be in the postgres directory, but that's where the default data directory goes, so it's a reasonable choice.
initdb "C:\PostgreSQL\9.3\data2"
Edit C:\PostgreSQL\9.3\data2\postgresql.conf so that port = 5433 (the default instance uses 5432, and you shouldn't have two instances on the same port)
Leave the Network Service cmd prompt and in your standard prompt enter the following command to register the new service. Here I've named my new instance "pg_test"
pg_ctl register -N pg_test -U "nt authority\network service" -D "C:\PostgreSQL\9.3\data2"
Run the following command to start the service.
net start pg_test
The database owner role will be 'YOURMACHINENAME$'. If you want to change this to the standard 'postgres', you have to first create a new super user role that can rename the owner. From the command prompt, enter the following to create this super user.
createuser -s -r -l -i -P -h localhost -p 5433 -U YOURMACHINENAME$ mysuperuser
Finally, connect to the server with psql (psql -U mysuperuser -h localhost -p 5433 postgres) and enter the following commands to rename your database owner and add a password.
ALTER USER "YOURMACHINENAME$" RENAME TO postgres;
ALTER USER postgres WITH PASSWORD 'yourpassword';
Something like this should work (if not it's probably bug):
postgresql-9.0.4-1-windows_x64.exe ^
--mode unattended ^
--prefix c:\postgres\9.0-second ^
--servicename postgresql-x64-9.0-second ^
--serviceaccount postgres2 ^
--servicepassword <password> ^
--serverport 5433 ^
--superaccount postgres ^
--superpassword <password>
EDIT: after a couple of tests I believe it's not possible to create different Postgres instances of the same version using the One-click installer. Sorry.
OTOH you could always play with initdb and pg_ctl and use the existing installation to create a new instance. It would not be as easy as just starting the installer but it's doable.