This works:
curl --url http://someurl.tld --form "apikey=39485730"
This does not:
curl --url http://someurl.tld --form "apikey=<keyfile"
Error:
Invalid API keys
The cURL manual (http://curl.haxx.se/docs/manpage.html) explicitly states
-F/--form (HTTP) This lets curl emulate a filled-in form in which a user has pressed the submit button. [...] To force the 'content' part to be a file, prefix the file name with an # sign. To just get the content part from a file, prefix the file name with the symbol <. The difference between # and < is then that # makes a file get attached in the post as a file upload, while the < makes a text field and just get the contents for that text field from a file.
FWIW:
If I use --form "apikey=#keyfile" I get the error: "API key is required".
This confirms that # is definately wrong (which I am okay with).
But why does < not work, and what to do about it?
It looks as if the content of the file is either not passed on, or wrong. Thus I've made quadruple sure, that only the api key (here: 39485730) and nothing else is in the file.
If it's important why I am trying to do this:
I need this curl-command in scripts, and don't want to put the API key there. Instead, it should be in the HOME of the user who runs this script, only readable by him, and nobody else.
Thank you in advance for any insight... :-)
I think you might have a trailing newline in keyfile. You can check this with:
xxd keyfile
There should be no 0a at the end. If there is, you can re-create the keyfile without the trailing newline like this:
echo -n 39485730 > keyfile
and try again with the new keyfile.
Related
I am writing a system service on FreeBSD where I need to take user credentials to verify identity, something like this:
./compression_bin -i <input_file> --type=<type> --password=<secret key>
Here, the secret key is used to authenticate user before compressing the given file. Currently, the secret key shows up in history which is bad and can be exploited. Is there a way where above can be invoked without displaying password field:
./compression_bin -i <input_file> --type=<type> --password=*********
The history will always record the text of the commands as they were issued. You could conceivably go back and modify the history file, but not only is that nasty, it still affords a window within which the password can be read.
Moreover, that's not even the easiest exploit. If the password is given on the command line then there are other ways it can be read while the command is running, such as from the output of the ps command, which is accessible to all users.
So don't take the password as a command-line argument. Read it from a file or from the standard input or from a socket, or some other such thing.
I was trying to upload a file through application i wrote in c.
As i did not find any API, i decided to go through commands.
Input command line looked like this.
ftp -u ftp://ftpuser:password#123#x.x.x.x/test.txt /tmp/test.txt
Whenever a special character is present, login will fail. when i tried with different user without any special characters in the password upload works.
How this issue can be resolved or is there any another method available like API which can be made use of.
If any sample code available then it will be of great help.
Special character means #, $, # (Ex : password#123, password$123)
code snippet:
RunCommandWithPipe(PSTRING CmdLine)
{
FILE *fp;
int status;
fp = popen(CmdLine, "r");
if (fp == NULL)
{
ErrGen(constErrOpenFile);
}
status = pclose(fp);
if (status == -1)
{
ErrGen(constErrCloseFile);
}
}
The reason why this doesn't work is because you are passing unfiltered meta characters into the shell. This is very dangerous. If someone untrustworthy gets to decide the value of any of the parameters to your ftp command, such as the username, password, ftp server, or file name, then that person will be able to run arbitrary shell commands.
You can see what's going on by putting an "echo" in front of your ftp command:
echo ftp -u ftp://ftpuser:password$123#x.x.x.x/test.txt /tmp/test.txt
You'll get this result:
ftp -u ftp://ftpuser:password23#x.x.x.x/test.txt /tmp/test.txt
The shell is trying to evaluate $1 as a variable, leaving an empty result.
There's a couple of things you can do.
1) Make the command safe by escaping all the meta characters. Here you need to be very careful, using a whitelist approach rather than just trying to get rid of the special characters you've thought of. In the whitelist approach you accept that some set of characters are safe, such as [A-Za-z0-9:_-]. Every other character you either strip out or escape by preceding it with a backslash. (eg. "foo:bar$baz&abc" becomes "foo:bar\$bazabc") If you do this way don't try to think of all the characters you know of that are special and escape those. You will most likely forget some, and not handle input this like:
ftp -u ftp://ftpuser:; rm -rf /;echo #x.x.x.x/test.txt /tmp/test.txt
2) Don't pass arguments on the shell, instead control the FTP client through fread()/fwrite() on the pipe that popen() gave you.
In this case what you do is launch the ftp client with no arguments. Then you write "OPEN 192.168.1.1" or wherever you want to connect. Then you write the username. Then you write the password. Then you write the GET or PUT command want. Then you write "EXIT" or write an EOF. You should read the result codes from the server. You'll get 200 series results on success. You'll get a 500 series result if the login is bad, etc.
You still have to watch out when piping into the FTP command because it will take shell escapes like "!rm -rf /", but there is much less opportunity for that than on the shell. You just need to make sure the strings you get to build your FTP commands are one line and that you always precede them with a valid FTP command. You should also watch out for any funny business with untrustworthy filenames. (eg. don't allow absolute paths, "..", and so forth)
You propably using a wrong charset to send the password
I am trying to put a file into a Google Cloud Storage (GCS) bucket from the command line. At a later stage this shall be used in a deployed script at the user end without any type of user-visible authentication.
So far I generate a signed url like this:
gsutil signurl -p notasecret -m PUT -d 1d myserviceaccount.p12 gs://mybucket/testfile
which will generate something like
https://storage.googleapis.com/mybucket/testfile?GoogleAccessId=myserviceaccount#developer.gserviceaccount.com&Expires=1430963040&Signature=gMf2h95bNmolizUGYrsQ%2F%2F%2FiHxW14I%2F0EOU3ZSFWtfCwNqSyok3iweQiuPxYXH4b26FeDSrmFOXB58%2B%2B%2BiAOJ%2B1gdLC9Y%2BkeUdbrjH0eGTW0NVsM1AWY2LsQ3dYf5Ho%2Bos1Fk26EsLJlD096Ku9aWqLW%2FpL%2FBSsUIfHijrFJPdI%3D
The next step (at the user end) would be curl uploading the file with a PUT request. Like so:
curl -X PUT --data-binary #testfile 'https://storage.googleapis.com/mybucket/testfile?GoogleAccessId=myserviceaccount#developer.gserviceaccount.com&Expires=1430963040&Signature=gMf2h95bNmolizUGYrsQ%2F%2F%2FiHxW14I%2F0EOU3ZSFWtfCwNqSyok3iweQiuPxYXH4b26FeDSrmFOXB58%2B%2B%2BiAOJ%2B1gdLC9Y%2BkeUdbrjH0eGTW0NVsM1AWY2LsQ3dYf5Ho%2Bos1Fk26EsLJlD096Ku9aWqLW%2FpL%2FBSsUIfHijrFJPdI%3D'
I can get this to work with an existing file in the bucket and a GET request (for downloading), but it does not seem to work for uploading. curl throws the server's response with error messages like this at me:
<?xml version='1.0' encoding='UTF-8'?>
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided.
Check your Google secret key and signing method.</Message>
<StringToSign>PUT
application/x-www-form-urlencoded
1430963040
/mybucket/testfile</StringToSign>
</Error>
And this makes sense to me, as obviously I am not just making a bare PUT request, but one for a particular file of a specific size, whereas the signature computed by 'gsutil signurl' would not know about these details at the time it is computed.
Somehow I was under the impression (e.g., based on the last usage case described in gsutil signurl documentation and also in the post How to allow anonymous uploads to cloud storage) that it should be possible to generate a generic signed url for uploading purposes and then use it later. Am I just mistaken about this point or is there a way to fix the curl request?
Any thoughts about this are appreciated. However, I'd like this to work with "minimal tools", i.e., ideally shell and curl only, but no other programming languages.
EDIT:
Organising one's thoughts by formulating the exact problem is the first step towards the solution. I realise now that
curl -X PUT -T - [request-url] < testfile
does actually solve the immediate problem. However, this means multiple users would write to the same file if they use the same signed url. The documentation suggests you can omit the object name in the creation of the signed url, i.e., use
gsutil signurl -p notasecret -m PUT -d 1d myserviceaccount.p12 gs://mybucket/
This, supposedly, would allow anyone with the resulting signed url to put any object of any type into my bucket. Only I do not get this work, as I don't see how you can then tell GCS which object you are actually writing to.
This was driving me mad too. Turns out it was the binary-file part of the curl command. Try this instead:
curl -X PUT --upload-file me.jpeg $SIGNED_URL
If the resource does not specify a single object, you can do so on an individual basis by adding a URL param to the request with the name of the object. For example:
curl -X PUT -T - [request-url]?name=[object-name] < testfile
This surely works with storage/v1, although I have not tried myself with a signed URL yet.
I encountered the similar problem(403 forbidden).
It turned out that my json library, which I use it to marshal each response, would replace & by \u0026 for security concern. So the url may be correct in the program but invalid in client side
So I guess that there might be some string encoding bug inside the Signature query string of your url since the signature string is harder to detect error in comparison with my \u0026.
I am decrypting a gpg file using a batch file with the below code.
gpg.exe --output test.csv --batch --passphrase-fd 0 --decrypt WSB330TJ.CSTDJIDF.TXT.asc.14.04.22_00.59.gpg
Although it does Decrypt the file but I have to enter the passphrase manually.
How can I improve it so that it automatically pick the passphrase and decrypt the file without any manual intervention?
What should I add here?
You tell GnuPG to read the passphrase from stdin by using --passphrase-fd 0. There are different options to read the passphrase, from man gpg:
--passphrase-fd n
Read the passphrase from file descriptor n. Only the first line
will be read from file descriptor n. If you use 0 for n, the
passphrase will be read from STDIN. This can only be used if only
one passphrase is supplied.
--passphrase-file file
Read the passphrase from file file. Only the first line will be
read from file file. This can only be used if only one passphrase
is supplied. Obviously, a passphrase stored in a file is of ques-
tionable security if other users can read this file. Don't use this
option if you can avoid it.
--passphrase string
Use string as the passphrase. This can only be used if only one
passphrase is supplied. Obviously, this is of very questionable
security on a multi-user system. Don't use this option if you can
avoid it.
If you use GnuPG 2, remember to use --batch, otherwise the passphrase options will be ignored.
If you stored the passphrase in a file, use --passphrase-file password.txt, if you want to pass it as a string use --passphrase "f00b4r" (both times using appropriate parameter values, of course).
#Thierry noted in the comments that (especially when using Windows) make sure to end the file with a UNIX line feed (\n / LN) instead of a Windows line feed + carriage return (\n\r / LNRF).
For myself I had to do *gpg --batch --passphrase "MyPassword" --decrypt-files C:\PGPFiles\\\*.pgp*. Doing it with --passphrase not before --decrypt-files would always prompt me for the password. Also as Thierry said when I was using a password file I had to use Notepad++ to convert to unix style (Edit->EOL->Unix/OSX Format)
Try this:
gpg2 -se --passphrase yourpassword --batch --yes -r user#mydomain.com filename
I'm trying to use OpenGPG and when trying to decrypt something It opens up a prompt for the password. Now I'm trying to run this automatically therefore none to enter password. So My question is how do you pass in the password to this new prompt opened up by the exe I'm running form the batch file. Ive looked in gpg2.exe -help and there is no way to pass in the password as a parameter if anyone is familiar with OpenGPG or if there is a command I can run to pass the password into the new prompt, that would be great.
gpg2.exe -o output.txt -d series.txt.gpg
After many attempts at trying to get this working I, finally checked out the manual for gpg2.exe at http://linux.die.net/man/1/gpg2 and after adding the command line argument --batch the --passphrase is accepted by the application.
Not doing so results in the user being prompted.
Hope this helps anyone in the future attempting to do this hack.
echo password | gpg2.exe -o output.txt -d series.txt.gpg
GnuPG offers multiple ways to pass the passphrase non-interactively. Using the parameter --passphrase [password] is probably the most simple one, depending on your use case the others also could be of interest (for example if you do not want to store the passphrase within your application code).
From man gpg:
--passphrase-fd n
Read the passphrase from file descriptor n. Only the first line will be read
from file descriptor n. If you use 0 for n, the passphrase will be read from
STDIN. This can only be used if only one passphrase is supplied.
--passphrase-file file
Read the passphrase from file file. Only the first line will be read from
file file. This can only be used if only one passphrase is supplied. Obvi-
ously, a passphrase stored in a file is of questionable security if other
users can read this file. Don't use this option if you can avoid it.
--passphrase string
Use string as the passphrase. This can only be used if only one passphrase
is supplied. Obviously, this is of very questionable security on a multi-
user system. Don't use this option if you can avoid it.