I'm trying to have a cgi-bin script given the path to a .docx file add a watermark to indicate it is a copy and present it. I have used Libreoffice to create the docx file. So my starting script just takes in the path and tries to present it to make sure the HTTP header is formed correctly. Good thing I have started out with baby steps as things do not work. When I run it with my Apache2 server, I get an error in the error.log file:
[Mon Jan 23 11:06:45.883760 2023] [cgid:error] [pid 1191] [client 127.0.0.1:42386] malformed header from script 'copydocx': Bad header: PK\x03\x04\x14, referer: http://robert-linux.local/builtsys/
and the file is not presented.
So what am I doing wrong?
Here are my scripts.
copydocx
#!/bin/bash
# -*-sh-*-
# copydocx - Present a .docx file with a copy watermark for a browser with a CGI script
# Arguments
# docx the path to a docx file
# Bring in URL forms arguments. Get from: https://github.com/adriansev/bin-scripts/blob/master /urlgetopt
eval `./urlgetopt -l "${QUERY_STRING}" 2>/dev/null`
# If an error present it and exit
present_error()
{
cat <<EOF
Content-type: text/html
<html><head>
<style>
red {
color: red
}
</style>
</head><body>
echo '<table width=100%><tr><td><form><input type=button value=Back onclick="history.back()"></form></td>
<td valign=top><font size=+2><red>ERROR: copydocx: $*</red></font></td>
</tr></table>
</body></html>
EOF
exit
}
# A terse set of error checks
test "${docx}" = "" && present_error No docx argument.
test ! -f "${docx}" && present_error File not found. "${docx}"
test "${docx##*.}" != "docx" && present_error File "${docx}" does not have a .docx suffix.
test ! -f copydocx.py && present_error File not found. copydocx.py
# Now get a copy of the docx file and present it
python3 copydocx.py
copydoc.py
import cgi
import sys
import os
# python3 script
# copydocx.py - Present a .docx file with a copy watermark for a browser
# Arguments
# docx the path to a docx file
def present_docx(filename):
# This if is so that 'python3 copydocx.py' runs
if filename != "":
statinfo = os.stat(filename)
size = statinfo.st_size
else:
size = 1234
filename = "notexisting.docx"
print("""Content-Type: application/octet-stream
Content-Disposition: attachment; filename="%s"
Content-Transfer-Encoding: binary
Content-Length: %d
""" % (filename, size))
os.system("cat %s" % filename)
form = cgi.FieldStorage()
docx = ""
if "docx" in form:
docx = form["docx"].value
# For now just present input to make sure we can present it at all.
present_docx(docx)
I hoped that this would work.
Related
I try to Upload a File into a Website with cURL on Windows CMD. The html-code looks like this:
<form enctype="multipart/form-data" action="/Filebrowser?Path=/S71500/" method="POST" onsubmit="return checkUploadFile()">
<td><input id="filebrowser_upload_filename" type="file" name="filename" size="30" maxlength="80" style="background-color: transparent;"></td>
<td><input type="submit" value="Datei laden"></td>
</form>
The command i am using is:
curl -F "filename=#/Users/Me/FILE.so" http://localhost:8080
The fail-message is:
Warning: setting file FILE.so failed!
curl: (26) read function returned funny value
What am I doing wrong?
for me, it was a permissions thing. Curl wasn't able to read my file because it wasn't readable. try:
sudo chmod a+r /path/to/your/file
also, make sure to verify that you are logged in as either root or the owner of the file
ls -l /path/to/your/file
will give you info about permissions, ownership, and group for example:
-rw-r--r-- 1 me users 792139 Jul 2 11:23 file.txt
file.txt is readable by root, "me" and members of the group "users" - writable only by root, and not executable by anyone. it contains 792139 bytes
I had the exact same error, while trying to upload with curl:
Warning: setting file testimage.png’ failed!
It comes out to be because my curl command line contained spurious invisible characters from a copy/paste from browser. I copied the command line into a text editor like Sublime Text and then back into terminal: suddenly everything worked.
Probably some UTF8 / ASCII issue.
Instead of -F, we have to use --form-string. Then it will work.
You should place a PHP code in the root directory of your server as below
upload.php
<?php
$target_path = "uploads/";
$target_path = $target_path . basename( $_FILES['filename']['name']);
if(move_uploaded_file($_FILES['filename']['tmp_name'], $target_path)) {
echo "The file ". basename( $_FILES['filename']['name']).
" has been uploaded";
} else{
echo "There was an error uploading the file, please try again!";
}
?>
keep in mind that you must use the argument a capital F (for Form). Lowercase f is for fail
I am trying to upload a csv file to GAE but having errors.
I am following this blog post and using same yml file. I can download data but upload command is giving following errors:
C:\Users\ADMIN>appcfg.py upload_data --config_file=C:\bulk\config.yml --file
name=C:\bulk\data.csv --url=http://encryptedsearch.appspot.com/remote_api --appl
ication=encryptedsearch --kind=Greeting
06:44 PM Uploading data records.
[INFO ] Logging to bulkloader-log-20151206.184424
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 250000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 20/second
[INFO ] Batch Size: 10
[INFO ] Opening database: bulkloader-progress-20151206.184424.sql3
2015-12-06 18:44:24,763 INFO client.py:669 access_token is expired. Now: 2015-12
-06 13:44:24.763000, token_expiry: 2015-12-05 20:24:05
2015-12-06 18:44:25,891 INFO client.py:669 access_token is expired. Now: 2015-12
-06 13:44:25.891000, token_expiry: 2015-12-05 20:24:05
2015-12-06 18:44:26,576 INFO client.py:669 access_token is expired. Now: 2015-12
-06 13:44:26.577000, token_expiry: 2015-12-05 20:24:05
2015-12-06 18:44:27,687 INFO client.py:669 access_token is expired. Now: 2015-12
-06 13:44:27.687000, token_expiry: 2015-12-05 20:24:05
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\appcfg.py", line 133, in
<module>
run_file(__file__, globals())
File "C:\Program Files (x86)\Google\google_appengine\appcfg.py", line 129, in
run_file
execfile(_PATHS.script_file(script_name), globals_)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 5469, in <module>
main(sys.argv)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 5460, in main
result = AppCfgApp(argv).Run()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 3010, in Run
self.action(self)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 5116, in __call__
return method()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 4919, in PerformUpload
run_fn(args)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\ap
pcfg.py", line 4803, in RunBulkloader
sys.exit(bulkloader.Run(arg_dict, self._GetOAuth2Parameters()))
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\bu
lkloader.py", line 4349, in Run
return _PerformBulkload(arg_dict, oauth2_parameters)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\bu
lkloader.py", line 4211, in _PerformBulkload
loader.finalize()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\bulk
load\bulkloader_config.py", line 383, in finalize
self.reserve_keys(self.keys_to_reserve)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\bu
lkloader.py", line 1211, in ReserveKeys
datastore._GetConnection()._reserve_keys(ConvertKeys(keys))
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastor
e\datastore_rpc.py", line 2172, in _reserve_keys
self._async_reserve_keys(None, keys).get_result()
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastor
e\datastore_rpc.py", line 2210, in _async_reserve_keys
service_name=_DATASTORE_V4))
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastor
e\datastore_rpc.py", line 1341, in _make_rpc_call
rpc = self._create_rpc(config, service_name)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\datastor
e\datastore_rpc.py", line 1236, in _create_rpc
rpc = apiproxy_stub_map.UserRPC(service_name, deadline, callback)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apip
roxy_stub_map.py", line 414, in __init__
self.__rpc = CreateRPC(service, stubmap)
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apip
roxy_stub_map.py", line 68, in CreateRPC
assert stub, 'No api proxy found for service "%s"' % service
AssertionError: No api proxy found for service "datastore_v4"
I am uploading same file back to datastore which I downloaded with command:
appcfg.py download_data --config_file=C:\bulk\config.yml --filename=C:\bulk\data.csv --kind=Greeting --url=http://encryptedsearch.appspot.com/remote_api --application=encryptedsearch
My config.yml is below:
# Autogenerated bulkloader.yaml file.
# You must edit this file before using it. TODO: Remove this line when done.
# At a minimum address the items marked with TODO:
# * Fill in connector and connector_options
# * Review the property_map.
# - Ensure the 'external_name' matches the name of your CSV column,
# XML tag, etc.
# - Check that __key__ property is what you want. Its value will become
# the key name on import, and on export the value will be the Key
# object. If you would like automatic key generation on import and
# omitting the key on export, you can remove the entire __key__
# property from the property map.
# If you have module(s) with your model classes, add them here. Also
# change the kind properties to model_class.
python_preamble:
- import: base64
- import: re
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.api.datastore
- import: google.appengine.api.users
transformers:
- kind: Greeting
connector: csv
connector_options:
# TODO: Add connector options here--these are specific to each connector.
property_map:
- property: __key__
external_name: key
import_transform: transform.key_id_or_name_as_string
- property: content
external_name: content
# Type: String Stats: 7 properties of this type in this kind.
- property: date
external_name: date
# Type: Date/Time Stats: 7 properties of this type in this kind.
import_transform: transform.import_date_time('%Y-%m-%dT%H:%M:%S')
export_transform: transform.export_date_time('%Y-%m-%dT%H:%M:%S')
- property: name
external_name: name
# Type: String Stats: 7 properties of this type in this kind.
CSV file contents:
content,date,name,key
Hi,2015-12-05T17:24:14,,ahFzfmVuY3J5cHRlZHNlYXJjaHIrCxIJR3Vlc3Rib29rIgdkZWZhdWx0DAsSCEdyZWV0aW5nGICAgICAgIAKDA
3rd,2015-12-05T17:30:04,,ahFzfmVuY3J5cHRlZHNlYXJjaHIrCxIJR3Vlc3Rib29rIgdkZWZhdWx0DAsSCEdyZWV0aW5nGICAgICA8ogKDA
2nd,2015-12-05T17:25:59,,ahFzfmVuY3J5cHRlZHNlYXJjaHIrCxIJR3Vlc3Rib29rIgdkZWZhdWx0DAsSCEdyZWV0aW5nGICAgICA5JEKDA
Datastore entries:
link to image
For all others that are fighting with the same issue. I found that there was no other solution for me than commenting that line in actual client.py.
Solution:
locate the client.py file ( ../google-cloud-sdk/platform/google_cloud/lib/oauth2client/oauth2client/client.py in my case)
navigate to access_token_expired() method
comment following code:
if now >= self.token_expiry:
logger.info('access_token is expired. Now: %s, token_expiry: %s',
now, self.token_expiry)
return True
In Spring-XD the file source detects new files in an input directory and streams their content through the pipeline.
Is there an analogous sink which creates separate result files in an output directory (e.g. with the original file names) and not a single file to which all results were appended, http://docs.spring.io/spring-xd/docs/current/reference/html/#file-sink: "The file sink uses the stream name as the default name for the file it creates, and places the file in the /tmp/xd/output/ directory."?
Scroll down to the options in that document you referenced.
Use --nameExpression=....
If you are using mode=contents; the original file name is available in the file_name header:
--nameExpression=headers[file_name]
mode=lines doesn't currently capture the file name (it will be fixed in the next release).
If you are using mode=ref, you need to set a header.
Minimal Working Example
In Spring-XD
stream create --name test --definition "file --mode=contents | b:file --binary=true --dirExpression='''/tmp/out''' --nameExpression=headers[file_name]" --deploy
than
echo "1111" > /tmp/xd/input/test/file1.txt
echo "2222" > /tmp/xd/input/test/file2.txt
results in
ll /tmp/out/
>
> -rw-rw-r-- 1 rmv rmv 5 Jul 7 10:19 file1.txt
> -rw-rw-r-- 1 rmv rmv 5 Jul 7 10:19 file2.txt
I am using SOLR 5 and I want to scan documents that have no extensions. Unfortunately changing the file to have extensions is not an option in my case.
the command I am using is simply:
$bin/post -c mycore ../foldertobescaned -type application/pdf
the command works fine for documents that do have extension but I am getting:
Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
If renaming the files is not an option, you can use the following script as a workaround until Solr improves its post method. It is a simple bash for loop that submits each file individually and works regardless of the file extension. Note that this script will be slower than using post on the whole folder, because each individual file transfer needs to be initialized.
Save the script below as postFolderToSolr.sh inside your Solr folder (so that Solrs bin/ folder is a subdirectory), make it executable with chmod +x postFolderToSolr.sh and then use it as follows: ./postFolderToSolr.sh mycore /home/user1/foldertobescaned/ application/pdf
Using no arguments or the wrong number of arguments prints a short usage message as help.
#!/bin/bash
set -o nounset
if [ "$#" -ne 3 ]
then
echo "Post contents of a folder to Solr."
echo
echo "Usage: postFolderToSolr.sh <colletionName> </path/to/folder> <MIME>"
echo
exit 1
fi
collection=$1
inputPath=${2%/} # remove suffix / if it exists
mime=$3
for element in $inputPath"/"*; do
bin/post -c $collection -type $mime $element
done
I have a .bat file that makes the following gsutil call to push a file to the Google Cloud:
python "C:\Program Files (x86)\gsutil\gsutil" -D -m cp -a public-read C:\Temp\MyMSI.msi gs://downloads-gs.mywebsite.com/binaries/myapplication/auto_installer/
Here is my output(with a few names changed for obscurity):
----------------------------------------------------------
Copy to google 11:02:33 AM 11:02:34 AM 00:00:00:479 Failed (Ignored)
Collapse Messages
Messages
Collapse
Executing external process: C:\Windows\system32\cmd.exe
Starting Directory: C:\BuildScripts
Parameters: /c "c:\buildscripts\push_goog.bat"
Collapse
Output from C:\Windows\system32\cmd.exe
The system cannot find the drive specified.
C:\BuildScripts>pushd c:\BuildScripts\
The system cannot find the drive specified.
Traceback (most recent call last):
File "C:\Program Files (x86)\gsutil\gsutil", line 67, in <module>
from gslib.util import UsingCrcmodExtension
File "C:\Program Files (x86)\gsutil\gslib\util.py", line 121, in <module>
os.path.join(CreateTrackerDirIfNeeded(), '.last_software_update_check'))
File "C:\Program Files (x86)\gsutil\gslib\util.py", line 108, in CreateTrackerDirIfNeeded
os.makedirs(tracker_dir)
File "C:\Python27\lib\os.py", line 150, in makedirs
makedirs(head, mode)
**C:\BuildScripts>python "C:\Program Files (x86)\gsutil\gsutil" -D -m cp -a public-read C:\Temp\MyMSI.msi gs://downloads-gs.mywebsite.com/binaries/myapplication/auto_installer/**
File "C:\Python27\lib\os.py", line 157, in makedirs
mkdir(name, mode)
WindowsError: [Error 3] The system cannot find the path specified: 'H:\\\\'
Program returned code : 1
It is mentioning something about H:\\\\ which we don't use and never mentioned in the .bat file or in the params to gsutil. I'm not sure why it works perfectly when run from the FinalBuilder application and then it gets this error when the FinalBuilder website runs the same FinalBuilder application file but is just triggered through the FinalBuilder Server.
Any help would be great.
The root cause for this is that your home directory is set to an invalid location.
As a workaround, you can change the tracker directory in your .boto configuration file. Here's the relevant section:
[GSUtil]
# 'resumable_tracker_dir' specifies the base location where resumable
# transfer tracker files are saved. By default they're in ~/.gsutil
#resumable_tracker_dir = <file path>
Uncomment the resumable_tracker_dir variable and set it to a location on disk that does exist.