I'm trying to build a website that has one page for downloading some resources, and it just happened that my local flask version finds perfectly any file name (when using send_from_directory()), but once deployed at PythonAnywhere it just doesn't work for the filenames that have spanish accented characters such as á.
I guess it has something to do with unicode, but I couldn't find how to fix it (the logs at pythonanywhere don't seem to show anything, since flask simply delivers a "Not found" page to the user).
...and I'd really like to have those accents in the name of the files people download (they're anki decks, some of them intended for education purposes, and it just feels wrong to deliver bad ortography in the deck name).
My code looks like this:
#app.route('/anki/d/<file>')
def d_anki(file):
if file == "verbscat":
ankideck = "[Rusca] Temps Verbals Catalans.apkg"
elif file == "irregular":
ankideck = "[Rusca] Verbs Irregulars Anglès.apkg"
# ...
else:
return f"The file {file} wasn't found."
return send_from_directory("./static/anki/", ankideck, as_attachment=True, cache_timeout=0)
(then I link to this url in a button by <a href="/anki/d/irregular" ...>)
Oh I just realized I can choose a different name for the downloaded file by adding attachment_filename="Whatever I want to call it" to the parameters in send_from_directory.
So I guess we can do with this workaround (having the original files with simple non-accented names and adding the proper name afterwards).
if file == "irregular":
ankideck = "irregular.apkg"
name = "[Rusca] Verbs Irregulars Anglès.apkg"
# ...
return send_from_directory("./static/anki/", ankideck, as_attachment=True, attachment_filename=name cache_timeout=0)
Related
I'm trying to get JSON formatted logs on a Compute Engine VM instance to appear in the Log Viewer of the Google Developer Console. According to this documentation it should be possible to do so:
Applications using App Engine Managed VMs should write custom log
files to the VM's log directory at /var/log/app_engine/custom_logs.
These files are automatically collected and made available in the Logs
Viewer.
Custom log files must have the suffix .log or .log.json. If the suffix
is .log.json, the logs must be in JSON format with one JSON object per
line. If the suffix is .log, log entries are treated as plain text.
This doesn't seem to be working for me: logs ending with .log are visible in the Log Viewer, but displayed as plain text. Logs ending with .log.json aren't visible at all.
It also contradicts another recent article that states that file names must end in .log and its contents are treated as plain text.
As far as I can tell Google uses fluentd to index the log files into the Log Viewer. In the GitHub repository I cannot find any evidence that .log.json files are being indexed.
Does anyone know how to get this working? Or is the documentation out-of-date and has this feature been removed for some reason?
Here is one way to generate JSON logs for the Managed VMs logviewer:
The desired JSON format
The goal is to create a single line JSON object for each log line containing:
{
"message": "Error occurred!.",
"severity": "ERROR",
"timestamp": {
"seconds": 1437712034000,
"nanos": 905
}
}
(information sourced from Google: https://code.google.com/p/googleappengine/issues/detail?id=11678#c5)
Using python-json-logger
See: https://github.com/madzak/python-json-logger
def get_timestamp_dict(when=None):
"""Converts a datetime.datetime to integer milliseconds since the epoch.
Requires special handling to preserve microseconds.
Args:
when:
A datetime.datetime instance. If None, the timestamp for 'now'
will be used.
Returns:
Integer time since the epoch in milliseconds. If the supplied 'when' is
None, the return value will be None.
"""
if when is None:
when = datetime.datetime.utcnow()
ms_since_epoch = float(time.mktime(when.utctimetuple()) * 1000.0)
return {
'seconds': int(ms_since_epoch),
'nanos': int(when.microsecond / 1000.0),
}
def setup_json_logger(suffix=''):
try:
from pythonjsonlogger import jsonlogger
class GoogleJsonFormatter(jsonlogger.JsonFormatter):
FORMAT_STRING = "{message}"
def add_fields(self, log_record, record, message_dict):
super(GoogleJsonFormatter, self).add_fields(log_record,
record,
message_dict)
log_record['severity'] = record.levelname
log_record['timestamp'] = get_timestamp_dict()
log_record['message'] = self.FORMAT_STRING.format(
message=record.message,
filename=record.filename,
)
formatter = GoogleJsonFormatter()
log_path = '/var/log/app_engine/custom_logs/worker'+suffix+'.log.json'
make_sure_path_exists(log_path)
file_handler = logging.FileHandler(log_path)
file_handler.setFormatter(formatter)
logging.getLogger().addHandler(file_handler)
except OSError:
logging.warn("Custom log path not found for production logging")
except ImportError:
logging.warn("JSON Formatting not available")
To use, simply call setup_json_logger - you may also want to change the name of worker for your log.
I am currently working on a NodeJS app running on a managed VM and I am also trying to get my logs to be printed on the Google Developper Console. I created my log files in the ‘/var/log/app_engine’ directory as described in the documentation. Unfortunately this doesn’t seem to be working for me, even for the ‘.log’ files.
Could you describe where your logs are created ? Also, is your managed VM configured as "Managed by Google" or "Managed by User" ? Thanks!
Please be patient and check this problem.
I wrote some simple PHP code for uploading images.
Here is the code (snippets)
<?php
header('Content-Type: text/plain; charset=utf-8');
//set encoding for prepared statements
$dbh->exec("SET NAMES UTF8");
$dbh->query('SET NAMES UTF8');
//check if file is actually an image etc.
//send image to "upload" folder
move_uploaded_file($_FILES["file"]["tmp_name"],"upload/" . $_FILES["file"]["name"]);
//save to the database a string like "upload/myImage.jpg", so I can render it on the site later
$stu = $dbh->prepare("UPDATE multi SET m_place=:name WHERE m_id = :id");
$stu->bindParam(':name', $n, PDO::PARAM_STR);
$n= "upload/".$_FILES["file"]["name"];
$stu->execute();
If the name of the image is in English, everything is fine. If is in Greek, it saved ok in the database , but not in the folder. In the database I see χωρις τιτλο.jpg (which is right) and in the folder χωΟΞ―Ο‚ τίτλο.jpg which is wrong.
I've tried everything and cannot fix this. To get the right titles in the folder.
The encoding of the database (postgreSQL 9.1) is UTF8, the Collation and the Character Type are Greek_Greece.1235. The collaction in table's column which I save the image's title is pg_catalog."C".
I use DreamWeaver. The file that handles the uploads is a php file. Encoding is utf8, Unicode Normalization Form is C and the Include Unicode Signature (BOM) is unchecked.
The default language is Greek in Region and Language in the control panel. (Windows 7 Home Premium)
The encoding of the browser is utf8
I also use Apache 2.2.22. Is it Apache's fault? Or its the php file? Or the database?
I dont know what else to do...What am I missing? Please please please advice
This seems to be a StackOverflow issue and not related to ServerFault. However, you should take a look at this post:
For naming files in UTF-8:
Can a PHP file name (or a dir in its full path) have UTF-8 characters?
For writing files in UTF-8:
How to write file in UTF-8 format?
Turns out , I had not tried everything.
Thanks to this I found the solution.
It has to do with PHP and how "PHP filesystem functions can only handle characters that are in system codepage"
So I used the iconv function.
I changed the code like so
move_uploaded_file($_FILES["file"]["tmp_name"],"upload/" . iconv('UTF-8', 'Windows-1253',$_FILES["file"]["name"]));
For the application using japanese, the code page is cp932
This can be comfirm by using chcp command in commandline.
Sample Code
$fullPath = $uploadFolderPath."\\".iconv("utf-8", "cp932", $fileName);
move_uploaded_file($this->request->data["file_name"]['tmp_name'], $fullPath);
I just upgraded from cakephp 1.1 to 1.3. I have everything on the site updated and working great, except for creating and downloading zip files.
Here is the code in my accounts_controller.php:
function zip() {
$this->checkSession();
$this->checkUpgradedAccount();
$files = array();
$this->layout="zip";
/*
code where I locate the files to zip, combine them, etc
*/
$tmp_file = "/home/[userdirectory]/tmp/".md5(mktime()).".zip"; //directory name edited
$command = "/usr/bin/zip -j $tmp_file ".implode(" ",$zip_files);
exec($command);
foreach($zip_files as $zf) {
unlink($zf);
}
$file_path = $tmp_file;
$this->set("path",$file_path);
$this->render();
}
When I call this action, however, I get an error:
Error: The requested address '/accounts/zip' was not found on this
server.
It worked just like this in version 1.1. I'm assuming something has changed, but I'm not sure what, and was unable to find anything pertinent in the documentation.
The zip.ctp view file does exists, but it has nothing in it other than: <?php ?>
I suspect something is different with layouts. There is NO "zip.ctp" in the /layouts directory. However, I have changed that line to $this->layout('default'); and it renders a blank page with NO ERROR, but also with no download.
Please direct me on the proper way to download my zip file in cake 1.3. Thanks in advance.
You have two different problems here. That error you're getting is because you don't have a zip layout file. As for your problem with getting the zip file, you should be using the media view class - http://book.cakephp.org/1.3/en/The-Manual/Developing-with-CakePHP/Views.html#media-views
I use pysvn 1.7.5 to access my svn server.
If I want to copy a single file from a svn Server to my local disk there are no pysvn function implemented. But if I make a connection by https I can copy the single file, without doing a hole checkout on a directory.
def fetch_svn_file(self, file_url, local_path):
local_path = local_path.replace('\\', '/')
# Set up a HTTPS request with username/password authentication
try:
# create a password manager
password_mgr = HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
password_mgr.add_password(None, 'https://www.xyz.com', self.default_user, self.default_passwd)
opener = build_opener(HTTPBasicAuthHandler(password_mgr))
remote_file = opener.open(file_url)
content = remote_file.read()
try:
local_file = open(local_path,"w")
local_file.write(content)
local_file.close()
except IOError:
return -1
except URLError, e:
print 'URLError: "%s"' % e
return -2
return 0
The same way tortoise do it, if i drag a file from the Retro browser to my local disk, but tortoise can also copy single files in another revision. Anyone now how I can realise this in pysvn or in simple python code?? If these function is implemented by tortoise it has to be possible in pysvn too... because of the same developer team.
I already got the answer. :-)
there is a standart pysvn function called "pysvn.export". The name was so strange for a copy function....
I'd like to use GAE to allow a few users to upload files and later retrieve them. Files will be relatively small (a few hundred KB), so just storing stuff as a blob should work. I haven't been able to find any examples of something like this. There are a few image uploading examples out there but I'd like to be able to store word documents, pdfs, tiffs, etc. Any ideas/pointers/links? Thanks!
The same logic used for image uploads apply for other archive types. To make the file downloadable, you add a Content-Disposition header so the user is prompted to download it. A webapp simple example:
class DownloadHandler(webapp.RequestHandler):
def get(self, file_id):
# Files is a model.
f = Files.get_by_id(file_id)
if not f:
return self.error(404)
# Set headers to prompt for download.
headers = self.response.headers
headers['Content-Type'] = f.content_type or 'application/octet-stream'
headers['Content-Disposition'] = 'attachment; filename="%s"' % f.filename
# Add the file contents to the response.
self.response.out.write(f.contents)
(untested code, but you get the idea :)
It sounds like you want to use the Blobstore API.
You don't mention if you are using Python or Java so here are links to both.
I use blobstore API that admits any file upload/download up to 50 MB.