i am a java programmer trying to use python
'appcfg.py upload_data' on appengine
Is it possible to set a csv delimeter on config.yml (as in -> 1#2#2#3 where # is the delimeter)
I see that there is a 'connector_options:' on config.yml
Thanks!
The appcfg.py upload_data operation is not present in the current appcfg.py documentation.
There used to be a related section in the Backing Up and Restoring guide, but it's now gone. From a snapshot of it captured in the wayback machine:
csv connector
encoding
Any Python standard encoding format, such as utf-8 (the
default) or windows-1252.
column_list
Use a sequence of names specified here for columns on import and
export. If not specified, use first row of data to calculate
external_name of each column, then read or write data starting with
second row.
skip_import_header_row
If true, header line will be ignored on import.
print_export_header_row
If true, header line will be printed on export.
import_options
Additional keyword arguments for the Python CSV module on import.
Use dialect: excel-tab for a TSV file.
export_options
Additional keyword arguments for the Python CSV module on export.
Note that the links in the quote are also snapshots, you might need to check the actual links valid today. The code might be incompatible.
IMHO it would be simpler to just re-write the CSVs to replace the delimiters... See Python parse csv file - replace commas with colons
Related
I understand the PUT command requires a file path to copy a file to a staging area. For what I'm doing,
I need to wrap the file path in quotation marks. According to the Snowflake documentation, it is required that the file path use forward slashes (/) instead of the windows-style backslashes () in the file path (for example, if you are trying to PUT the file C:\Documents\file.csv you would have to convert the file path to C:/Documents/file.csv). When I try to execute the PUT command with the forward slashes in the file path, quoted, I get an error from the SnowflakeODBC_sb64_dll. Example: ‘C:/Documents/file.csv’. The error I receive is: “An invalid parameter was passed to a function that considers invalid parameters fatal”. If I convert the file path back to backslashes it appears to work fine. Is this a known issue with the Snowflake driver?
I am doing this on Windows, using native Snowflake driver Version 2.20.1
For windows you can use the (//) with PUT Statement.
Check the below Doc:
https://docs.snowflake.com/en/user-guide/data-load-local-file-system-stage.html#staging-the-data-files3
I need to store data in a file in this format
word, audio, jpeg
How would I store that all in one file? Is it even possible do would I need to store links to other data files in place of the audio and jpeg. Would I need a custom file format?
1. Your own filetype
As mentioned by #Ken White you would need to be creating your own custom file format for this sort of thing, which would then mean creating your own parser type. This could be achieved in almost any language you wanted but since you are planning on using word format, then maybe C# would be best for you. However, this technique could be quite complicated and take a relatively large amount of time to thoroughly test your file compresser / decompressor, but may be best depending on your needs.
2. Command line utilities
Another way to go about this would be to use a bash script to combine all of the files into one file, and then decompress it at the other end. For example the steps could involve:
Combine files using windows copy / linux cat command on command line
Create a metdata file of your own that says how many files are in this custom file, and how much memory each one takes up (could be a short XML or JSON file for example...)
Use the linux split command or install a Windows command line file splitter program (here's just one example) to split the file back into whatever components have made it up.
This way you only have to create a really small file type, and let the OS utilities handle the combining of them for you.
Example on Windows:
Copy all of the files in your current directory into one output file called 'file.custom'
copy /b * file.custom
Generate custom file format describing metadata (i.e. get the file size on disk in C# example here). This is just maybe what I would do in JSON. SO formatting was being annoying so here's a link (Copy paste it into an editor or online JSON viewer).
Use a decompress windows / linux command line tool to decompress each files to the exact length (and export it back to the exact name) specified in the JSON (metadata) file. (More info on splitting files on this post).
3. ZIP files
You could always store all of the files in a compressed zip file, and then just use a zip compressor, expander as and when you like to retreive any number of file formats stored within.
I found a couple of examples of :
Combining multiple files into one ZIP file in only C# .net,
Unzipping ZIP files in C#
Zipping & Unzipping with only windows built-in utilities
Zipping & Unzipping in Linux command line
Good Zipping/Unzipping library in Java
Zipping/Unzipping in Python
I would like to know how to fix the syntax indentation for batch in ST3.
Currently, it only indents like the code below.
IF %num_args%==3 (
GOTO somewhere
GOTO somewhere_else
) ELSE (
ECHO some text
GOTO END
)
I checked the file Batch File.sublime-syntax but can't find a way to fix this.
Perhaps make some changes in this section?
parens:
- match: \(
scope: punctuation.section.group.begin.dosbatch
push:
- meta_scope: meta.group.dosbatch
- match: \)
scope: punctuation.section.group.end.dosbatch
pop: true
- match: ',|;'
scope: punctuation.separator.dosbatch
- include: expressions
if not where / what else can I do?
Indentation rules in Sublime aren't affected by the syntax definition outside of the syntax providing the scope that tells Sublime which rules to apply at any given point.
The rules for this are specified via tmPreferences style configuration files which contain the keys:
decreaseIndentPattern
increaseIndentPattern
disableIndentNextLinePattern
bracketIndentNextLinePattern
These keys represent regular expressions that indicate what the indent level should be based on the file content. A side effect of this system is that it can be hard to dial in exactly the indent that you want.
That said, the Batch File package that ships with Sublime doesn't contain any indentation rules by default. If that's the package that you're using for your batch file support, that would be why it's not indenting the way that you'd like.
If you're using that package, you would need to create a file with the relevant patterns in order for the file to be indented. In that case you could create an appropriate tmPreferences file and store it in your User package, as it's the scope specified in the file that determines which set of rules get used.
Alternatively if you're using a third party package, it may or may not have such a file in it, in which case you would either need to create one or override the existing one.
Creating one would still work the same as the above. If instead the third party package has some indent rules that you would like to tweak, the process is slightly different.
To override the file, use PackageResourceViewer to open the appropriate package file. If you use this package to open the file, any saves you make will automatically create an override that will make your file take precedence over the one in the package.
In the case that you need to create a file from scratch, the best course of action is to look at an existing package that is already doing this, to see how it's doing things.
The best course of action would be to find a package with a language as similar as possible in structure to the one you're working with to use as a base, although for a batch file I'm not sure if that's actually possible.
For illustration purposes, the Indentation.tmPreferences file that is included within the Shell Script package (which I erroneously made this answer about the first time) looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<plist version="1.0">
<dict>
<key>name</key>
<string>Indentation</string>
<key>scope</key>
<string>source.shell</string>
<key>settings</key>
<dict>
<key>decreaseIndentPattern</key>
<string>^\s*(\}|(elif|else|fi|esac|done)\b)</string>
<key>increaseIndentPattern</key>
<string>(?x)
^\s*(elif|else|case)\b
| ^.*(\{|\b(do)\b)\s*$
| ^.*\bthen\b\s*$
</string>
<key>indentNextLinePattern</key>
<string>^.*[^\\]\\$</string>
<key>disableIndentNextLinePattern</key>
<string>^\s*then\s*$</string>
<key>bracketIndentNextLinePattern</key>
<string>^\s*if\b.*(?!\bthen\b)$</string>
</dict>
</dict>
</plist>
For your purposes you would need to alter the regular expressions as needed in order to hint to Sublime what the indent should be, as well as modifying the scope that the rules apply to.
There is a forum post on the Sublime Forum that describes how the indent rules are applied by Sublime.
Additionally, if you're using the shipped Batch File syntax, it may be worth logging an issue on the Sublime Packages Issue Tracker. The packages that ship with Sublime are open source and the community helps to maintain them.
As such it's possible that (with some discussion) the default package for batch files could be updated to support this directly.
I want to analyze a few thousand EPS files and sort them according to their application in which they have been created (in my case Freehand or Illustrator).
I know that the the information is stored in the file header (%%Creator), but how can I analyze it thoughout all files?
I am looking for a tool to analyze the files and to give me a txt or csv file...
I already tried the following:
Windows Explorer --> shows lots of properties, but not the %%Creator
Adobe Bridge --> shows file metadata, but only of single files
Adobe Bridge, MetaData Script (See the Bridge section -- Extract Metadata) --> the script should export all metadata, but doesn't do it's job for all files...
pdfinfo.exe --> part of Xpdf, I assumed, if it can analyze PDF files, it may also analyze EPS, but does't...
Ghostscript --> I searched Google, but did not find a solution
ImageMagick --> I searched Google, but did not find a solution
ExifTool --> although it seems to be powerful, I could not manage to get the information intended
I am happy for any help!
Best regards
Hape
Write a simple script here a python 2.x script for example:
import glob
for fname in glob.glob('*.eps'):
with open(fname) as fp:
line = fp.readline()
if not '%!PS' in line:
continue
print "-- %s --" % fname
line = fp.readline().strip()
while line.startswith("%"):
if line.startswith("%%"):
print line
line = fp.readline().strip()
Edity: Updated script to understand windows style EPS previews. And relaxed the metadata requirements to match spec better
Is there a way to use Gamemaker with Gource?
Or is there something like a tool or library to export / convert .gmk files to .txt files each of them with one class?
While pre-Studio projects are stored as single files, you can use Gmk-Splitter to split the .gmk file into a number of XML files (not too unlike the format used by GameMaker: Studio) or recompose the file from a set.
While not ideal, this permits to use source control with older versions of GameMaker. See the author's "Gang-Garrison-2" repository for an example of that in action.
With .gmk it's impossible. But if you will use GMS then you can try do it, because GMS uses .xml (i.e. text) files. I checked it with git and mercurial and it works fine.