How to save table output in an exportable manner in Sage - export

I am a sage novice trying to export a table output to some image format (so that it might be shared). I tried using the .save() function as so:
my_table1 = table(my inputs)
result = my_table1.transpose()
result.save('here')
My table outputs properly after I run the program (not featured), but for some reason I receive the following error when I try and save the table:
"Error! /home/sage/Documents/here.sobj is not UTF-8 encoded
Saving disabled.
See Console for more details."
Any help in exporting this table is greatly appreciated. Additionally, if you require any more information please do not hesitate to ask!
Background: I am working in Jupyter for Sage on a browser via localhost/8000; not sure if that matters. My OS is Windows 10 and I am using Sage version 7.6.

Would LaTeX output help? Using an example in the documentation for table?:
rows = [['a', 'b', 'c'], [100,2,3], [4,5,60]]
table(rows)._latex_()
'\\begin{tabular}{lll}\na & b & c \\\\\n$100$ & $2$ & $3$ \\\\\n$4$ & $5$ & $60$ \\\\\n\\end{tabular}'
You could try for the html but that would be harder, because it returns a HTMLFragment object, and you'd need the MathJax for it to look right.
str(table(rows)._html_())
'<div class="notruncate">\n<table class="table_form">\n<tbody>\n<tr class ="row-a">\n<td>a</td>\n<td>b</td>\n<td>c</td>\n</tr>\n<tr class ="row-b">\n<td><script type="math/tex">100</script></td>\n<td><script type="math/tex">2</script></td>\n<td><script type="math/tex">3</script></td>\n</tr>\n<tr class ="row-a">\n<td><script type="math/tex">4</script></td>\n<td><script type="math/tex">5</script></td>\n<td><script type="math/tex">60</script></td>\n</tr>\n</tbody>\n</table>\n</div>'
In any case, the image wouldn't be savable as-is. Unless you took a screenshot!

Related

Biblatex doesn't compile. Probably .bib file not recognised

I've spent many hours trying to get my bibliography working - unsuccessfully. I suspect that, somehow, my .bib file doesn't get recognised.
Help would be greatly appreciated.
MWE:
\documentclass[a4paper, 12pt]{article}
\usepackage{array}
\usepackage{lscape}
\usepackage[paper=portrait,pagesize]{typearea}
\usepackage[showframe=false]{geometry}
\usepackage{changepage}
\usepackage{tabularx}
\usepackage{graphicx}
\usepackage{adjustbox}
\usepackage[utf8]{inputenc}
\usepackage{babel,csquotes,xpatch}
\usepackage[backend=biber,style=authoryear, natbib]{biblatex}
\addbibresource{test.bib}
\usepackage{xurl}
\usepackage[colorlinks,allcolors=blue]{hyperref}
\begin{document}
This is a test... test test\\
\cite{glaeser_gyourko}\\
\cite{hsieh-moretti:2019}\\
\cite{glaeser_gyourko}\\
\printbibliography
\end{document}
test.bib file:
#article{hsieh-moretti:2019,
Author = {Hsieh, Chang-Tai and Moretti, Enrico},
Title = {Housing Constraints and Spatial Misallocation},
Journal = {American Economic Journal: Macroeconomics},
Volume = {11},
Number = {2},
Year = {2019},
Month = {4},
Pages = {1-39},
DOI = {10.1257/mac.20170388},
URL = {https://www.aeaweb.org/articles?id=10.1257/mac.20170388}
}
#article{glaeser_gyourko,
Author = {Glaeser, Edward and Gyourko, Joseph},
Title = {The Economic Implications of Housing Supply},
Journal = {Journal of Economic Perspectives},
Volume = {32},
Number = {1},
Year = {2018},
Month = {2},
Pages = {3-30},
DOI = {10.1257/jep.32.1.3},
URL = {https://www.aeaweb.org/articles?id=10.1257/jep.32.1.3}
}
In PDF it looks like this: enter image description here
I get the following information in the source viewer:
Process started
INFO - This is Biber 2.14 INFO - Logfile is 'test.blg' INFO - Reading
'test.bcf' INFO - Found 2 citekeys in bib section 0 INFO - Processing
section 0 INFO - Globbing data source 'test.bib' INFO - Globbed data
source 'test.bib' to test.bib INFO - Looking for bibtex format file
'test.bib' for section 0 INFO - LaTeX decoding ... INFO - Found
BibTeX data source 'test.bib'
Process exited with error(s)
I use texmaker 5.0.4 on MacOS and I post my configurations here:
enter image description here enter image description here
I really have very little idea on what goes on. Today, I started a work session, added a new source and it didn't work. I deleted the new source so that the bibliography would be the same as prior to me changing it, and it didn't work either. So, this let's me assume that, somehow, the program doesn't understand where the bibliography is. The .bib file and the document are in the same folder.
What I tried:
Triple checked code in bibliography using tools such as https://biblatex-linter.herokuapp.com/
Clear the cache of all documents.
change the natbib in the command \usepackage[backend=biber,style=authoryear, natbib]{biblatex} to biber -> doesn't seem to work.
Left out natbib and got same result. \usepackage[backend=biber,style=authoryear, natbib]{biblatex} => \usepackage[backend=biber,style=authoryear]{biblatex}
Add the command \usepackgage{natbitb} in addition to biblatex but this produces compatibility issues.
Add the codes \usepackage[utf8]{inputenc} &
\usepackage{babel,csquotes,xpatch} because they are recommendet by this biblatex cheat sheet: http://tug.ctan.org/info/biblatex-cheatsheet/biblatex-cheatsheet.pdf. Didn't change anything.
Thanks for your time!
I had a similar problem, what helped me was looking up the articles and rewriting them via the Google Scholar bibTex version.
The problem arose as I changed the name manually. This results in an error which is not recognized. And this threw me into researching exactly the same kind of .bib file not recognized error.
Your housing article should be formatted like this:
#article{hsieh2019housing,
title={Housing constraints and spatial misallocation},
author={Hsieh, Chang-Tai and Moretti, Enrico},
journal={American Economic Journal: Macroeconomics},
volume={11},
number={2},
pages={1--39},
year={2019}
}
I found another source of this problem Citavi generates invalid bibtex syntax. Often the year field is not correctly filled or special characters are not escaped properly. Maybe these are data errors which have their origin in the sources not in Citavi, but nonetheless often Citavi does not export valid bibtex format.

SSIS WMI event watcher WQL Direct Input query

I'm using SSIS in Visual Stdio 2010 to check a specific folder for any csv files that are dropped in from a 3rd party where the filename begins with 'Despatch'. This is my WQL code:
SELECT *
FROM __InstanceCreationEvent WITHIN 5
WHERE TargetInstance ISA 'CIM_DataFile'
AND TargetInstance.Drive = 'E:'
AND TargetInstance.Path = '\\Hybris URL REST Callbacks\\Despatched\\'
AND TargetInstance.FileName LIKE 'Despatch%'
AND TargetInstance.Extension = 'csv'
However, when I run it, I just get the error "Invalid parameter".
If I modify to the below it works - but how do I search for despatch* csv files only?
SELECT * FROM __InstanceCreationEvent WITHIN 10
WHERE TargetInstance ISA "CIM_DirectoryContainsFile"
and TargetInstance.GroupComponent= "Win32_Directory.Name=\"e:\\\\Hybris URL REST Callbacks\\\\Despatched\""
Where am I going wrong?
Regards,
Michael
If anyone is interested I couldn't get this to work so using free SSIS file watcher addin:
http://www.sqlis.com/sqlis/post/File-Watcher-Task.aspx
It looks like you found another option, but if you decide to try this again it looks like the only problem was additional tabs/space in the first example you posted. After modifying it, I can run the following without any errors whereas before I was able to reproduce the same invalid parameter error.
SELECT * FROM __InstanceCreationEvent WITHIN 5
WHERE TargetInstance ISA 'CIM_DataFile' AND TargetInstance.Drive = 'E:'
AND TargetInstance.Path = '\\Hybris URL REST Callbacks\\Despatched\\' AND TargetInstance.FileName LIKE 'Despatch%'
AND TargetInstance.Extension = 'csv'

Displaying Parse Data to ContainerList

I want to display data from Parse in a list from GamesScores class using Container in Codename One, this is what I've tried so far and it's not showing anything nor giving any errors:
Container container = findListCont();
container.setLayout(BoxLayout.y());
container.setScrollableY(true);
ParseQuery<ParseObject> query = ParseQuery.getQuery("GameScore");
List<ParseObject> results = (List<ParseObject>) query.find();
System.out.println("Size: " + results.size());
container.addComponent(results, f);
Please help me out, I'm a new in Codename One. If there tutorials on it, please share or anything to help me achieve the desired results.
I'm actually shocked this isn't failing. You are using the add constraint to place the object result as a constraint and you add the form object into the container...
You need to loop over the results and convert them to components to add into the layout. It also seems that you are using the old GUI builder which I would recommend against.
Generally something like this rough pseudo code should work assuming you are using a box Y layout:
for(ParseObject o : results) {
MultiButton mb = new MultiButton(o.getDisplayValue());
f.add(mb);
}
f.revalidate();

Simple / Smart, Pythonic database solution, can use Python types + syntax? (Key / Value Dict, Array, maybe Ordered Dict)

Looking for solutions that push the envelope and:
Avoid
Manually writing SQL queries(Python can be more OO not passing DSL strings)
Using non-Python datatypes for a supposedly required model definition
Using a new class of types rather than perfectly good native Python types
Boast
Using Python objects
Using Object Oriented and key based retrieval and creation
Quick protoyping
No SQL table to make
Model /Type inference or no model
Less lines and characters to type
Easily output to and from JSON, maybe XML or even Protocol Buffers.
I do web, desktop and mobile software development so the more portable the better.
python
>> from someAmazingDB import *
>> db.taskList = []
>> db['taskList'].append({title:'Beat old sql interfaces','done':False})
>> db.taskList.append({title:'Illustrate different syntax modes','done':True})
#at this point it should autosave
#we should be able to reload the console and access like:
python
>> from someAmazingDB import *
>> print 'Done tasks'
>> for task in db.taskList:
>> if task.done:
>> print task
'Illustrate different syntax modes'
Here is the challenge: The above code should work with very little modification or thinking required. Like a different import statement and maybe a little more but Django Models and SQLAlchemy DO NOT CUT IT.
I'm looking for more interesting library suggestions than just "Try Shelve" or "use pickle"
I'm not opposed to Python classes being used for models but they should be really straight forward, unlike the stuff you see with Django and similar.
I've was actually working on something like this earlier today. There is no readme or sufficient tests yet, but... http://github.com/mikeboers/LiteMap/blob/master/litemap.py
The LiteMap class behaves much like the builtin dict, but it persists into a SQLite database. You did not indicate what particular database you were interested in, but this could be almost trivially modified to any back end.
It also does not track changes to mutable classes (e.g. like appending to the list in your example), but the API is really simple.
Database access doesn't get better than SQLAlchemy.
Care to explain what about Django's models you don't find straightforward? Here's how I'd do what you have in Django:
from django.db import models
class Task(models.Model):
title = models.CharField(max_length=...)
is_done = models.BooleanField()
def __unicode__(self):
return self.title
----
from mysite.tasks.models import Task
t = Task(title='Beat old sql interfaces', is_done=True)
t.save()
----
from mysite.tasks.models import Task
print 'Done tasks'
for task in Task.objects.filter(is_done=True):
print task
Seems pretty straightforward to me! Also, results in a slightly cleaner table/object naming scheme IMO. The trickier part is using Django's DB module separate from the rest of Django, if that's what you're after, but it can be done.
Using web2py:
>>> from gluon.sql import DAL, Field
>>> db=DAL('sqlite://stoarge.db')
>>> db.define_table('taskList',Field('title'),Field('done','boolean')) # creates the table
>>> db['taskList'].insert(title='Beat old sql interfaces',done=False)
>>> db.taskList.insert(title='Beat old sql interfaces',done=False)
>> for task in db(db.taskList.done==True).select():
>> print task.title
Supports 10 different database back-ends + google app engine.
Question looks strikingly similar to http://api.mongodb.org/python/1.9%2B/tutorial.html
So answer is pymongo, what else ;)
from pymongo import Connection
connection = Connection()
connection = Connection('localhost', 27017)
tasklist = db['test-tasklist']
tasklist.append({title:'Beat old sql interfaces','done':False})
db.tasklist.append({title:'Illustrate different syntax modes','done':True})
for task in db.tasklist.find({done:True}):
print task.title
I haven't tested the code but wont be very different than this
BTW Redish is also interesting and fun.

Plotting a word-cloud by date for a twitter search result? (using R)

I wish to search twitter for a word (let's say #google), and then be able to generate a tag cloud of the words used in twitts, but according to dates (for example, having a moving window of an hour, that moves by 10 minutes each time, and shows me how different words gotten more often used throughout the day).
I would appreciate any help on how to go about doing this regarding: resources for the information, code for the programming (R is the only language I am apt in using) and ideas on visualization. Questions:
How do I get the information?
In R, I found that the twitteR package has the searchTwitter command. But I don't know how big an "n" I can get from it. Also, It doesn't return the dates in which the twitt originated from.
I see here that I could get until 1500 twitts, but this requires me to do the parsing manually (which leads me to step 2). Also, for my purposes, I would need tens of thousands of twitts. Is it even possible to get them in retrospect?? (for example, asking older posts each time through the API URL ?) If not, there is the more general question of how to create a personal storage of twitts on your home computer? (a question which might be better left to another SO thread - although any insights from people here would be very interesting for me to read)
How to parse the information (in R)? I know that R has functions that could help from the rcurl and twitteR packages. But I don't know which, or how to use them. Any suggestions would be of help.
How to analyse? how to remove all the "not interesting" words? I found that the "tm" package in R has this example:
reuters <- tm_map(reuters, removeWords, stopwords("english"))
Would this do the trick? I should I do something else/more ?
Also, I imagine I would like to do that after cutting my dataset according to time (which will require some posix-like functions (which I am not exactly sure which would be needed here, or how to use it).
And lastly, there is the question of visualization. How do I create a tag cloud of the words? I found a solution for this here, any other suggestion/recommendations?
I believe I am asking a huge question here but I tried to break it to as many straightforward questions as possible. Any help will be welcomed!
Best,
Tal
Word/Tag cloud in R using "snippets" package
www.wordle.net
Using openNLP package you could pos-tag the tweets(pos=Part of speech) and then extract just the nouns, verbs or adjectives for visualization in a wordcloud.
Maybe you can query twitter and use the current system-time as a time-stamp, write to a local database and query again in increments of x secs/mins, etc.
There is historical data available at http://www.readwriteweb.com/archives/twitter_data_dump_infochimp_puts_1b_connections_up.php and http://www.wired.com/epicenter/2010/04/loc-google-twitter/
As for the plotting piece: I did a word cloud here: http://trends.techcrunch.com/2009/09/25/describe-yourself-in-3-or-4-words/ using the snippets package, my code is in there. I manually pulled out certain words. Check it out and let me know if you have more specific questions.
I note that this is an old question, and there are several solutions available via web search, but here's one answer (via http://blog.ouseful.info/2012/02/15/generating-twitter-wordclouds-in-r-prompted-by-an-open-learning-blogpost/):
require(twitteR)
searchTerm='#dev8d'
#Grab the tweets
rdmTweets <- searchTwitter(searchTerm, n=500)
#Use a handy helper function to put the tweets into a dataframe
tw.df=twListToDF(rdmTweets)
##Note: there are some handy, basic Twitter related functions here:
##https://github.com/matteoredaelli/twitter-r-utils
#For example:
RemoveAtPeople <- function(tweet) {
gsub("#\\w+", "", tweet)
}
#Then for example, remove #d names
tweets <- as.vector(sapply(tw.df$text, RemoveAtPeople))
##Wordcloud - scripts available from various sources; I used:
#http://rdatamining.wordpress.com/2011/11/09/using-text-mining-to-find-out-what-rdatamining-tweets-are-about/
#Call with eg: tw.c=generateCorpus(tw.df$text)
generateCorpus= function(df,my.stopwords=c()){
#Install the textmining library
require(tm)
#The following is cribbed and seems to do what it says on the can
tw.corpus= Corpus(VectorSource(df))
# remove punctuation
tw.corpus = tm_map(tw.corpus, removePunctuation)
#normalise case
tw.corpus = tm_map(tw.corpus, tolower)
# remove stopwords
tw.corpus = tm_map(tw.corpus, removeWords, stopwords('english'))
tw.corpus = tm_map(tw.corpus, removeWords, my.stopwords)
tw.corpus
}
wordcloud.generate=function(corpus,min.freq=3){
require(wordcloud)
doc.m = TermDocumentMatrix(corpus, control = list(minWordLength = 1))
dm = as.matrix(doc.m)
# calculate the frequency of words
v = sort(rowSums(dm), decreasing=TRUE)
d = data.frame(word=names(v), freq=v)
#Generate the wordcloud
wc=wordcloud(d$word, d$freq, min.freq=min.freq)
wc
}
print(wordcloud.generate(generateCorpus(tweets,'dev8d'),7))
##Generate an image file of the wordcloud
png('test.png', width=600,height=600)
wordcloud.generate(generateCorpus(tweets,'dev8d'),7)
dev.off()
#We could make it even easier if we hide away the tweet grabbing code. eg:
tweets.grabber=function(searchTerm,num=500){
require(twitteR)
rdmTweets = searchTwitter(searchTerm, n=num)
tw.df=twListToDF(rdmTweets)
as.vector(sapply(tw.df$text, RemoveAtPeople))
}
#Then we could do something like:
tweets=tweets.grabber('ukgc12')
wordcloud.generate(generateCorpus(tweets),3)
I would like to answer your question in making big word cloud.
What I did is
Use s0.tweet <- searchTwitter(KEYWORD,n=1500) for 7 days or more, such as THIS.
Combine them by this command :
rdmTweets = c(s0.tweet,s1.tweet,s2.tweet,s3.tweet,s4.tweet,s5.tweet,s6.tweet,s7.tweet)
The result:
This Square Cloud consists of about 9000 tweets.
Source: People voice about Lynas Malaysia through Twitter Analysis with R CloudStat
Hope it help!

Resources