I use AngularJS to create a page where a user can correct a text (for example grammar, typo ... ).
I cannot use a Textarea because I want to keep trace about changes and let user rollback a correction on each word.
The following code work but it take a few seconds to render with page freeze, specialy on IE like 30 seconds), the text to correct can be very long like in the example.
I use a ng-repeat to display the text (which is an array of word). For each word I text in html if it is punctuation or a <br> or an editable word.
Is there a way to optimize this or to create in a JS way (like a compile html or anything faster)?
PLUNKER
HTML
<div ng-controller="Ctrl1">
Correct the text
<span ng-repeat="word in words track by $index">
<br ng-if="word.br"/>
<span ng-show="(!word.br)&& !word.edited">
<span ng-if="word.editable" class="correct-span" ng-click="word.edited = true">{{word.u}}</span>
<span ng-if="!word.editable">{{word.u}}</span>
</span>
<span class="my-danger" ng-show="(!word.br)&& word.edited">
<input type="text" ng-model="word.u">
<button ng-click="word.edited = false;word.u = word.o">X</button>
</span>
</span>
</div>
My controller :
var myApp = angular.module('myApp', []);
myApp.controller('Ctrl1', ['$scope', function($scope) {
function tools_isString(myVar){
return (typeof myVar == 'string' || myVar instanceof String);
}
/***
* test if object if defined
* #param object
* #returns {boolean}
*/
function tools_defined(object){
return (( typeof object !== undefined) && ( typeof object !== 'undefined') && ( object !== null ) && (object !== "")) ;
}
/**
* test if a word is in array
* #param mot : string
* #param tableau : array list
* #returns {boolean}
*/
function tools_inArray(word, array) {
if(tools_defined(array)&&tools_defined(word)) {
var length = array.length;
if (tools_isString(word)) {
word = word.toLowerCase();
}
for (var i = 0; i < length; i++) {
if (tools_isString(array[i])) {
array[i] = (array[i]).toLowerCase();
}
if (array[i] == word) return true;
}
}
return false;
}
function escapeRegExp(string) {
return string.replace(/([.*+?^=!:${}()|\[\]\/\\])/g, "\\$1");
}
function tools_replaceAll(str, find, replace) {
if(str == null ){
return null
}
return str.replace(new RegExp(escapeRegExp(find), 'g'), replace);
}
var prepareTextToCorrect = function(inputstring){
//encode new lines
inputstring = tools_replaceAll(inputstring,"<br/>","*br*");
inputstring = tools_replaceAll(inputstring,"<br>","*br*");
// unescape
inputstring = inputstring.replace(/&(lt|gt);/g, function (strMatch, p1){
return (p1 == "lt")? "<" : ">";
});
// remove all the hmtl tags
var rex = /(<([^>]+)>)|(<([^>]+)>)/ig;
inputstring = inputstring.replace(rex , "");
// re encode new lines
inputstring = tools_replaceAll(inputstring,"*br*"," <br/> ");
// separating punctuation from words
var ponctuations = [",","?",",",";",".",":","!","-","_","(",")","«","»","—"];
for(var p in ponctuations){
inputstring = tools_replaceAll(inputstring,ponctuations[p]," "+ponctuations[p]);
}
inputstring = tools_replaceAll(inputstring," "," ");
inputstring = tools_replaceAll(inputstring," "," ");
var elements = inputstring.split(" ");
var res = [];
/**
* "o" : original word
* "u" : word edited by user
* "edited" : if user edited this word
* "editable" : if the word can be edited ( ponctuation and <br> cannot )
*/
for(var i in elements){
if(elements[i].length>0) {
if(elements[i] == "<br/>") {
res.push({
"o": null, "u": null, "edited": false, "br":true
});
} else if (tools_inArray(elements[i], ponctuations)) {
res.push({
"o": elements[i], "u": elements[i], "edited": false,"editable": false , "br":false
});
}else{
res.push({
"o": elements[i], "u": elements[i], "edited": false,"editable": true , "br":false
});
}
}
}
return res ;
};
var text = "Stack Overflow is a question and answer site for professional and enthusiast programmers. It's built and run by you as part of the Stack Exchange network of Q&A sites. With your help, we're working together to build a library of detailed answers to every question about programming.<br/><br/>We're a little bit different from other sites. Here's how:<br/>Ask questions, get answers, no distractions<br/><br/>This site is all about getting answers. It's not a discussion forum. There's no chit-chat.<br/><br/>Just questions...<br/>...and answers.<br/>Good answers are voted up and rise to the top.<br/><br/>The best answers show up first so that they are always easy to find.<br/>accept<br/><br/>The person who asked can mark one answer as accepted.<br/><br/>Accepting doesn't mean it's the best answer, it just means that it worked for the person who asked.<br/>Do Swift-based applications work on OS X 10.9/iOS 7 and lower?<br/>up vote 14 down vote favorite<br/><br/>Will Swift-based applications work on OS X 10.9 (Mavericks)/iOS 7 and lower?<br/><br/>For example, I have a machine running OS X 10.8 (Mountain Lion), and I am wondering if an application I write in Swift will run on it.<br/>ios osx swift<br/>asked Jun 2 '14 at 19:25<br/>MeIr<br/>3,27752557<br/>2 Answers<br/>up vote 4 down vote accept<br/><br/>Swift code can be deployed to OS X 10.9 and iOS 7.0. It will usually crash at launch on older OS versions.<br/>answered Jun 3 '14 at 8:25<br/>Greg Parker<br/>6,21011118<br/>up vote 3 down vote<br/><br/>Apple has announced that Swift apps will be backward compatible with iOS 7 and OS X Mavericks. The WWDC app is written in Swift.<br/>answered Jun 3 '14 at 0:03<br/>Ben Gottlieb<br/>73.3k19161166<br/>Get answers to practical, detailed questions<br/><br/>Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.<br/><br/>Ask about...<br/><br/>Specific programming problems<br/>Software algorithms<br/>Coding techniques<br/>Software development tools<br/><br/>Not all questions work well in our format. Avoid questions that are primarily opinion-based, or that are likely to generate discussion rather than answers.<br/><br/>Questions that need improvement may be closed until someone fixes them.<br/><br/>Don't ask about...<br/><br/>Questions you haven't tried to find an answer for (show your work!)<br/>Product or service recommendations or comparisons<br/>Requests for lists of things, polls, opinions, discussions, etc.<br/>Anything not directly related to writing computer programs<br/><br/>Tags make it easy to find interesting questions<br/><br/>Stack Overflow is a question and answer site for professional and enthusiast programmers. It's built and run by you as part of the Stack Exchange network of Q&A sites. With your help, we're working together to build a library of detailed answers to every question about programming.<br/><br/>We're a little bit different from other sites. Here's how:<br/>Ask questions, get answers, no distractions<br/><br/>This site is all about getting answers. It's not a discussion forum. There's no chit-chat.<br/><br/>Just questions...<br/>...and answers.<br/>Good answers are voted up and rise to the top.<br/><br/>The best answers show up first so that they are always easy to find.<br/>accept<br/><br/>The person who asked can mark one answer as accepted.<br/><br/>Accepting doesn't mean it's the best answer, it just means that it worked for the person who asked.<br/>Do Swift-based applications work on OS X 10.9/iOS 7 and lower?<br/>up vote 14 down vote favorite<br/><br/>Will Swift-based applications work on OS X 10.9 (Mavericks)/iOS 7 and lower?<br/><br/>For example, I have a machine running OS X 10.8 (Mountain Lion), and I am wondering if an application I write in Swift will run on it.<br/>ios osx swift<br/>asked Jun 2 '14 at 19:25<br/>MeIr<br/>3,27752557<br/>2 Answers<br/>up vote 4 down vote accept<br/><br/>Swift code can be deployed to OS X 10.9 and iOS 7.0. It will usually crash at launch on older OS versions.<br/>answered Jun 3 '14 at 8:25<br/>Greg Parker<br/>6,21011118<br/>up vote 3 down vote<br/><br/>Apple has announced that Swift apps will be backward compatible with iOS 7 and OS X Mavericks. The WWDC app is written in Swift.<br/>answered Jun 3 '14 at 0:03<br/>Ben Gottlieb<br/>73.3k19161166<br/>Get answers to practical, detailed questions<br/><br/>Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.<br/><br/>Ask about...<br/><br/>Specific programming problems<br/>Software algorithms<br/>Coding techniques<br/>Software development tools<br/><br/>Not all questions work well in our format. Avoid questions that are primarily opinion-based, or that are likely to generate discussion rather than answers.<br/><br/>Questions that need improvement may be closed until someone fixes them.<br/><br/>Don't ask about...<br/><br/>Questions you haven't tried to find an answer for (show your work!)<br/>Product or service recommendations or comparisons<br/>Requests for lists of things, polls, opinions, discussions, etc.<br/>Anything not directly related to writing computer programs<br/><br/>Tags make it easy to find interesting questions<br/><br/>Stack Overflow is a question and answer site for professional and enthusiast programmers. It's built and run by you as part of the Stack Exchange network of Q&A sites. With your help, we're working together to build a library of detailed answers to every question about programming.<br/><br/>We're a little bit different from other sites. Here's how:<br/>Ask questions, get answers, no distractions<br/><br/>This site is all about getting answers. It's not a discussion forum. There's no chit-chat.<br/><br/>Just questions...<br/>...and answers.<br/>Good answers are voted up and rise to the top.<br/><br/>The best answers show up first so that they are always easy to find.<br/>accept<br/><br/>The person who asked can mark one answer as accepted.<br/><br/>Accepting doesn't mean it's the best answer, it just means that it worked for the person who asked.<br/>Do Swift-based applications work on OS X 10.9/iOS 7 and lower?<br/>up vote 14 down vote favorite<br/><br/>Will Swift-based applications work on OS X 10.9 (Mavericks)/iOS 7 and lower?<br/><br/>For example, I have a machine running OS X 10.8 (Mountain Lion), and I am wondering if an application I write in Swift will run on it.<br/>ios osx swift<br/>asked Jun 2 '14 at 19:25<br/>MeIr<br/>3,27752557<br/>2 Answers<br/>up vote 4 down vote accept<br/><br/>Swift code can be deployed to OS X 10.9 and iOS 7.0. It will usually crash at launch on older OS versions.<br/>answered Jun 3 '14 at 8:25<br/>Greg Parker<br/>6,21011118<br/>up vote 3 down vote<br/><br/>Apple has announced that Swift apps will be backward compatible with iOS 7 and OS X Mavericks. The WWDC app is written in Swift.<br/>answered Jun 3 '14 at 0:03<br/>Ben Gottlieb<br/>73.3k19161166<br/>Get answers to practical, detailed questions<br/><br/>Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.<br/><br/>Ask about...<br/><br/>Specific programming problems<br/>Software algorithms<br/>Coding techniques<br/>Software development tools<br/><br/>Not all questions work well in our format. Avoid questions that are primarily opinion-based, or that are likely to generate discussion rather than answers.<br/><br/>Questions that need improvement may be closed until someone fixes them.<br/><br/>Don't ask about...<br/><br/>Questions you haven't tried to find an answer for (show your work!)<br/>Product or service recommendations or comparisons<br/>Requests for lists of things, polls, opinions, discussions, etc.<br/>Anything not directly related to writing computer programs<br/><br/>Tags make it easy to find interesting questions<br/><br/>Stack Overflow is a question and answer site for professional and enthusiast programmers. It's built and run by you as part of the Stack Exchange network of Q&A sites. With your help, we're working together to build a library of detailed answers to every question about programming.<br/><br/>We're a little bit different from other sites. Here's how:<br/>Ask questions, get answers, no distractions<br/><br/>This site is all about getting answers. It's not a discussion forum. There's no chit-chat.<br/><br/>Just questions...<br/>...and answers.<br/>Good answers are voted up and rise to the top.<br/><br/>The best answers show up first so that they are always easy to find.<br/>accept<br/><br/>The person who asked can mark one answer as accepted.<br/><br/>Accepting doesn't mean it's the best answer, it just means that it worked for the person who asked.<br/>Do Swift-based applications work on OS X 10.9/iOS 7 and lower?<br/>up vote 14 down vote favorite<br/><br/>Will Swift-based applications work on OS X 10.9 (Mavericks)/iOS 7 and lower?<br/><br/>For example, I have a machine running OS X 10.8 (Mountain Lion), and I am wondering if an application I write in Swift will run on it.<br/>ios osx swift<br/>asked Jun 2 '14 at 19:25<br/>MeIr<br/>3,27752557<br/>2 Answers<br/>up vote 4 down vote accept<br/><br/>Swift code can be deployed to OS X 10.9 and iOS 7.0. It will usually crash at launch on older OS versions.<br/>answered Jun 3 '14 at 8:25<br/>Greg Parker<br/>6,21011118<br/>up vote 3 down vote<br/><br/>Apple has announced that Swift apps will be backward compatible with iOS 7 and OS X Mavericks. The WWDC app is written in Swift.<br/>answered Jun 3 '14 at 0:03<br/>Ben Gottlieb<br/>73.3k19161166<br/>Get answers to practical, detailed questions<br/><br/>Focus on questions about an actual problem you have faced. Include details about what you have tried and exactly what you are trying to do.<br/><br/>Ask about...<br/><br/>Specific programming problems<br/>Software algorithms<br/>Coding techniques<br/>Software development tools<br/><br/>Not all questions work well in our format. Avoid questions that are primarily opinion-based, or that are likely to generate discussion rather than answers.<br/><br/>Questions that need improvement may be closed until someone fixes them.<br/><br/>Don't ask about...<br/><br/>Questions you haven't tried to find an answer for (show your work!)<br/>Product or service recommendations or comparisons<br/>Requests for lists of things, polls, opinions, discussions, etc.<br/>Anything not directly related to writing computer programs<br/><br/>Tags make it easy to find interesting questions<br/><br/>" ;
$scope.words = prepareTextToCorrect(text) ;
}]);
Try using ng-if instead of ng-show in your <span> tags. In such a way the browser does not need to render all the DOM nodes that you use when the word is edited. With ng-show the nodes are rendered and then hidden from the DOM using CSS. This means that the browser has to render nodes that you potentially do not use, it is highly probable that you only need to change few words and not the entire document! Try to see if this can improve the rendering time.
Whatever the front-end framework, tracking each word of a text is going to put the browser on its knees, no matter if you have V8, Turbo, 4x4 or whatever.
Just picture the number of nodes. Take a deep deep look at your DOM element, just one of your ng-if spans in your case, and imagine each one of its endless list of attributes being tracked. But you probably already know.
With angular 1.x, you can check if a textarea is $dirty on mouseup, and/or blur, and/on mousemove with a simple directive.
Just wire a service that stores any change made to the whole textarea whenever one of the above events triggers.
In short, it's going to be less expensive to store the whole textarea on every event (after all, its content is just a string, nothing too hard for a browser to handle, even if the string is big — but I'm sure you care about your users and your textarea won't end up to be massive).
To store every change made to the textarea, you can use localStorage and/or a remote DB, using possibly angular locker for localStorage abstractions, and Firebase (AngularFire), which is going to handle automagically any changes made to the textarea given you previously wire the textarea content to a Firebase object.
But your back-end could be of course any data API. I would to suggest to store a limited amount of "Ctrl/Cmd+Z" into localStorage, and for the external DB, well, it's up to you to store infinite versions. That's where Firebase would come in handy, because by forcing you to make adhere to a JSON, you can store by month, week, day, therefore accelerating your retrieval queries for when the end user wants to go back in history.
Related
I downloaded the pdf from the below link
http://ap.meeseva.gov.in/DeptPortal/Application%20Forms%20New/Revenue-pdf/Income%20General%20Application%20Form.pdf
What I need is I would like to fill out the blanks with the given text, I tried with the following code
using (FileStream outFile = new FileStream(#"E:\\residence(VRO)1.pdf", FileMode.Create))
{
PdfReader pdfReader = new PdfReader(#"E:\\residence(VRO).pdf");
PdfStamper pdfStamper = new PdfStamper(pdfReader, outFile);
pdfStamper.FormFlattening = true;
AcroFields af = pdfReader.AcroFields;
string[] fields = pdfStamper.AcroFields.Fields.Select(x => x.Key).ToArray();
for (int key = 0; key <= fields.Count() - 1; key++)
{
}
}
But I am not getting the fields so can some one help me
The so-called form you refer to, isn't a form. That is: it's not an interactive form. I took that PDF and I added interactive fields. Please download adapted.pdf and examine it to discover the differences.
Now when you run your code on it, you'll see the fields, and you will be able to fill out the form with Western text. You are currently using iTextSharp 5 or earlier. That version of iText doesn't support Hindi. Hindi wasn't introduced up until iText 7. Hence if you want to fill out the form using an Indic writing system (Devanagari, Tamil,...), you need iText 7 for C# and the pdfCalligraph add-on. Note that the pdfCalligraph add-on is kept closed source. You need a commercial license to use it.
We kept that part closed source because:
Too many companies are using iText without respecting the AGPL license and without purchasing a license,
As far as I know, no other free software supports Indic writing systems. We'd be giving away too much value if we released pdfCalligraph as open source software.
Summarized:
your form is not a form. Make it a form.
use software that can fill out such a form using an Indic writing system (e.g. iText 7 + the pdfCalligraph add-on).
(I am not sure if this question belongs to the meta website or not, but here we go)
I want to add stackoverflow to the bibliography of a research paper I am writing, and wonder if there is any bibTeX code to do so. I already did that for gnuplot
I searched online, but in most cases the citation goes to a specific thread. In this case, I want to acknowledge SO as a whole, and add a proper citation, probably to the website itself. Hopefully somebody already did this in the past?
As an example, below are the codes I use for R and gnuplot:
#Manual{rproject,
title = {R: A Language and Environment for Statistical Computing},
author = {{R Core Team}},
organization = {R Foundation for Statistical Computing},
address = {Vienna, Austria},
year = {2015},
url = {https://www.R-project.org/},
}
#MISC{gnuplot,
author = {Thomas Williams and Colin Kelley and {many others}},
title = {Gnuplot 5.0: an interactive plotting program},
month = {June},
year = {2015},
howpublished = {\href{http://www.gnuplot.info/}{http://www.gnuplot.info/}}
}
I know that both are software, not website resources, but maybe something along those lines would work. Any feedback is appreciated!
Thanks!
I did not realize this question never got answered. The solution I found was to acknowledge the SO website in the LaTeX code with the following:
This research has made use of the online Q\&A platform {\texttt{stackoverflow}}
(\href{http://stackoverflow.com/}{http://stackoverflow.com/}).
Hope it helps somebody in the future!
Actually, for my paper I am using the following citation:
#misc{stackoverflow,
url={https://stackoverflow.com/},
title={Stack overflow},
year={2008}
}
I hope it helps!
When try to pick an item from my ipod library some items play and some dont.
Looking at my log some items have a NULL for assetURL.
Why would that be?
all item are DRM protected which return null assetURL. You can't access such item
I want to add something to this. There are three reasons that I know of why assetURL will be null
The item is in the cloud (in which case isCloudItem will return true).
As stated in another answer, if the track is DRM protected.
This is the kicker, tho: in some cases, and item downloaded on the device, which is not DRM, will play in Music (the built-in App), but will still have assetURL returning NULL.
This means that any non-Apple App that uses the MediaPlayer framework may encounter some media items that play in Music, but which cannot be played in the App. Your end user can usually "fix" this problem by deleting the offending track in Music, and downloading it again.
I find that if I download a complete album and see this issue, then downloading the album again (after deleting it) will lead to some different tracks having the problem, so that is not a good way to go.
I have entered an Apple bug-report for this (21477730). I also used a DTS to ask for a work-around: there is none. If you are encountering this too, a "me too" to the bug report. This may increase the chances of a fix.
If you want to try this out for yourself, below is the code that I sent in with the bug report.
MPMediaQuery *allAlbumsQuery = [MPMediaQuery albumsQuery];
NSArray *allAlbumsArray = [allAlbumsQuery collections];
for (MPMediaItemCollection *collection in allAlbumsArray)
{
NSArray* items = collection.items;
MPMediaItem* rep = collection.representativeItem;
NSString* name = rep.albumTitle;
for(MPMediaItem* item in items)
{
NSURL* url = item.assetURL;
BOOL isCloudItem = item.isCloudItem;
if(!isCloudItem && (url==nil))
{
NSString* albumTitle = item.albumTitle;
NSString* trackTitle = item.title;
NSLog(#"****Nil: %# %#",albumTitle,trackTitle);
}
}
}
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I try to make an survey/history of all C-to-hardware compilers.
For all that dont know them: they take C code then translate it into some hardware description language (like VHDL or Verilog), which then can be used to generate hardware (usually it gets mapped to an FPGA - but I am not limited to that, ASIC as target would be fine too).
I already have quite collected some (+ info about them). So my question is: Do you know any other compilers (and if so have any references, pointers, information about them)?
My list so far:
AutoESL
Bach-C (Sharp)
C2H (Altera)
C2R (Cebatech)
C2Verilog (CompiLogic / C Level Design / Synposys)
Carte / MAP (SRC Computers)
Cascade (CriticalBlue)
CASH (Carnegie Mellon University, Pittsburgh)
Catapult-C (Mentor Graphics)
CHC (Altium)
CHiMPS (University of Washington (Seattle) / Xilinx / soon commercial?)
C-to-Verilog (Haifa)
Comrade (TU Braunschweig E.I.S. + TU Darmstadt E.S.A.)
CVC (Hitachi)
Cyber (NEC)
Daedalus (Uni Amsterdam, Uni Leiden)
DIME-C (Nallatech)
eXCite (YXI)
FP-Compiler (Altera)
FpgaC (OpenSource)
GarpCC (Callahan, University of California at Berkeley)
GAUT (UBS-Universität Frankreich)
Handel-C (Celoxica)
Hthreads (University of Kansas)
Impulse-C (Impulse Accelerated Technologies)
Mitrion-C (Mitrionics)
DWARV (TU Delft)
NIMBLE (Synopsys, E.I.S. Braunschweig)
NISC (University of California, Irvine)
PICO-Express (Synfora => Synopsys)
PRISC (Harvard University, Cambridge)
ROCCC (University of California, Riverside)
SPARK (University of California, Irvine)
SpecC (Gajski et al.)
Trident (OpenSource, Los Alamos National Laboratory)
UGH
VEAL
vfTools (Vector Fabric)
xPilot (University of California, Los Angeles)
(I know not all on the list have C as soure, some use C-similar dialect, and almost all support only a subset, I am also interrested in such).
EDIT: I know how to use google, so I already checked the ususal suspects and have included the results. So it is very likely that someone can only answer here if he does really know some paper or exotic tool (or maybe not so exotic but that implements the functionality somehow hidden, and the compiler is not advertised).
System-C?
Rotem CToVerilog, I don't know anything about it, just googled it up.
LegUp: http://legup.eecg.utoronto.ca/
There is also HercuLeS (provisional name), which is MY HLS tool.
Get the (old) tech. demo from here
http://www.nkavvadias.co.cc/misc/hls-demo-linux-0.0.1.tar.gz
Things have progressed since then.
Let me know if you want a tech. presentation detailing a real-life examples, e.g. a multi-function CORDIC.
Cheers,
Nikolaos Kavvadias
OpenCL support at Altera and Xilinx.
OpenCV support by Xilinx. OpenCL + OpenCV support by Altera. See this post. I talk about the OpenCL+OpenCV both based on C languages.
Altera has OpenCL SDK which is used with Quartus. Xilinx has Vivado HLS.
Cynthesizer, which is SystemC based. CellMath will go the other way, take Verilog and create a C model.
I wish to search twitter for a word (let's say #google), and then be able to generate a tag cloud of the words used in twitts, but according to dates (for example, having a moving window of an hour, that moves by 10 minutes each time, and shows me how different words gotten more often used throughout the day).
I would appreciate any help on how to go about doing this regarding: resources for the information, code for the programming (R is the only language I am apt in using) and ideas on visualization. Questions:
How do I get the information?
In R, I found that the twitteR package has the searchTwitter command. But I don't know how big an "n" I can get from it. Also, It doesn't return the dates in which the twitt originated from.
I see here that I could get until 1500 twitts, but this requires me to do the parsing manually (which leads me to step 2). Also, for my purposes, I would need tens of thousands of twitts. Is it even possible to get them in retrospect?? (for example, asking older posts each time through the API URL ?) If not, there is the more general question of how to create a personal storage of twitts on your home computer? (a question which might be better left to another SO thread - although any insights from people here would be very interesting for me to read)
How to parse the information (in R)? I know that R has functions that could help from the rcurl and twitteR packages. But I don't know which, or how to use them. Any suggestions would be of help.
How to analyse? how to remove all the "not interesting" words? I found that the "tm" package in R has this example:
reuters <- tm_map(reuters, removeWords, stopwords("english"))
Would this do the trick? I should I do something else/more ?
Also, I imagine I would like to do that after cutting my dataset according to time (which will require some posix-like functions (which I am not exactly sure which would be needed here, or how to use it).
And lastly, there is the question of visualization. How do I create a tag cloud of the words? I found a solution for this here, any other suggestion/recommendations?
I believe I am asking a huge question here but I tried to break it to as many straightforward questions as possible. Any help will be welcomed!
Best,
Tal
Word/Tag cloud in R using "snippets" package
www.wordle.net
Using openNLP package you could pos-tag the tweets(pos=Part of speech) and then extract just the nouns, verbs or adjectives for visualization in a wordcloud.
Maybe you can query twitter and use the current system-time as a time-stamp, write to a local database and query again in increments of x secs/mins, etc.
There is historical data available at http://www.readwriteweb.com/archives/twitter_data_dump_infochimp_puts_1b_connections_up.php and http://www.wired.com/epicenter/2010/04/loc-google-twitter/
As for the plotting piece: I did a word cloud here: http://trends.techcrunch.com/2009/09/25/describe-yourself-in-3-or-4-words/ using the snippets package, my code is in there. I manually pulled out certain words. Check it out and let me know if you have more specific questions.
I note that this is an old question, and there are several solutions available via web search, but here's one answer (via http://blog.ouseful.info/2012/02/15/generating-twitter-wordclouds-in-r-prompted-by-an-open-learning-blogpost/):
require(twitteR)
searchTerm='#dev8d'
#Grab the tweets
rdmTweets <- searchTwitter(searchTerm, n=500)
#Use a handy helper function to put the tweets into a dataframe
tw.df=twListToDF(rdmTweets)
##Note: there are some handy, basic Twitter related functions here:
##https://github.com/matteoredaelli/twitter-r-utils
#For example:
RemoveAtPeople <- function(tweet) {
gsub("#\\w+", "", tweet)
}
#Then for example, remove #d names
tweets <- as.vector(sapply(tw.df$text, RemoveAtPeople))
##Wordcloud - scripts available from various sources; I used:
#http://rdatamining.wordpress.com/2011/11/09/using-text-mining-to-find-out-what-rdatamining-tweets-are-about/
#Call with eg: tw.c=generateCorpus(tw.df$text)
generateCorpus= function(df,my.stopwords=c()){
#Install the textmining library
require(tm)
#The following is cribbed and seems to do what it says on the can
tw.corpus= Corpus(VectorSource(df))
# remove punctuation
tw.corpus = tm_map(tw.corpus, removePunctuation)
#normalise case
tw.corpus = tm_map(tw.corpus, tolower)
# remove stopwords
tw.corpus = tm_map(tw.corpus, removeWords, stopwords('english'))
tw.corpus = tm_map(tw.corpus, removeWords, my.stopwords)
tw.corpus
}
wordcloud.generate=function(corpus,min.freq=3){
require(wordcloud)
doc.m = TermDocumentMatrix(corpus, control = list(minWordLength = 1))
dm = as.matrix(doc.m)
# calculate the frequency of words
v = sort(rowSums(dm), decreasing=TRUE)
d = data.frame(word=names(v), freq=v)
#Generate the wordcloud
wc=wordcloud(d$word, d$freq, min.freq=min.freq)
wc
}
print(wordcloud.generate(generateCorpus(tweets,'dev8d'),7))
##Generate an image file of the wordcloud
png('test.png', width=600,height=600)
wordcloud.generate(generateCorpus(tweets,'dev8d'),7)
dev.off()
#We could make it even easier if we hide away the tweet grabbing code. eg:
tweets.grabber=function(searchTerm,num=500){
require(twitteR)
rdmTweets = searchTwitter(searchTerm, n=num)
tw.df=twListToDF(rdmTweets)
as.vector(sapply(tw.df$text, RemoveAtPeople))
}
#Then we could do something like:
tweets=tweets.grabber('ukgc12')
wordcloud.generate(generateCorpus(tweets),3)
I would like to answer your question in making big word cloud.
What I did is
Use s0.tweet <- searchTwitter(KEYWORD,n=1500) for 7 days or more, such as THIS.
Combine them by this command :
rdmTweets = c(s0.tweet,s1.tweet,s2.tweet,s3.tweet,s4.tweet,s5.tweet,s6.tweet,s7.tweet)
The result:
This Square Cloud consists of about 9000 tweets.
Source: People voice about Lynas Malaysia through Twitter Analysis with R CloudStat
Hope it help!