Missing MSDN documentation to develop xll add ins? - c

I spent quite a lot of time in looking for the full documentation of all the C API XLM Functions without success.
I found this page which illustrate a few of them:
http://msdn.microsoft.com/en-us/library/office/bb687910%28v=office.12%29.aspx
But for instance I wanted to understand and use xlfAddMenu, and I cannot find a page on MSDN that explain me.
Do you know if there is any documentation available? Apparently it is not so easy to get there.

There is no exhaustive official documentation for all C API XLM functions. However, as the documentation says following regarding C API XLM functions:
Many more functions are exposed by Excel via the C API that are useful
when you are developing XLLs. They correspond to the Excel worksheet
functions and functions and commands that are available from XLM macro
sheets."
Also, the EXAMPLE.[C,H] files which comme with the installation of the SDK use some of these functions and you can use it to learn how to use them. For instance, the xlfAddMenu is used in the xlAutoOpen callback function.
// In the following block of code, the Generic drop-down menu is created.
// Before creation, a check is made to determine if Generic already
// exists. If not, it is added. If the menu needs to be added, memory is
// allocated to hold the array of menu items. The g_rgMenu[] table is then
// transferred into the newly created array. The array is passed as an
// argument to xlfAddMenu to actually add the drop-down menu before the
// help menu. As a last step the memory allocated for the array is
// released.
//
// This block uses TempStr12() and TempNum12(). Both create a temporary
// XLOPER12. The XLOPER12 created by TempStr12() contains the string passed to
// it. The XLOPER12 created by TempNum12() contains the number passed to it.
// The Excel12f() function frees the allocated temporary memory. Both
// functions are part of the framework library.
Excel12f(xlfGetBar, &xTest, 3, TempInt12(10), TempStr12(L"Generic"), TempInt12(0));
if (xTest.xltype == xltypeErr)
{
hMenu = GlobalAlloc(GMEM_MOVEABLE,sizeof(XLOPER12) * g_rgMenuCols * g_rgMenuRows);
px = pxMenu = (LPXLOPER12) GlobalLock(hMenu);
for (i=0; i < g_rgMenuRows; i++)
{
for (j=0; j < g_rgMenuCols; j++)
{
px->xltype = xltypeStr;
px->val.str = TempStr12(g_rgMenu[i][j])->val.str;
px++;
}
}
xMenu.xltype = xltypeMulti;
xMenu.val.array.lparray = pxMenu;
xMenu.val.array.rows = g_rgMenuRows;
xMenu.val.array.columns = g_rgMenuCols;
Excel12f(xlfAddMenu,0,3,TempNum12(10),(LPXLOPER12)&xMenu,TempStr12(L"Help"));
GlobalUnlock(hMenu);
GlobalFree(hMenu);
}

According to me the best documentation (but not updated..) is the following book : Financial Applications using Excel Add-in Development in C / C++, 2nd Edition by Steve Dalton. You can find description of the xlfAddMenu function page 332. You can also find some useful information in the chm file of the Microsoft Excel XLL Software Development Kit, including codes examples (note I did not found the xlfAddMenu in it so I guess it is a depreciated function).

Related

Release a pointer to a Direct2D Factory COM object in C [d2d1.h]

The problem
I am following a guide from the Microsoft Documentation and the examples there are given in C++. I could've just used the default samples in C++, but I wanted to further understand how the API works so I decided to rewrite the default C++ sample in C.
The problem I encountered is that I can easily call D2D1CreateFactory and create an ID2D1Factory, though when I was previously reading the documentation, it was stated that you have to always release the COM object after you've used it and don't need it anymore. The fact is that in C++ there's an inherited method Release from IUnknown. In C though there's no even a lpVtbl, which as far as I understand is usually needed for that purpose. The ID2D1Factory is just provided as a typedef and is an incomplete type.
And now I'm stuck, because I don't know how to release pointer. I've spent a couple of hours searching for ways to do that in C.
Is it even possible?
The header file: d2d1.h
The Code
Window Procedure simplified:
switch (uMessage)
{
case WM_CREATE:
{
HRESULT hResult = D2D1CreateFactory(D2D1_FACTORY_TYPE_SINGLE_THREADED, &IID_ID2D1Factory, NULL, &pFactory);
if (FAILED(hResult))
{
return -1;
}
return 0;
}
...
}
Obviously pFactory is of an ID2D1Factory* type in global scope. It is zero-initialized by default (if that helps).

Configurable custom code

Our customer provided source code has portions of code that will be executed based on tool type. A sample code portion is given below. The function has common portions and tool specific(hardware platform) portions. The code is written in C and runs in VxWorks. Addition or deletion of new tool type has code modification. The customer wants addition or deletion new tool type with minimal code change and testing effort
int vsp_recv(char *const recv_text)
{
int rc = 0;
const int type = get_tool_type();
// Common Code
if (MODEL_CR == type)
{
rc = beamoff(recv_text);
}
else
{
rc = vsp_set(recv_text);
}
return(rc);
}
Is it the right technique to separate the code to two methods as given below, keep them in separate source files and define separate make files to generate tool specific binary? Is there any better ways to do this?
Tool type MODEL_CR code
int vsp_recv_tool_speccific(char *const recv_text)
{
return beamoff(recv_text);
}
Tool type MODEL_CV code
int vsp_recv_tool_speccific(char *const recv_text)
{
return vsp_set(recv_text);
}
Refactored method
int vsp_recv(char *const recv_text)
{
int rc = 0;
const int type = get_tool_type();
// Common Code
rc = vsp_recv_tool_speccific(recv_text);
}
Define a shared library for each tool and a configuration file that defines what functions get called for each tool. Load the shared libraries at startup and provide a signal catcher to reload if the configuration file changes.
the OPs question (and posted code) says that 3 places will need to be modified.
the function: get_tool_type()
the header file with the definitions of MODEL_CV, MODEL_CR, etc
the if-then-else list.
were it me, I would implement a table of function pointers, have get_tool_type() return an index into that table. Then all the if/then/else code would become a single statement that invokes a function from the table.
Then any updates would be additions to the table, modifications to 'get_too_type(), and the additional functions likebeam_off()`
The loss of a tool type would not require any code change.
the addition of a tool type would require appending an entry to the table, mod to get_tool_type() to recognize the new tool, and the new function to process the new tool type.
Of course, this could result in code that is never executed.

customizing completion of GtkComboBoxText

How can I customize the completion of a GtkComboBoxText with both a "static" aspect and a "dynamic" one? The static aspect is because some entries are known and added to the combo-box-text at construction time with gtk_combo_box_text_append_text. The dynamic aspect is because I also need to complete thru some callback function(s), that is to complete dynamically -after creation of the GtkComboBoxText widget- once several characters has been typed.
My application uses Boehm's GC (except for GTK objects of course) like Guile or SCM or Bigloo are doing. It can be seen as an experimental persistent dynamic-typed programming language implementation with an integrated editor coded on and for Debian/Linux/x86-64 with the system GTK3.21 library, it is coded in C99 (some of which is generated) and is compiled with GCC6.
(I don't care about non-Linux systems, GTK3 libraries older than GTK3.20, GCC compiler older that GCC6)
question details
I'm entering (inputting into the GtkComboBoxText) either a name, or an object-id.
The name is C-identifier-like but starts with a letter and cannot end with an underscore. For example, comment, if, the_GUI, the_system, payload_json, or x1 are valid names (but _a0bcd or foobar_ are invalid names, because they start or end with an underscore). I currently have a big dozen of names, but I could have a few thousands of them. So it would be reasonable to offer a completion once only a single or perhaps two letters has been typed, and completion for names can happen statically because they are not many of them (so I feel reasonable to call gtk_combo_box_append_text for each name).
The object-id starts with an underscore followed by a digit and has exactly 18 alphanumeric (sort-of random) characters. For example, _5Hf0fFKvRVa71ZPM0, _8261sbF1f9ohzu2Iu, _0BV96V94PJIn9si1K are possible object-ids. Actually it is 96 almost random bits (probably only 294 are possible). The object-id plays the role of UUIDs (in the sense that it is assumed to be world-wide unique for distinct objects) but has a C friendly syntax. I currently have a few dozen of objects-ids, but I could have a few hundred of thousands (or maybe a million) of them. But given a prefix of four characters like _6S3 or _22z, I am assuming that only a reasonable number (probably at most a dozen, and surely no more than a thousand) object-ids exist in my application with that prefix. Of course it would be unreasonable to register (statically) a priori all the object ids (the completion has to happen after four characters have been typed, and should happen dynamically).
So I want a completion that works both on names (e.g. typing one letter perhaps followed by another alphanum character should be enough to propose a completion of at most a hundred choices), and on object-ids (typing four characters like _826 should be enough to trigger a completion of probably at most a few dozen choices, perhaps a thousand ones if unlucky).
Hence typing the three keys p a tab would offer completion with a few names like payload_json or payload_vectval etc... and typing the five keys _ 5 H f tab would offer completion with very few object-ids, notably _5Hf0fFKvRVa71ZPM0
sample incomplete code
So far I coded the following:
static GtkWidget *
mom_objectentry (void)
{
GtkWidget *obent = gtk_combo_box_text_new_with_entry ();
gtk_widget_set_size_request (obent, 30, 10);
mo_value_t namsetv = mo_named_objects_set ();
I have Boehm-garbage-collected values, and mo_value_t is a pointer to any of them. Values can be tagged integers, pointers to strings, objects, or tuples or sets of objects. So namesetv now contains the set of named objects (probably less than a few thousand of named objects).
int nbnam = mo_set_size (namsetv);
MOM_ASSERTPRINTF (nbnam > 0, "bad nbnam");
mo_value_t *namarr = mom_gc_alloc (nbnam * sizeof (mo_value_t));
int cntnam = 0;
for (int ix = 0; ix < nbnam; ix++)
{
mo_objref_t curobr = mo_set_nth (namsetv, ix);
mo_value_t curnamv = mo_objref_namev (curobr);
if (mo_dyncast_string (curnamv))
namarr[cntnam++] = curnamv;
}
qsort (namarr, cntnam, sizeof (mo_value_t), mom_obname_cmp);
for (int ix = 0; ix < cntnam; ix++)
gtk_combo_box_text_append_text (GTK_COMBO_BOX_TEXT (obent),
mo_string_cstr (namarr[ix]));
at this point I have sorted all the (few thousands at most) names and added "statically" them using gtk_combo_box_text_append_text.
GtkWidget *combtextent = gtk_bin_get_child (GTK_BIN (obent));
MOM_ASSERTPRINTF (GTK_IS_ENTRY (combtextent), "bad combtextent");
MOM_ASSERTPRINTF (gtk_entry_get_completion (GTK_ENTRY (combtextent)) ==
NULL, "got completion in combtextent");
I noticed with a bit of surprise that gtk_entry_get_completion (GTK_ENTRY (combtextent)) is null.
But I am stuck here. I am thinking of:
Having some mom_set_complete_objectid(const char*prefix) which given a prefix like "_47n" of at least four characters would return a garbage collected mo_value_t representing the set of objects with that prefix. This is very easy to code for me, and is nearly done.
Make my own local GtkEntryCompletion* mycompl = ..., which would complete like I want. Then I would put it in the text entry combtextent of my gtk-combo-box-text using gtk_entry_set_completion(GTK_ENTRY(combtextent), mycompl);
Should it use the entries added with gtk_combo_box_text_append_text for the "static" name completion role? How should I dynamically complete using the dynamic set value returned from my mom_set_complete_objectid; given some object-pointer obr and some char bufid[20]; I am easily and quickly able to fill it with the object-id of that object obr with mo_cstring_from_hi_lo_ids(bufid, obr->mo_ob_hid, obr->mo_ob_loid)..
I don't know how to code the above. For reference, I am now just returning the combo-box-text:
// if the entered text starts with a letter, I want it to be
// completed with the appended text above if the entered text starts
// with an undersore, then a digit, then two alphanum (like _0BV or
// _6S3 for example), I want to call a completion function.
#warning objectentry: what should I code here?
return obent;
} /* end mom_objectentry */
Is my approach the right one?
The mom_objectentry function above is used to fill modal dialogs with short lifetime.
I am favoring simple code over efficiency. Actually, my code is temporary (I'm hoping to bootstrap my language, and generate all its C code!) and in practice I'll probably have only a few hundred names and at most a few dozen of thousands of object-ids. So performance is not very important, but simplicity of coding (some conceptually "throw away" code) is more important.
I don't want (if possible) to add my own GTK classes. I prefer using existing GTK classes and widgets, customizing them with GTK signals and callbacks.
context
My application is an experimental persistent programming language and implementation with a near Scheme or Python (or JavaScript, ignoring the prototype aspect, ...) semantics but with a widely different (not yet implemented in september 7th, 2016) syntax (to be shown & input in GTK widgets), using the Boehm garbage collector for values (including objects, sets, tuples, strings...)... Values (including objects) are generally persistent (except the GTK related data : the application starts with a nearly empty window). The entire language heap is persisted in JSON-like syntax in some Sqlite "database" (generated at application exit) dumped into _momstate.sql which is re-loaded at application startup. Object-ids are useful to show object references to the user in GTK widgets, for persistence, and to generate C code related to the objects (e.g. the object of id _76f7e2VcL8IJC1hq6 could be related to a mo_76f7e2VcL8IJC1hq6 identifier in some generated C code; this is partly why I have my object-id format instead of using UUIDs).
PS. My C code is GPLv3 free software and available on github. It is the MELT monitor, branch expjs, commit e2b3b99ef66394...
NB: The objects mentioned here are implicitly my language objects, not GTK objects. The all have a unique object-id, and some but not most of them are named.
I will not show exact code on how to do it because I never did GTK & C only GTK & Python, but it should be fine as the functions in C and Python functions can easily be translated.
OP's approach is actually the right one, so I will try to fill in the gaps. As the amount of static options is limited probably won't change to much it indeed makes sense to add them using gtk_combo_box_text_append which will add them to the internal model of the GtkComboBoxText.
Thats covers the static part, for the dynamic part it would be perfect if we could just store this static model and replace it with a temporay model using gtk_combo_box_set_model() when a _ was found at the start of the string. But we shouldn't do this as the documentation says:
You should not call gtk_combo_box_set_model() or attempt to pack more cells into this combo box via its GtkCellLayout interface.
So we need to work around this, one way of doing this is by adding a GtkEntryCompletion to the entry of the GtkComboBoxText. This will make the entry attempt to complete the current string based on its current model. As an added bonus it can also add all the character all options have in common like this:
As we don't want to load all the dynamic options before hand I think the best approach will be to connect a changed listener to the GtkEntry, this way we can load the dynamic options when we have a underscore and some characters.
As the GtkEntryCompletion uses a GtkListStore internally, we can reuse part of the code Nominal Animal provided in his answer. The main difference being: the connect is done on the GtkEntry and the replacing of GtkComboText with GtkEntryCompletion inside the populator. Then everything should be fine, I wish I would be able to write decent C then I would have provided you with code but this will have to do.
Edit: A small demo in Python with GTK3
import gi
gi.require_version('Gtk', '3.0')
import gi.repository.Gtk as Gtk
class CompletingComboBoxText(Gtk.ComboBoxText):
def __init__(self, static_options, populator, **kwargs):
# Set up the ComboBox with the Entry
Gtk.ComboBoxText.__init__(self, has_entry=True, **kwargs)
# Store the populator reference in the object
self.populator = populator
# Create the completion
completion = Gtk.EntryCompletion(inline_completion=True)
# Specify that we want to use the first col of the model for completion
completion.set_text_column(0)
completion.set_minimum_key_length(2)
# Set the completion model to the combobox model such that we can also autocomplete these options
self.static_options_model = self.get_model()
completion.set_model(self.static_options_model)
# The child of the combobox is the entry if 'has_entry' was set to True
entry = self.get_child()
entry.set_completion(completion)
# Set the active option of the combobox to 0 (which is an empty field)
self.set_active(0)
# Fill the model with the static options (could also be used for a history or something)
for option in static_options:
self.append_text(option)
# Connect a listener to adjust the model when the user types something
entry.connect("changed", self.update_completion, True)
def update_completion(self, entry, editable):
# Get the current content of the entry
text = entry.get_text()
# Get the completion which needs to be updated
completion = entry.get_completion()
if text.startswith("_") and len(text) >= completion.get_minimum_key_length():
# Fetch the options from the populator for a given text
completion_options = self.populator(text)
# Create a temporary model for the completion and fill it
dynamic_model = Gtk.ListStore.new([str])
for completion_option in completion_options:
dynamic_model.append([completion_option])
completion.set_model(dynamic_model)
else:
# Restore the default static options
completion.set_model(self.static_options_model)
def demo():
# Create the window
window = Gtk.Window()
# Add some static options
fake_static_options = [
"comment",
"if",
"the_GUI",
"the_system",
"payload_json",
"x1",
"payload_json",
"payload_vectval"
]
# Add the the Combobox
ccb = CompletingComboBoxText(fake_static_options, dynamic_option_populator)
window.add(ccb)
# Show it
window.show_all()
Gtk.main()
def dynamic_option_populator(text):
# Some fake returns for the populator
fake_dynamic_options = [
"_5Hf0fFKvRVa71ZPM0",
"_8261sbF1f9ohzu2Iu",
"_0BV96V94PJIn9si1K",
"_0BV1sbF1f9ohzu2Iu",
"_0BV0fFKvRVa71ZPM0",
"_0Hf0fF4PJIn9si1Ks",
"_6KvRVa71JIn9si1Kw",
"_5HKvRVa71Va71ZPM0",
"_8261sbF1KvRVa71ZP",
"_0BKvRVa71JIn9si1K",
"_0BV1KvRVa71ZPu2Iu",
"_0BV0fKvRVa71ZZPM0",
"_0Hf0fF4PJIbF1f9oh",
"_61sbFV0fFKn9si1Kw",
"_5Hf0fFKvRVa71ozu2",
]
# Only return those that start with the text
return [fake_dynamic_option for fake_dynamic_option in fake_dynamic_options if fake_dynamic_option.startswith(text)]
if __name__ == '__main__':
demo()
Gtk.main()
Here is my suggestion:
Use a GtkListStore to contain a list of GTK-managed strings (essentially, copies of your identifier string) that match the current prefix string.
(As documented for gtk_list_store_set(), a G_TYPE_STRING item is copied. I consider the overhead of the extra copy acceptable here; it should not affect real-world performance much anyway, I think, and in return, GTK+ will manage the reference counting for us.)
The above is implemented in a GTK+ callback function, which gets an extra pointer as payload (set at the time the GUI is created or activated; I suggest you use some structure to keep references you need to generate the matches). The callback is connected to the combobox popup signal, so that it gets called whenever the list is expanded.
Note that as B8vrede noted in a comment, a GtkComboBoxText should not be modified via its model; that is why one should/must use a GtkComboBox instead.
Practical example
For simplicity, let's assume all the data you need to find or generate all known identifiers matched against is held in a structure, say
struct generator {
/* Whatever data you need to generate prefix matches */
};
and the combo box populator helper function is then something like
static void combo_box_populator(GtkComboBox *combobox, gpointer genptr)
{
struct generator *const generator = genptr;
GtkListStore *combo_list = GTK_LIST_STORE(gtk_combo_box_get_model(combobox));
GtkWidget *entry = gtk_bin_get_child(GTK_BIN(combobox));
const char *prefix = gtk_entry_get_text(GTK_ENTRY(entry));
const size_t prefix_len = (prefix) ? strlen(prefix) : 0;
GtkTreeIter iterator;
/* Clear the current store */
gtk_list_store_clear(combo_list);
/* Initialize the list iterator */
gtk_tree_model_get_iter_first(GTK_TREE_MODEL(combo_list), &iterator);
/* Find all you want to have in the combo box;
for each const char *match, do:
*/
gtk_list_store_append(combo_list, &iterator);
gtk_list_store_set(combo_list, &iterator, 0, match, -1);
/* Note that the string pointed to by match is copied;
match is not referred to after the _set() returns.
*/
}
When the UI is built or activated, you need to ensure the GtkComboBox has an entry (so the user can write text into it), and a GtkListStore model:
struct generator *generator;
GtkWidget *combobox;
GtkListStore *combo_list;
combo_list = gtk_list_store_new(1, G_TYPE_STRING);
combobox = gtk_combo_box_new_with_model_and_entry(GTK_TREE_MODEL(combo_list));
gtk_combo_box_set_id_column(GTK_COMBO_BOX(combobox), 0);
gtk_combo_box_set_entry_text_column(GTK_COMBO_BOX(combobox), 0);
gtk_combo_box_set_button_sensitivity(GTK_COMBO_BOX(combobox), GTK_SENSITIVITY_ON);
g_signal_connect(combobox, "popup", G_CALLBACK(combo_box_populator), generator);
On my system, the default pop-up accelerator is Alt+Down, but I assume you've already changed that to Tab.
I have a crude working example here (a .tar.xz tarball, CC0): it reads lines from standard input, and lists the ones matching the user prefix in reverse order in the combo box list (when popped-up). If the entry is empty, the combobox will contain all input lines. I didn't change the default accelerators, so instead of Tab, try Alt+Down.
I also have the same example, but using GtkComboBoxText instead, here (also CC0). This does not use a GtkListStore model, but uses gtk_combo_box_text_remove_all() and gtk_combo_box_text_append_text() functions to manipulate the list contents directly. (There is just a few different lines in the two examples.) Unfortunately, the documentation is not explicit whether this interface references or copies the strings. Although copying is the only option that makes sense, and this can be verified from the current Gtk+ sources, the lack of explicit documentation makes me hesitant.
Comparing the two examples I linked to above (both grab some 500 random words from /usr/share/dict/words if you compile and run it with make), I don't see any speed difference. Both use the same naïve way of picking prefix matches from a linked list, which means the two methods (GtkComboBox + model, or GtkComboBoxText) should be about equally fast.
On my own machine, both get annoyingly slow with more than 1000 or so matches in the popup; with just a hundred or less matches, it feels instantaneous. This, to me, indicates that the slow/naïve way of picking prefix matches from a linked list is not the culprit (because the entire list is traversed in both cases), but that the GTK+ combo boxes are just not designed for large lists. (The slowdown is definitely much, much worse than linear.)

Composite C1 reference to Composite.dll not working at build time

I'm trying to implement a console library that reads data from Composite C1 (global datatype called RSS Feeds) and then, foreach RSS feed, the application must retrieve rss entries from the "link" attribute and insert all entries into a global datatype called "RSSItem".
Here is what i've done:
1. Open the website composite Solution
2. Create a new Console Library Project
3. Reference Composite.dll, Composite.generated.dll, ... into my new project
4. Implement the functionnality
Here is the problem:
At the design time, i have all reference working perfectly fine, I can write my code with intellisense. But when i want to launch the project (debug | release mode), the reference to composite is not working anymore ...
"Error 15 The type or namespace name 'Composite' could not be found (are you missing a using directive or an assembly reference?)"
When i do a refresh in the project browser, intelisense works again.
Thanks for your help.
Best regards,
Jonathan
PS: sorry for my english, not my native language
For info: here is a little bit of the code:
List<MCG.RSSItem> rssItemList = new List<MCG.RSSItem>();
for (int i = 0; i < 10; i++)
{
MCG.RSSItem rssItem = DataConnection.New<MCG.RSSItem>();
rssItem.Link = rss.Link;
rssItem.RSSFeed = rssFeed.Id;
rssItem.Summary = rss.Description;
rssItem.Title = rss.Title;
rssItem.PublicationStatus = "published";
rssItem.Id = Guid.NewGuid();
connection.Add<MCG.RSSItem>(rssItemList);
}
This problem was discussed here - http://compositec1.codeplex.com/discussions/357939
The problem was that Composite C1 from a std. Windows application is not supported.
Based on this problem the feature request was created - Refactor core parts of C1 to be used in "selfhost"

Plotting a word-cloud by date for a twitter search result? (using R)

I wish to search twitter for a word (let's say #google), and then be able to generate a tag cloud of the words used in twitts, but according to dates (for example, having a moving window of an hour, that moves by 10 minutes each time, and shows me how different words gotten more often used throughout the day).
I would appreciate any help on how to go about doing this regarding: resources for the information, code for the programming (R is the only language I am apt in using) and ideas on visualization. Questions:
How do I get the information?
In R, I found that the twitteR package has the searchTwitter command. But I don't know how big an "n" I can get from it. Also, It doesn't return the dates in which the twitt originated from.
I see here that I could get until 1500 twitts, but this requires me to do the parsing manually (which leads me to step 2). Also, for my purposes, I would need tens of thousands of twitts. Is it even possible to get them in retrospect?? (for example, asking older posts each time through the API URL ?) If not, there is the more general question of how to create a personal storage of twitts on your home computer? (a question which might be better left to another SO thread - although any insights from people here would be very interesting for me to read)
How to parse the information (in R)? I know that R has functions that could help from the rcurl and twitteR packages. But I don't know which, or how to use them. Any suggestions would be of help.
How to analyse? how to remove all the "not interesting" words? I found that the "tm" package in R has this example:
reuters <- tm_map(reuters, removeWords, stopwords("english"))
Would this do the trick? I should I do something else/more ?
Also, I imagine I would like to do that after cutting my dataset according to time (which will require some posix-like functions (which I am not exactly sure which would be needed here, or how to use it).
And lastly, there is the question of visualization. How do I create a tag cloud of the words? I found a solution for this here, any other suggestion/recommendations?
I believe I am asking a huge question here but I tried to break it to as many straightforward questions as possible. Any help will be welcomed!
Best,
Tal
Word/Tag cloud in R using "snippets" package
www.wordle.net
Using openNLP package you could pos-tag the tweets(pos=Part of speech) and then extract just the nouns, verbs or adjectives for visualization in a wordcloud.
Maybe you can query twitter and use the current system-time as a time-stamp, write to a local database and query again in increments of x secs/mins, etc.
There is historical data available at http://www.readwriteweb.com/archives/twitter_data_dump_infochimp_puts_1b_connections_up.php and http://www.wired.com/epicenter/2010/04/loc-google-twitter/
As for the plotting piece: I did a word cloud here: http://trends.techcrunch.com/2009/09/25/describe-yourself-in-3-or-4-words/ using the snippets package, my code is in there. I manually pulled out certain words. Check it out and let me know if you have more specific questions.
I note that this is an old question, and there are several solutions available via web search, but here's one answer (via http://blog.ouseful.info/2012/02/15/generating-twitter-wordclouds-in-r-prompted-by-an-open-learning-blogpost/):
require(twitteR)
searchTerm='#dev8d'
#Grab the tweets
rdmTweets <- searchTwitter(searchTerm, n=500)
#Use a handy helper function to put the tweets into a dataframe
tw.df=twListToDF(rdmTweets)
##Note: there are some handy, basic Twitter related functions here:
##https://github.com/matteoredaelli/twitter-r-utils
#For example:
RemoveAtPeople <- function(tweet) {
gsub("#\\w+", "", tweet)
}
#Then for example, remove #d names
tweets <- as.vector(sapply(tw.df$text, RemoveAtPeople))
##Wordcloud - scripts available from various sources; I used:
#http://rdatamining.wordpress.com/2011/11/09/using-text-mining-to-find-out-what-rdatamining-tweets-are-about/
#Call with eg: tw.c=generateCorpus(tw.df$text)
generateCorpus= function(df,my.stopwords=c()){
#Install the textmining library
require(tm)
#The following is cribbed and seems to do what it says on the can
tw.corpus= Corpus(VectorSource(df))
# remove punctuation
tw.corpus = tm_map(tw.corpus, removePunctuation)
#normalise case
tw.corpus = tm_map(tw.corpus, tolower)
# remove stopwords
tw.corpus = tm_map(tw.corpus, removeWords, stopwords('english'))
tw.corpus = tm_map(tw.corpus, removeWords, my.stopwords)
tw.corpus
}
wordcloud.generate=function(corpus,min.freq=3){
require(wordcloud)
doc.m = TermDocumentMatrix(corpus, control = list(minWordLength = 1))
dm = as.matrix(doc.m)
# calculate the frequency of words
v = sort(rowSums(dm), decreasing=TRUE)
d = data.frame(word=names(v), freq=v)
#Generate the wordcloud
wc=wordcloud(d$word, d$freq, min.freq=min.freq)
wc
}
print(wordcloud.generate(generateCorpus(tweets,'dev8d'),7))
##Generate an image file of the wordcloud
png('test.png', width=600,height=600)
wordcloud.generate(generateCorpus(tweets,'dev8d'),7)
dev.off()
#We could make it even easier if we hide away the tweet grabbing code. eg:
tweets.grabber=function(searchTerm,num=500){
require(twitteR)
rdmTweets = searchTwitter(searchTerm, n=num)
tw.df=twListToDF(rdmTweets)
as.vector(sapply(tw.df$text, RemoveAtPeople))
}
#Then we could do something like:
tweets=tweets.grabber('ukgc12')
wordcloud.generate(generateCorpus(tweets),3)
I would like to answer your question in making big word cloud.
What I did is
Use s0.tweet <- searchTwitter(KEYWORD,n=1500) for 7 days or more, such as THIS.
Combine them by this command :
rdmTweets = c(s0.tweet,s1.tweet,s2.tweet,s3.tweet,s4.tweet,s5.tweet,s6.tweet,s7.tweet)
The result:
This Square Cloud consists of about 9000 tweets.
Source: People voice about Lynas Malaysia through Twitter Analysis with R CloudStat
Hope it help!

Resources