Per the API documentation, manipulating the vales of "start" and "end" will result in different data sets being returned. Strangely, changing the values of start and end resulted in the same result being returned. What am I missing? Thanks!
qopts = {'query': '/automotive and vehicles',
'aggregation' : '[term(yyymmdd).term(docSentiment.type,count:3)]',
'return': 'docSentiment.type,yyyymmdd',
'count': '50',
'start': 'now-2w',
'end' : 'now-1w',
'offset': my_offset}
my_query = discovery.query(my_disc_environment_id, my_disc_collection_id, qopts)
I hope it is helpful for you.
I am not sure if this is right answer for you because I have limited information.
First of all, please check the number of return-sets. if the return-set has more then 50 dataset, result could be same. (might count param. -1 = unlimited in the rest API of WCA (Watson Context Analytics)
Second, if you can check the log from the server side, you can see the full query which manipulated from WATSON engine.
Last, I am not really sure that watson REST-API can recognize 'now-2w' style start-end form. Would you please link the tutorial? In my previous project, I wrote the start-end date by Y-M-D form.
Good Luck
Related
I am new to API testing and am using Katalon to develop the tests. I've Googled any question I could think of and couldn't find anything to answer my question.
We have an API with the following Body
[
"${idValue1}",
"${idValue2}",
"${idValue3}",
"${idValue4}",
"${idValue5}",
]
I believe the purpose of this one is to delete multiple records at once by id. The script that we have in the step definition is
response = WS.sendRequest(findTestObject('EquipmentAPI/data-objects/DELETE Equipment By Ids', [('host') : GlobalVariable.host, ('idValues') : GlobalVariable.equipId]))
GlobalVariable.equipId = WS.getElementPropertyValue(response, 'data[0].id')
There are other step definitions that run before this one to set the Global Variables for use. I was able to generate the string array without issue.
Is this something that's possible? Please help!
Please let me know if further information is needed. Thanks.
I have been going through the Intuit Developer documentation for about 10 hours trying to find a way to get a list of invoice "custom fields" that have been set up for a Quickbooks Online Company file. I'm not sure if it is even possible. It if is, can anyone point me to some code, documentation, or anything that could possibly help me get such a list. Is it available somehow through the QueryService? Any assistance is greatly appreciated!
This gets custom fields:
public static List<Preferences> getCustomFields()
{
ServiceContext serviceContext = getServiceContext();
QueryService<Preferences> preferencesQueryService = new QueryService<Preferences>(serviceContext);
return preferencesQueryService.ExecuteIdsQuery("Select * FROM Preferences").ToList<Preferences>();
}
List<Preferences> prefs = RestHelper.getCustomFields();
List<CustomFieldDefinition> cusfieldDefs = prefs[0].SalesFormsPrefs.CustomField.ToList() as List<CustomFieldDefinition>;
List<StringTypeCustomFieldDefinition> customFields = cusfieldDefs.OfType<StringTypeCustomFieldDefinition>().ToList();
I started here: API Reference.
As well, to grab all of the fields with values I made a query to the Quickbooks record, with a SELECT * statement. That brings everything back, with data.
I'm working on my first Angular project and I've been thinking of the best way to word this question for a while now but I'm going to give this a shot.
I'm building an app that uses the Veralite (MiOS) to return device data in json format. The first request returns all of the devices on the system (example below).
"devices":[
{
"name":"Bedroom Light",
"altid":"4",
"id":6,
"category":2,
"subcategory":0,
"room":0,
"parent":1,
"status":"0",
"level":"0",
"state":-1,
"comment":""
},
{
"name":"Office Light",
"altid":"6",
"id":18,
"category":2,
"subcategory":0,
"room":0,
"parent":1,
"level":"0",
"status":"0",
"state":-1,
"comment":""
}
Once all of the devices are returned, my script begins long polling the vera engine. Once a change to a device is made the results of the long poll are returned, but the results only include the devices that were changed (example below).
"devices":[
{
"altid":"6",
"id":"18",
"subcategory":"0",
"room":"0",
"parent":"1",
"level":"20",
"status":"1",
"state":"4",
"comment":"Office Light: Transmit was ok"
}
What I am trying to wrap my head around, is the proper way to update the existing devices array with the newly updated data. Would I need to convert them to arrays, then loop through each array and try to match them by keys?
Hopefully I asked this as clearly as possible.
EDIT: Just to update this a bit for anyone that stumbles across this, specifically people interested in developing for the Veralite. The ID returned from the original request will be returned as an integer, but when long polling the engine, the ID will be returned as a string. So even though the selected answer is correct, you'll need to either parse the updated device ID as an integer (parseInt), or only use a == instead of a === when filtering the devices.
You can loop through the object fields just like you can loop through an array with Object.keys.
Conceptually something like:
Step 1) Find the updated product, by the field id:
var previousVersionDevice = $scope.devices.filter(function(item) {
return item.id === updateDevice.id; // Keep only the one existing device that matches id
})[0];
Step 2) Loop through the keys in the product and overwrite the previous values with the ones received.
Object.keys(updatedDevice).forEach(function(key) {
previousVersionDevice[key] = updatedDevice[key]; // Overwrite/add all from updated version
});
yes, you can loop through and find the matching device via some sort of unique key (assuming id is unique and won't change, you can use that).
I have several entities that I am searching across that include dates, and the Search API works great across all of them except for one thing - sorting.
Here's the data model for one of my entities (simplified of course):
class DepositReceipt(ndb.Expando):
#Sets creation date
creation_date = ndb.DateTimeProperty(auto_now_add=True)
And the code to create the search.Document where de is an instance of the entity:
document = search.Document(doc_id=de.key.urlsafe(),
fields=[search.TextField(name='deposit_key', value=de.key.urlsafe()),
search.DateField(name='created', value=de.creation_date),
search.TextField(name='settings', value=de.settings.urlsafe()),
])
This returns a valid document.
And finally the problem line. I took this snippet from the official GAE Search API tutorial and just changed the direction of the sort to DESCENDING and changed the search expression to created (the date property from the Document above).
expr_list = [search.SortExpression(
expression="created", default_value='',
direction=search.SortExpression.DESCENDING)]
I don't think this is important, but the rest of the search code looks like this:
sort_opts = search.SortOptions(expressions=expr_list)
query_options = search.QueryOptions(
cursor=query_cursor,
limit=_NUM_RESULTS,
sort_options=sort_opts)
query_obj = search.Query(query_string=query, options=query_options)
search_results = search.Index(name=index_name).search(query=query_obj)
In production, I get this error message:
InvalidRequest: Failed to parse search request "settings:ag5zfmdoaWRvbmF0aW9uc3IQCxIIU2V0dGluZ3MYmewDDA"; failed to parse date
Changing the expression="created" to anything else works perfectly fine. This also happens across my other entity types that use dates, so I have no idea what's going on. Advice?
I think default_value needs to be a valid date, rather than '' as you have it.
I wish to search twitter for a word (let's say #google), and then be able to generate a tag cloud of the words used in twitts, but according to dates (for example, having a moving window of an hour, that moves by 10 minutes each time, and shows me how different words gotten more often used throughout the day).
I would appreciate any help on how to go about doing this regarding: resources for the information, code for the programming (R is the only language I am apt in using) and ideas on visualization. Questions:
How do I get the information?
In R, I found that the twitteR package has the searchTwitter command. But I don't know how big an "n" I can get from it. Also, It doesn't return the dates in which the twitt originated from.
I see here that I could get until 1500 twitts, but this requires me to do the parsing manually (which leads me to step 2). Also, for my purposes, I would need tens of thousands of twitts. Is it even possible to get them in retrospect?? (for example, asking older posts each time through the API URL ?) If not, there is the more general question of how to create a personal storage of twitts on your home computer? (a question which might be better left to another SO thread - although any insights from people here would be very interesting for me to read)
How to parse the information (in R)? I know that R has functions that could help from the rcurl and twitteR packages. But I don't know which, or how to use them. Any suggestions would be of help.
How to analyse? how to remove all the "not interesting" words? I found that the "tm" package in R has this example:
reuters <- tm_map(reuters, removeWords, stopwords("english"))
Would this do the trick? I should I do something else/more ?
Also, I imagine I would like to do that after cutting my dataset according to time (which will require some posix-like functions (which I am not exactly sure which would be needed here, or how to use it).
And lastly, there is the question of visualization. How do I create a tag cloud of the words? I found a solution for this here, any other suggestion/recommendations?
I believe I am asking a huge question here but I tried to break it to as many straightforward questions as possible. Any help will be welcomed!
Best,
Tal
Word/Tag cloud in R using "snippets" package
www.wordle.net
Using openNLP package you could pos-tag the tweets(pos=Part of speech) and then extract just the nouns, verbs or adjectives for visualization in a wordcloud.
Maybe you can query twitter and use the current system-time as a time-stamp, write to a local database and query again in increments of x secs/mins, etc.
There is historical data available at http://www.readwriteweb.com/archives/twitter_data_dump_infochimp_puts_1b_connections_up.php and http://www.wired.com/epicenter/2010/04/loc-google-twitter/
As for the plotting piece: I did a word cloud here: http://trends.techcrunch.com/2009/09/25/describe-yourself-in-3-or-4-words/ using the snippets package, my code is in there. I manually pulled out certain words. Check it out and let me know if you have more specific questions.
I note that this is an old question, and there are several solutions available via web search, but here's one answer (via http://blog.ouseful.info/2012/02/15/generating-twitter-wordclouds-in-r-prompted-by-an-open-learning-blogpost/):
require(twitteR)
searchTerm='#dev8d'
#Grab the tweets
rdmTweets <- searchTwitter(searchTerm, n=500)
#Use a handy helper function to put the tweets into a dataframe
tw.df=twListToDF(rdmTweets)
##Note: there are some handy, basic Twitter related functions here:
##https://github.com/matteoredaelli/twitter-r-utils
#For example:
RemoveAtPeople <- function(tweet) {
gsub("#\\w+", "", tweet)
}
#Then for example, remove #d names
tweets <- as.vector(sapply(tw.df$text, RemoveAtPeople))
##Wordcloud - scripts available from various sources; I used:
#http://rdatamining.wordpress.com/2011/11/09/using-text-mining-to-find-out-what-rdatamining-tweets-are-about/
#Call with eg: tw.c=generateCorpus(tw.df$text)
generateCorpus= function(df,my.stopwords=c()){
#Install the textmining library
require(tm)
#The following is cribbed and seems to do what it says on the can
tw.corpus= Corpus(VectorSource(df))
# remove punctuation
tw.corpus = tm_map(tw.corpus, removePunctuation)
#normalise case
tw.corpus = tm_map(tw.corpus, tolower)
# remove stopwords
tw.corpus = tm_map(tw.corpus, removeWords, stopwords('english'))
tw.corpus = tm_map(tw.corpus, removeWords, my.stopwords)
tw.corpus
}
wordcloud.generate=function(corpus,min.freq=3){
require(wordcloud)
doc.m = TermDocumentMatrix(corpus, control = list(minWordLength = 1))
dm = as.matrix(doc.m)
# calculate the frequency of words
v = sort(rowSums(dm), decreasing=TRUE)
d = data.frame(word=names(v), freq=v)
#Generate the wordcloud
wc=wordcloud(d$word, d$freq, min.freq=min.freq)
wc
}
print(wordcloud.generate(generateCorpus(tweets,'dev8d'),7))
##Generate an image file of the wordcloud
png('test.png', width=600,height=600)
wordcloud.generate(generateCorpus(tweets,'dev8d'),7)
dev.off()
#We could make it even easier if we hide away the tweet grabbing code. eg:
tweets.grabber=function(searchTerm,num=500){
require(twitteR)
rdmTweets = searchTwitter(searchTerm, n=num)
tw.df=twListToDF(rdmTweets)
as.vector(sapply(tw.df$text, RemoveAtPeople))
}
#Then we could do something like:
tweets=tweets.grabber('ukgc12')
wordcloud.generate(generateCorpus(tweets),3)
I would like to answer your question in making big word cloud.
What I did is
Use s0.tweet <- searchTwitter(KEYWORD,n=1500) for 7 days or more, such as THIS.
Combine them by this command :
rdmTweets = c(s0.tweet,s1.tweet,s2.tweet,s3.tweet,s4.tweet,s5.tweet,s6.tweet,s7.tweet)
The result:
This Square Cloud consists of about 9000 tweets.
Source: People voice about Lynas Malaysia through Twitter Analysis with R CloudStat
Hope it help!