We are trying to build a histogram of session length in a given time period. Currently, we have sess:start and sess:end events which contains the session id and user id. I am wondering what's the best way to compute this data? Can this be achieve using the funnel api?
Have you checkout out the recipes section in Keen IO's docs? Here is an excerpt from the section on histogram recipes for Session Length that might be really helpful.
Excerpt
To create a histogram for session lengths, like the one shown above,
you can run a count analysis on an event collection for completed
sessions (e.g. session_end). Along the x-axis you’ll have segments of
time lapsed in a session, and along the y-axis you’ll have the
percentage of sessions that fit into a given session length cohort.
Note: this recipe incorporates the D3 histogram recipe, which is
explained further in the documentation.
histogram('chart-1', {
segment_length: 60, // In seconds
data_points: 10, // i.e. There will be 10 bars on our chart
analysis_type: 'count',
query_parameters: {
event_collection: 'session_end',
timeframe: timeframe,
filters: []
}
});
More information
Keen IO - Analytics for Developers
Keen IO - Documentation
Code excerpt: Keen IO - Recipes for Histograms
Lots of good stuff behind the link that Stephanie posted.
One extra thing I'll venture is that putting an integer sess:length property in the sess:end event would make things easier. You'd have to keep the start time for each session somewhere in your database so that you can compute the difference for the sess:end event. But then you'd have the difference as a plain old number of seconds and can do any type of numerical analysis on it.
Related
I am collecting data for a project. The data collection is done by recording videos of the subjects and the environment. However, while training the network, I would not want to train it with all the images collected in the video sequence.
The main objective is to not train the network with redundant images. The video sequence collected at 30 frames/sec can have redundant images (images that are very similar) within the short intervals. T(th) frame and (T+1)th frame can be similar.
Can someone suggest ways to extract only the images that can be useful for training ?
Update #2: Further resources,
https://github.com/JohannesBuchner/imagehash
https://www.pyimagesearch.com/2017/11/27/image-hashing-opencv-python/
https://www.pyimagesearch.com/2020/04/20/detect-and-remove-duplicate-images-from-a-dataset-for-deep-learning/
Update #1: You can use this repo to calculate similarity between given images. https://github.com/quickgrid/image-similarity**
If frames with certain objects(e.g., vehicle, device) are important, then use pretrained object detectors if available, to extract important frames.
Next, use a similarity method to remove similar images in nearby frames. Until a chosen threshold is exceeded keep removing nearby N frames.
This link should be helpful in finding right method for your case,
https://datascience.stackexchange.com/questions/48642/how-to-measure-the-similarity-between-two-images
This repository below should help implement the idea with few lines of code. It uses CNN to extract features then calculates there cosine distance as mentioned there.
https://github.com/ryanfwy/image-similarity
Good afternon! How can i get a route summary for each country in the route? For example, I want to move from Ukraine to France. I get route from GetRouteDirections, the route runs through 4 countries, and i want to see - how many kilometers this route takes over each of the countries of the route
Currently the Azure Maps routing service does not provide an option to return this information, although it is something being considered as a future feature. I have seen this question in the past with other mapping platforms, and there was nothing out of the box for this in those platforms either.
That said, if you want to achieve this today you can do the following:
Get a set of country boundaries that are suitable for your users (if you have users in countries that have disputed borders, you will likely need several sets of country boundaries).
Calculate a route and use path to create a line.
Calculate the intersection of the line with each country boundary and then measure the length of the section of line that intersects.
I've done this as a proof of concept in the past using a set of GeoJSON country boundaries and the JavaScript topology suite (https://github.com/bjornharrtell/jsts). I haven't posted the code online as country borders are disputed in some regions and thus, any sample I put out there wouldn't work globally.
A global solution could be achieved by retrieving the country boundaries from Azure Maps, but you would need to retrieve the boundaries for all countries first, which would be a bit time consuming and generate a lot of transactions.
I am trying to understand how the metrics.rollingStats.timeInMilliseconds and metrics.rollingStats.numBuckets work together.
If I have the following configuration:
circuitBreaker.requestVolumeThreshold=20
circuitBreaker.errorThresholdPercentage=50
metrics.rollingStats.timeInMilliseconds=10000
metrics.rollingStats.numBuckets=10
To me this means:
1) I need a min of 20 request in my window before a decision will be made
2) At or more than 50% of the requests will need to fail for the breaker to open
But how does the number of buckets come into play? Is the requestVolumeThreshHold and error threshold per bucket? I am trying to understand if/how the buckets are used in determining to trip the breaker.
I was able to answer my own question by creating a simple app that tests the circuit breaker.
The equation to determine when to open the circuit uses request volume threshold, error threshold percentage during the entire rolling stats timeInMilliseconds window. The buckets are only used for how/when the rolling window is updated.
I have build an analytic app for a company that shows Key Performance Indicators(KPIs) based on selling/buying of items of their employees.
Now what they want is to add event functionality like:
* alert them if any employee sell more than $10,000 worth items
* or alert them if some employee is tired of life and killed himself/herself
basically these silly events.
I am totally blank about what to do, they recommended using ESPER but I find it very tough to understand how event processing works.
I want to know how Event Processing works in such cases and where can i learn more about it.
See besides programming and DB i know nothing and I am not a PRO too.
Please share your opinions on what am I supposed to do?
Basically, complex event processing is used when you need to analyze "streams" of data, and when you have to react quickly when a seeked pattern emerges from it. For example when three guys flying from different airports are carrying ingredients to potentially build a bomb, and they're heading to the same destination, you need to find that correlation in data flowing from each of the airports, and react to the event quickly.
Employee selling more than 10000 is not something you need to know about in real-time. You can reward him next month, and for that "normal" reporting will work just fine.
Some papers to read:
Oracle CEP
CEP is an excellent way to accomplish your task actually. You can think of CEP as a pattern matching technique. Once it's matched you can be notified or launch another process if your CEP is integrated with other tools.
Here's the example from Wikipedia. We can easily infer that it's a wedding based on these incoming signals:
church bells ringing.
the appearance of a man in a tuxedo with a woman in a flowing white gown.
rice flying through the air.
This is an abstract example, and things like this probably aren't getting put into any system, but you get the idea. It's best to see the code to understand a CEP implimentation. Here's exactly the script you can write on top of NebriOS to build the inference. Other tools like Drools will accomplish the same thing.
class wedding_detect(NebriOS):
def check(self):
if self.church_bells == "ringing" and \
(self.clothes_type == "tuxedo" or \
self.clothes_type == "wedding gown") and \
self.rice_flying == True:
return True
else
return False
def action(self):
# fires if the check() is true
send_email("me#example.com", "A wedding is underway")
For your situation, CEP is easy to program also:
class sales_alert(NebriOS):
def check(self):
return self.sales_total > 10000
def action(self):
send_email("me#example.com", "You got a $10k sale!")
The sales_total can come into the system from your CRM for example.
That's CEP - Complex Event Processing! A start anyways.
how much time does it take to create a simple dance performing bot in second life using Linden Scripting Language ?? , i have no prior knowledge of lsl , but i know various object oriented and event driven programming languages .
Well, it's pretty simple to animate avatar: you'll need a dance animation (those are pretty easy to find, or you could create your own), put it in a prim (which is basic building object in SL), and then create a simple script which first requests permission to animate desired avatar:
llRequestPermissions(agent_key, PERMISSION_TRIGGER_ANIMATION);
You receive response in run_time_permissions event and trigger your animation by name:
run_time_permissions(integer param)
{
llStartAnimation("my animation");
}
Those are the essentials; you'll probably want to request permissions when an avatar touches your object, and stop animation on second touch... or you could request for permissions for each avatar which is in certain range.
As for the "bot" part, Second Life viewer code is open source; if you don't want to build it yourself, there are several customizable bots available. Or you could simply run an official SL viewer and leave it on; there is a way to run several instances of SL viewer at the same time. It all depends on what exactly you need it for.
Official LSL portal can be found here, though I prefer slightly outdated LSL wiki.
Slight language mismatch: To make an object perform a dance is currently known as "puppetry" in a SecondLife context. The term "bot" currently means control of an avatar by an external script api.
Anyway in one instance, it took me about two hours to write, when I did one for a teddy bear a few weeks back, but that was after learning alot from taking apart some old ones, and i never did finish making the dance, it just waggles the legs or hugs with the arms, but the script is capable for however many steps and parts you can cram in memory.
Puppetry of objects has not improved much in the past decade. It is highly restricted by movement update rates and script limitations. Movement is often delayed under server load and the client doesn't always get updates, which produces pauses and skips in varying measure. Scripts have a maximum size of 64k which should be plenty but in actuality runs out fast with the convolutions necessary in lsl. Moving each linked prim in an object seperately used to need a script in each prim, until new fuctions were introduced to apply attributes by linknumber, still many objects use old scripts which may never be updated. Laggy puppets make for a pitiful show, but most users would not know how to identify a good puppetry script.
The popular way to start making a puppet script is to find an older open source puppet script online, and update it to work from one script. Some archane versions are given as 'master' and 'slave' scripts which need to be merged placing the slave actions as a function into the master, changing llMessageLinked( ) for the function name. Others use an identical script for each prim. I said popular, not the simplest or easiest, and it will never be the best.
Scripting from scratch, the active flow needs to go in a timer event with nothing else in it. Use a different state for animating if a timer is needed when waiting because it's a heavy activity and you don't need any more ifs or buts.
The most crucial task is create a loop to build parameters from a list of linknumbers, positions, and rotations into a parameter list before the llSetLinkPrimitiveParamsFast( ). Yes, that's what they called it because it's a heavy list based function, you may just call it SLPPF inworld but not in the script. Because SLPPF requires the call to have certain constants for each parameter, the parameter list for each step will need to include PRIM_LINK_TARGET, linknumber, PRIM_POS, position, PRIM_ROT, rotation for each linked part in the animation step.
Here's an example of putting a list of a single puppetry step into SLPPF.
list parameters;
integer index;
while ( index < total ) { // total number of elements in each list
parameters += [
PRIM_LINK_TARGET, llList2Integer( currentstep, index ),
PRIM_POS, llList2Vector( currentstep, index+1 ),
PRIM_ROT, llList2Rotation( currentstep, index+2 )
];
index += 3;
}
llSetLinkPrimitiveParamsFast( 0, parameters );
How you create your currentstep list is another matter but the smoothest movement over many linked parts will only be possible if the script isn't moving lists around. So if running the timer at 0.2 seconds isn't any improvement over 0.3, it's because you've told lsl to shovel too many lists. This loop with three list calls may handle about 20 links at 0.1 seconds if the weather's good.
That's actually the worst of it over, just be careful of memory if cramming too many step lists into memory at once. Oh and if your object just vanishes completely, it's going to be hanging around near <0,0,0> because a 1 landed in the PRIM_LINK_TARGET hole.