Strava - Group Route Proximity with Latitude, Longitude & Time - maps

Question

Question: What is the most computationally efficient way to determine if two bike riders rode together given a stream of data with time, latitude, and longitude?
Background: I'm an avid cyclist and want to reverse engineer how Strava groups bike riders together. Here is their method to determine if cyclists are riding together (they use time and lat/lon of a ride): https://support.strava.com/hc/en-us/articles/216919497-Why-don-t-I-get-grouped-in-Activities-when-I-rode-ran-with-others-
After a bike ride is complete I have a file of latitude and longitude every second.
Rider 1 Route:
Rider 2 Route:
You can see Rider 1 and 2 rode together, but Rider 2 started from a different spot and joined Rider 1 later.
I want to come up with the least computational intensive way of determining these two riders rode together, despite starting from different locations.
I think Strava's approach is good - basically establish a proximity zone (150 meters) around each point on the route and compare routes of the rider to see if the riders spent 70% of their time within 150 meters of each other.
Rider 1 - Locations:
2016-03-27T11:47:45Z 42.113059 -87.736485
2016-03-27T11:47:46Z 42.113081 -87.736511
2016-03-27T11:47:47Z 42.113105 -87.736538
2016-03-27T11:47:48Z 42.113142 -87.736564
2016-03-27T11:47:49Z 42.113175 -87.736587
Rider 2 - Locations:
-2016-03-27T11:47:45Z 42.113049 -87.736394 <= Find same time of Rider 1 and determine if within 150 meters. If < 150 meters assign 1, if > 150 assign 0.
I would iterate over every point of Rider 2 against every point of Rider 1. Then sum up the 1s and 0s. If the (sum of 1s and 0s) / (total points) is greater than 70% riders are grouped together.
I think this method would generally work, but seems very computational intensive, especially if there are thousands of riders to evaluate. Also, the data does not always have latitude and longitude every second. One method would be to average the location every minute and compare the average location by minute. At least it would reduce iterations by 60 times.
I was hoping there was some statistical or GIS method to establish the "signature" of a route and compare signatures rather than a point by point comparison.
Any thoughts on how to compute the route comparison in the most efficient way?
Note: I posted a similar question on the GIS forum, but no one responded yet. Although, I do think the question written here is more clear. https://gis.stackexchange.com/questions/187019/strava-activity-route-grouping

Answer 1

I'm going to assume the following is true:
for each cyclist C, there is a data stream of time T, longitude X and latitude Y (we're using projected X and Y for simplification, not caring about the projection; however, we should)
data stream can be written into database or another kind of persistent data storage
the data stream for C is sampled at rate of 1s, given that there is no guarantee that every sample is taken; we have to assume that sample is taken in more than 50% cases (preferably > 95%; 99,7% would be perfect)
In that case, one table in database contains all of the data needed for analytics. Let's see what does it look like for two cyclist C1 and C2, compared one to another.
╔════╦════╦════╦════╦════╦═══════╗
║ T ║ X1 ║ Y1 ║ X2 ║ Y2 ║ D ║
╠════╬════╬════╬════╬════╬═══════╣
║ 1 ║ 10 ║ 15 ║ - ║ - ║ - ║
║ 2 ║ 11 ║ 16 ║ - ║ - ║ - ║
║ 3 ║ 11 ║ 17 ║ 19 ║ 11 ║ 10,00 ║
║ 4 ║ 12 ║ 18 ║ 18 ║ 11 ║ 9,22 ║
║ 5 ║ 12 ║ 17 ║ 17 ║ 12 ║ 7,07 ║
║ 6 ║ - ║ - ║ 15 ║ 12 ║ - ║
║ 7 ║ 13 ║ 16 ║ 14 ║ 13 ║ 3,16 ║
║ 8 ║ 13 ║ 15 ║ 13 ║ 14 ║ 1,00 ║
║ 9 ║ 14 ║ 14 ║ 13 ║ 14 ║ 1,00 ║
║ 10 ║ 14 ║ 13 ║ 14 ║ 13 ║ 0,00 ║
║ 11 ║ 14 ║ 14 ║ 14 ║ 14 ║ 0,00 ║
║ 12 ║ 14 ║ 15 ║ 14 ║ 14 ║ 1,00 ║
║ 13 ║ 15 ║ 15 ║ 15 ║ 15 ║ 0,00 ║
║ 14 ║ 15 ║ 16 ║ 15 ║ 16 ║ 0,00 ║
║ 15 ║ 16 ║ 16 ║ 16 ║ 17 ║ 1,00 ║
║ 16 ║ 17 ║ 18 ║ 16 ║ 16 ║ 2,24 ║
╚════╩════╩════╩════╩════╩═══════╝
This comparison can easily be done using e.g. SELECT in database, self-joining a table for two cyclists. For a reasonable number of rows (e.g. <10E5, <10E6) and correctly set indexes, this computation is not resource intensive at all. Especially if we take into the consideration that the database query can be written in such a way that value D is not output for every position, but calculated jut in order to aggregate (count) the value. In that case, all you need is a ratio of count of rows where D is less of equal your preferred treshold D0 vs total count of rows. If that ratio is equal or more than your limit (say, 70%), cyclists went on a ride together.
Let's see an example. If there is such table in the database, named CyclistPosition:
CyclistId - identifier of the cyclist
SamplingTime - UTC time of the sample (position) taken
Long - longitude
Lat - latitude
...with the following data:
╔═══════════╦═══════════════════════╦═══════════╦════════════╗
║ CyclistId ║ SamplingTime ║ Long ║ Lat ║
╠═══════════╬═══════════════════════╬═══════════╬════════════╣
║ 1 ║ 2016-03-27T11:47:45Z ║ 42,113059 ║ -87,736485 ║
║ 1 ║ 2016-03-27T11:47:46Z ║ 42,113081 ║ -87,736511 ║
║ 1 ║ 2016-03-27T11:47:47Z ║ 42,113105 ║ -87,736538 ║
║ 1 ║ 2016-03-27T11:47:48Z ║ 42,113142 ║ -87,736564 ║
║ 1 ║ 2016-03-27T11:47:49Z ║ 42,113175 ║ -87,736587 ║
║ 2 ║ 2016-03-27T11:47:45Z ║ 42,113059 ║ -87,736394 ║
║ 2 ║ 2016-03-27T11:47:46Z ║ 42,113085 ║ -87,736481 ║
║ 2 ║ 2016-03-27T11:47:47Z ║ 42,113103 ║ -87,736531 ║
║ 2 ║ 2016-03-27T11:47:48Z ║ 42,113139 ║ -87,736572 ║
║ 2 ║ 2016-03-27T11:47:49Z ║ 42,113147 ║ -87,736595 ║
╚═══════════╩═══════════════════════╩═══════════╩════════════╝
...then we can extract data for the cyclists 1 and 2 using:
SELECT SamplingTime, Long, Lat FROM CyclistPosition WHERE CyclistId = 1
SELECT SamplingTime, Long, Lat FROM CyclistPosition WHERE CyclistId = 2
...and cross-reference that data using this query...
SELECT
cp1.SamplingTime,
Long1 = cp1.Long,
Lat1 = cp1.Lat,
Long2 = cp2.Long,
Lat2 = cp2.Lat
FROM
CyclistPosition cp1
JOIN CyclistPosition cp2
ON cp2.SamplingTime = cp1.SamplingTime
WHERE
cp1.CyclistId = 1
AND cp2.CyclistId = 2
We now have this kind of output, and if we include rougly calculated X and Y (using Mercator), we get:
╔═══════════════════════╦═══════════╦════════════╦═══════════╦════════════╦══════════════╗
║ SamplingTime ║ Long1 ║ Lat1 ║ Long2 ║ Lat2 ║ Dm ║
╠═══════════════════════╬═══════════╬════════════╬═══════════╬════════════╬══════════════╣
║ 2016-03-27T11:47:45Z ║ 42,113059 ║ -87,736485 ║ 42,113059 ║ -87,736394 ║ 10,118517 ║
║ 2016-03-27T11:47:46Z ║ 42,113081 ║ -87,736511 ║ 42,113085 ║ -87,736481 ║ 3,334919 ║
║ 2016-03-27T11:47:47Z ║ 42,113105 ║ -87,736538 ║ 42,113103 ║ -87,736531 ║ 0,777079 ║
║ 2016-03-27T11:47:48Z ║ 42,113142 ║ -87,736564 ║ 42,113139 ║ -87,736572 ║ 0,890572 ║
║ 2016-03-27T11:47:49Z ║ 42,113175 ║ -87,736587 ║ 42,113147 ║ -87,736595 ║ 0,900635 ║
╚═══════════════════════╩═══════════╩════════════╩═══════════╩════════════╩══════════════╝
Note that for a rough calculation of distance in meters you have to find the formula; I used the one here:
http://bluemm.blogspot.hr/2007/01/excel-formula-to-calculate-distance.html
Now we have to aggregate the data and count it. We have to limit the data to start and end time (T1 and T2) and establish the maximum distance (D0) to say cyclists are riding together. The simple way to do that in SQL would be:
DECLARE #togetherPositions int
DECLARE #allPositions int
DECLARE #ratio decimal(18,2)
SELECT #togetherPositions = count(*)
FROM
CyclistPosition cp1
JOIN CyclistPosition cp2
ON cp2.SamplingTime = cp1.SamplingTime
WHERE
cp1.SamplingTime BETWEEN #T1 AND #T2
AND {formula to get distance in meters} <= #D0
SELECT #allPositions = count(*)
FROM
CyclistPosition cp1
JOIN CyclistPosition cp2
ON cp2.SamplingTime = cp1.SamplingTime
WHERE
cp1.SamplingTime BETWEEN #T1 AND #T2
SET #ratio = #togetherPositions / #allPositions * 1.0
Now you just have to decide if the ratio is 0.7, 0.8, 0.85...
HTH