Relate indices where two dataframes are equal with elements in another array - arrays

Question

I have an array cluster_true and a dataframe data containing in each row a 2D coordinate. I want to save in another dataframe information regarding how many times for a given 2D coordinate each element in cluster_true appeared. So, for instance, for the coordinate (1,1), I want to check all the rows in data whose first two columns have the value of 1, and then check the values of cluster_true at those indices. Here is an example to make it clearer (it gives the desired result):
# Example variables
cluster_true = c(1,2,1,1,2,2,1,2,2,2,2,1,1)
x = 3
y = 3
data = data.frame(X = c(1,1,0,0,2,1,1,0,0,0,1,1,1),
Y = c(1,1,2,1,2,2,1,0,0,0,0,2,0))
# Names of the columns
plot_colnames = c('X', 'Y', paste('cluster',unique(cluster_true),sep='_'))
# Empty dataframe with the right column names
plot_df = data.frame(matrix(vector(), x*y, length(plot_colnames),
dimnames=list(c(), plot_colnames)),
stringsAsFactors=F)
# Each row belongs to a certain 2D coordinate
plot_df$X = rep(1:x, y)-1
plot_df$Y = rep(1:x, each = y)-1
# This is what I don't know how to improve
for(i in 1:nrow(plot_df)){
idx = which(apply(data[,1:2], 1, function(x) all(x == plot_df[i,1:2])))
plot_df[i,3] = sum(cluster_true[idx] == 1)
plot_df[i,4] = sum(cluster_true[idx] == 2)
}
print(plot_df)
Things I need to change and I don't know how to:
I think the loop could be avoided in order to get a more elegant solution, but I don't know how. The dataframe data could have a very large amount of rows, so efficient code would be awesome.
Inside the loop, I've hardcoded the clusters to check (the last two lines inside the loop assume that I know which numbers are present in cluster_true and to which column of plot_df they correspond to). In fact, the elements in cluster_true could be anything, even non-consecutive numbers (i.e. cluster_true = c(1,5,5,5,56,10,19,10)).
So basically, I want to know if this could be done without the loop and as generic as possible.

Answer 1

If I understand correctly, the OP wants to
find the row indices for all unique combinations of X, Y coordinates in data,
look up the value in the corresponding rows of cluster_true,
count the number of occurrences of each value for the given X, Y combination, and
print the results in wide format.
This can be solved by joining and reshaping:
library(data.table) # version 1.11.4 used
library(magrittr) # use piping to improve readability
# unique coordinate pairs
uni_coords <- unique(setDT(data)[, .(X, Y)])[order(X, Y)]
# join and lookup values in cluster_true
data[uni_coords, on = .(X, Y), cluster_true[.I], by = .EACHI] %>%
# reshape from long to wide format, thereby counting occurrences
dcast(X + Y ~ sprintf("cluster_%02i", V1), length)
X Y cluster_01 cluster_02
1: 1 1 2 1
2: 1 2 1 1
3: 1 3 1 1
4: 2 2 0 1
5: 3 1 1 0
6: 3 2 1 0
7: 3 3 0 3
This is identical with OP's expected result except for the coordinate combinations which do not appear in data.
setDT(plot_df)[order(X, Y)]
X Y cluster_1 cluster_2
1: 1 1 2 1
2: 1 2 1 1
3: 1 3 1 1
4: 2 1 0 0
5: 2 2 0 1
6: 2 3 0 0
7: 3 1 1 0
8: 3 2 1 0
9: 3 3 0 3
Reshaping has the benefit that it can handle arbitrary values in cluster_true as requested by the OP.
Edit
The OP has requested that all possible combinations of X, Y coordinates should be included in the final result. This can be achieved by using a cross join CJ() to compute uni_coords:
# all possible coordinate pairs
uni_coords <- setDT(data)[, CJ(X = X, Y = Y, unique = TRUE)]
# join and lookup values in cluster_true
data[uni_coords, on = .(X, Y), cluster_true[.I], by = .EACHI][
uni_coords, on = .(X, Y)] %>%
# reshape from long to wide format, thereby counting occurrences
dcast(X + Y ~ sprintf("cluster_%02i", V1), length) %>%
# remove NA column from reshaped result
.[, cluster_NA := NULL] %>%
print()
X Y cluster_01 cluster_02
1: 1 1 2 1
2: 1 2 1 1
3: 1 3 1 1
4: 2 1 0 0
5: 2 2 0 1
6: 2 3 0 0
7: 3 1 1 0
8: 3 2 1 0
9: 3 3 0 3