Related
Let A and B be arrays, of dimension [3,4,5] and [4,5], respectively.
E.g.,
A <- array(100,c(3, 4,5))
B <- array(80, c(4,5))
My desired answer is an array C of dimension [3,4,5] such that
C[i,j,k] = A[i,j,k] - B[j,k]
for all i,j,k
Edit Which answer is fastest code?
To evaluate the following three answers, I executed the following code to quantify the time of the three codes.
The result is the following;
> mb
Unit: microseconds
expr min lq mean median uq max neval
f1 28.4 33.00 37.329 34.75 37.00 213.5 100
f2 32.5 37.65 40.069 38.95 40.55 103.0 100
f3 33.8 40.25 42.397 41.65 43.30 64.5 100
Thus the f1 is the most faster, thus I choose the answer of #user10488504 as an answer of this question.
Thank you, three persons #user10488504, #Stéphane Laurent and #Lyngbakr. I will use your suggesting code in my package. It helps me very much.
Code, which calculates running times.
f1 <- function(){
A <- array(1:100, c(3, 4, 5))
B <- array(1:80, c(4,5))
C <- array(aperm(sapply(1:dim(A)[1], function(i) A[i,,] - B)), dim(A))
}
f2<-function(){
A <- array(1:100, c(3, 4, 5))
B <- array(1:80, c(4,5))
sweep(A, c(2,3), B)
}
f3 <- function(){
A <- array(1:100, c(3, 4, 5))
B <- array(1:80, c(4,5))
# Perform calculation
res <- array(t(apply(A, MARGIN = 1, function(x)x-B)), c(3, 4, 5))
}
library(microbenchmark)
library(ggplot2)
mb <- microbenchmark(
f1 = f1(),
f2 = f2(),
f3 = f3()
)
mb
autoplot(mb)
With sweep:
A <- array(1:100, c(3, 4, 5))
B <- array(1:80, c(4,5))
> sweep(A, c(2,3), B)
, , 1
[,1] [,2] [,3] [,4]
[1,] 0 2 4 6
[2,] 1 3 5 7
[3,] 2 4 6 8
, , 2
[,1] [,2] [,3] [,4]
[1,] 8 10 12 14
[2,] 9 11 13 15
[3,] 10 12 14 16
, , 3
[,1] [,2] [,3] [,4]
[1,] 16 18 20 22
[2,] 17 19 21 23
[3,] 18 20 22 24
, , 4
[,1] [,2] [,3] [,4]
[1,] 24 26 28 30
[2,] 25 27 29 31
[3,] 26 28 30 32
, , 5
[,1] [,2] [,3] [,4]
[1,] 32 34 36 38
[2,] 33 35 37 39
[3,] 34 36 38 40
You can do this with sapply and aperm like:
C <- array(aperm(sapply(1:dim(A)[1], function(i) A[i,,] - B)), dim(A))
Here's an attempt that uses apply.
# Define arrays
A <- array(1:100, c(3, 4, 5))
B <- array(1:80, c(4,5))
# Perform calculation
res <- array(t(apply(A, MARGIN = 1, function(x)x-B)), c(3, 4, 5))
# Check result
res[1,,]
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 0 8 16 24 32
#> [2,] 2 10 18 26 34
#> [3,] 4 12 20 28 36
#> [4,] 6 14 22 30 38
A[1,,] - B
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 0 8 16 24 32
#> [2,] 2 10 18 26 34
#> [3,] 4 12 20 28 36
#> [4,] 6 14 22 30 38
Created on 2019-06-19 by the reprex package (v0.3.0)
I have the array A
A
,,A
[,1] [,2] [,3]
[1,] 3 7 8
[2,] 4 11 9
[3,] 2 12 4.3
,,B
[,1] [,2] [,3]
[1,] 31 7 8
[2,] 4.2 4 9.5
[3,] 1 1 7
,,C
[,1] [,2] [,3]
[1,] 4 71 8.3
[2,] 4 41 9
[3,] 11 0 73
,,D
[,1] [,2] [,3]
[1,] 7 7 8.3
[2,] 3 4.1 9
[3,] 1 0.5 73
dim(A)
3 3 4
dimnames(A)[3]
A B C D
and I have the data.frame df
df
X Y Z
2 1 A
3 2 D
I would like to put in a new column of df, the values of array A, based on df index-values X(row for the array), Y(column for the array) and third dimension Z, let's say, my aspect result is:
df
X Y Z Res
2 1 A 4 # Res is the value of array A in A[2,1,"A"]
3 2 D 0.5 # Res is the value of array A in A[3,2,"D"]
I tried this code:
df$Res <- NA
if (df$Z == dimnames(A)[3]){
for (i in 1:nrow(df)){
df[i,4] <- A[df[i,1],df[i,2],df[i,3]]
}
}
But it'doesn't work well...
Any idea to associate the dimnames of third dimension array and data frame index-value?
P.S. This is a simple example. My true array is:
dim(A)
137 93 227
and
dim(df)
6080 3
P.S.2 I prefer to don't use merge or other type of similar code for allocation problem
For each of my N variables, I have a (T * M) feature matrix, i.e., M observations per t \in T. The problem is how to convert this into a (T * N * M) array. For example, in the following example N=2, T=3, M=4 :
x1 <- matrix(1:24, 3,4)
> x1
[,1] [,2] [,3] [,4]
[1,] 1 4 7 10
[2,] 2 5 8 11
[3,] 3 6 9 12
x2 <- matrix(25:48, 3,4)
x2
[,1] [,2] [,3] [,4]
[1,] 25 28 31 34
[2,] 26 29 32 35
[3,] 27 30 33 36
And I need to make a 3 dimensional (number of rows) array, such that the first element is
[,1] [,2] [,3] [,4]
[1,] 1 4 7 10
[2,] 25 28 31 34
and the second is:
[,1] [,2] [,3] [,4]
[1,] 2 5 8 11
[2,] 26 29 32 35
and third:
[,1] [,2] [,3] [,4]
[1,] 3 6 9 12
[2,] 27 30 33 36
and so on and so forth. For the following example, the output's dimensions should be (3,2,4).
I need to do this for relatively large N and T, so appreciate extendable implementations!
Here is a base R option.
out <- `dim<-`(rbind(c(t(x1)), c(t(x2))), c(2, 4, 3))
out
#, , 1
#
# [,1] [,2] [,3] [,4]
#[1,] 1 4 7 10
#[2,] 25 28 31 34
#
#, , 2
#
# [,1] [,2] [,3] [,4]
#[1,] 2 5 8 11
#[2,] 26 29 32 35
#
#, , 3
#
# [,1] [,2] [,3] [,4]
#[1,] 3 6 9 12
#[2,] 27 30 33 36
When we call x <- rbind(c(t(x1)), c(t(x2))) we get the following matrix as a result
x
# [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
#[1,] 1 4 7 10 2 5 8 11 3 6 9 12
#[2,] 25 28 31 34 26 29 32 35 27 30 33 36
We need to change the dimensions of this object for which we can do
dim(x) <- c(2, 4, 3)
Another way to get the same result is to call the replacement method of dim in its functional form, i.e.
`dim<-`(...)
Which allows us to do all in one line.
I need to create a 3D array sorted by row, from left to right and descendent.
x <- 100
I have tried with this:
b <- array(1:96, dim= c(8,4,3))
but it sorts firstly descendently. Using apperm(b) doesn't work as well
The result I want is this:
, , 1
1 2 3 4 5
6 7 8 9 10
11 12 13 14
15 16 17 18
19 20 21 22
array by default fill values along 1st dimension, then 2nd dimension, then 3rd; What you are looking for is fill it in the order of (2nd, 1st, 3rd), you can initialize the array with the shape of 1st dimension and 2nd dimension switched and then use aperm on it:
b <- aperm(array(1:96, dim= c(4,8,3)), c(2,1,3))
# ^ ^ ^ ^ switch the dimension twice here
b
, , 1
[,1] [,2] [,3] [,4]
[1,] 1 2 3 4
[2,] 5 6 7 8
[3,] 9 10 11 12
Edit: My first try, but #Psidom's answer is the right way to do this.
You need to make it as a combination of 3 matrices and then combine them into an array. In the code below I used 96*i/3 to make it flexible for more than 3 matrices to be combined.
b <- array( c( aperm(array(1:(96*1/3), dim = c(4,8))),
aperm(array(33:(96*2/3), dim = c(4,8))),
aperm(array(65:(96*3/3), dim = c(4,8))) ) ,
dim = c(8, 4, 3))
This will be the output:
b[, , 1]
# [,1] [,2] [,3] [,4]
# [1,] 1 2 3 4
# [2,] 5 6 7 8
# [3,] 9 10 11 12
# [4,] 13 14 15 16
# [5,] 17 18 19 20
# [6,] 21 22 23 24
# [7,] 25 26 27 28
# [8,] 29 30 31 32
at first i have a matrix like this :
x <- matrix(rnorm(1e3),260)
and then an Array
lst <- lapply(seq(1,length(x[,1]), by=52), function(i) x[i:(i+51),])
Data_array <- array(unlist(lst), dim=c(52,length(x[1,]),(length(x[,1])/52)))
This array is a sequence of the Dataframe by 52 (weeks).
It's a temporal analysis (weekly)
I would like to compute an ecdf function on this array.
, , 1
[,1] [,2] [,3]
[1,] **0.66319631** 0.01004290 0.02133477
[2,] -1.64273648 0.23105503 1.02862145
[3,] 1.17083363 -0.49700717 -0.01119745
, , 2
[,1] [,2] [,3]
[1,] **-0.79365987** 1.28394049 -0.547763434
[2,] -0.09221301 1.07676841 0.570294731
[3,] 0.20293308 1.00182888 0.247373981
, , 3
[,1] [,2] [,3]
[1,] **1.03862172** -0.961678683 1.25334651
[2,] 0.58476540 0.745250484 -0.06183788
[3,] 0.24057690 1.226575038 0.23363005
compute ecdf function for each cell. It's for a weekly seasonal analysis.
i.e. calcul quantile for this time series (**): 0.66319631;-0.79365987;1.03862172
for MEAN it's works :
array_lag_sum<-apply(Data_array,c(1,2),FUN=function(x){mean(x,na.rm=TRUE)})
i tried a similar function whith ecdf, but it doesn't work.
percent_array<-apply(Data_array,c(1,2),FUN=function(u){ecdf(u)(u)})
Then...it is not finish, i would like to reformat this array like the original format of the data dataframe (x). (like a rbind but on an array.)
Thank you so much for your help.
edit :
sorry, but i don't know if i was so clear. It's sur that array is complicated for me;
but with your method, if i have this simple data frame :
B <- matrix(seq(1,20), 20, 3)
> B
[,1] [,2] [,3]
[1,] 1 1 1
[2,] 2 2 2
[3,] 3 3 3
[4,] 4 4 4
[5,] 5 5 5
[6,] 6 6 6
[7,] 7 7 7
[8,] 8 8 8
[9,] 9 9 9
[10,] 10 10 10
[11,] 11 11 11
[12,] 12 12 12
[13,] 13 13 13
[14,] 14 14 14
[15,] 15 15 15
[16,] 16 16 16
[17,] 17 17 17
[18,] 18 18 18
[19,] 19 19 19
[20,] 20 20 20
Your function gives :
Data_array <- array( B, dim=c(10,3,5))
, , 1
[,1] [,2] [,3]
[1,] 1 11 1
[2,] 2 12 2
[3,] 3 13 3
[4,] 4 14 4
[5,] 5 15 5
[6,] 6 16 6
[7,] 7 17 7
[8,] 8 18 8
[9,] 9 19 9
[10,] 10 20 10
, , 2
[,1] [,2] [,3]
[1,] 11 1 11
[2,] 12 2 12
[3,] 13 3 13
[4,] 14 4 14
[5,] 15 5 15
[6,] 16 6 16
[7,] 17 7 17
[8,] 18 8 18
[9,] 19 9 19
[10,] 20 10 20
or i would more something like this :
,,1
[,1] [,2] [,3]
[1,] 1 1 1
[2,] 2 2 2
[3,] 3 3 3
[4,] 4 4 4
[5,] 5 5 5
[6,] 6 6 6
[7,] 7 7 7
[8,] 8 8 8
[9,] 9 9 9
[10,] 10 10 10
,,2
[,1] [,2] [,3]
[1,] 11 11 11
[2,] 12 12 12
[3,] 13 13 13
[4,] 14 14 14
[5,] 15 15 15
[6,] 16 16 16
[7,] 17 17 17
[8,] 18 18 18
[9,] 19 19 19
[10,] 20 20 20
and get in result a table which is the percentile value of the time series.
percentile values of 1 and 11, 2 and 12 for each column and each row (i know it's not pertinent but it's just for exemple)
Sorry if my last question was not understandable
The answer is:
ecdf_mat <- apply( Data_array, 1:2, ecdf)
This passes values from each combination of the first two indices to the the function, ecdf. Each of those passes will return a function into a matrix location. You are getting something most people will not be able to use without a bit of coaching: one 52 x 4 matrix of functions. The functions are contained in lists which are valid matrix or array elements:
> dim(apply( Data_array, 1:2, ecdf) )
[1] 52 4
To access them you need to first pull them out of the matrix with standard "[" indexing but then pull them out of the list container with a call to "[[1]]":
> str(apply( Data_array, 1:2, ecdf)[1,1] )
List of 1
$ :function (v)
..- attr(*, "class")= chr [1:3] "ecdf" "stepfun" "function"
..- attr(*, "call")= language FUN(newX[, i], ...)
> apply( Data_array, 1:2, ecdf)[1,1][[1]]
Empirical CDF
Call: FUN(newX[, i], ...)
x[1:5] = -0.92217, -0.37471, 0.058284, 0.28502, 0.44391
> apply( Data_array, 1:2, ecdf)[1,1][[1]](0)
[1] 0.4
Edit:------
It appears you don't want the ecdf's themselves (despite getting no response to my efforts at getting you to recognize the distinction), but rather want an identically shaped array with the percentile values for the i-j positions considered as individual length k-sequences. I can think of two ways to do this. The first one would use that matrix of ecdf functions I built and demonstrated, but I believe that is the more baroque method and it would be easier to give you a more direct route. I've take the liberty of making this more manageable by making the long first dimension only 10-long.
x <- matrix(rnorm(1e3),260)
lst <- lapply(seq(1,length(x[,1]), by=10), function(i) x[i:(i+51),])
Data_array <- array(unlist(lst), dim=c(10,length(x[1,]),(length(x[,1])/52
pctiles2 <- apply( Data_array, 1:2, function(x) ecdf(x)(x) )
> str(pctiles2)
num [1:5, 1:10, 1:4] 0.8 0.4 0.6 0.2 1 0.4 1 0.2 0.6 0.8 ...
They aren't actually percentiles, but that could be easily remedied by slipping a 100* in from of the ecdf call (or multiplying the result by 100.. You will notice that the structure has been permuted so that the quantile/percentiles sequences run down the first column. That because apply always delivers its result in column major order. There is a function aperm which would allow you to re-arrange these in the original order:
re_pctiles <- aperm(pctiles, c(2,3,1) )