I have a function which is executed 200 times like this
for (l in 1:200) {
fun.ction(paramter1=g, paramter2=h)$element->u[z,,]
}
u is an array:
u<-array(NA, dim=c(2000,150,7))
of which I know it should have the right format. The element of func.tion is also an array which has the same dimensions. Hence, is there some way to fill the array u in each of the 200 runs with the array resulting from fun.ction()$element? I tried to use indexing via a list (u[[z]]). It saves the array but as a list so that I can't access the elements afterwards which I would need to. I appreciate any help.
I am not sure what it is that you want, but if you just want to store 200 arrays of dimensions (2000,150,7) you can just make another array with a fourth dimension of 200.
storage.array <- array(dim=c(2000,150,7, 200))
And then store your (2000, 150, 7) arrays in the fourth dimension:
for (i in 1:200){
storage.array[,,,i] <-
fun.ction(paramter1=g, paramter2=h)$element}
Then you can access each of the ith array by:
storage.array[,,,i]
But I guess that will be too big an array for R to handle, at least it is in my computer.
An example that you can easily reproduce with smaller arrays:
storage.array <- array(dim=c(20,2,7, 200))
fun.ction <- function(parameter1, parameter2){
array(rnorm(140, parameter1, parameter2), dim=c(20,2,7))
}
for (i in 1:200){
storage.array[,,,i]<- fun.ction(10, 10)
}
But as Roland and Thomas have said, you should make your code reproducible and define correctly what you want, so it is easier to answer without trying to guess what your problem is.
Best regards
Related
I have large 1D arrays a and b, and an array of pointers I that separates them into subarrays. My a and b barely fit into RAM and are of different dtypes (one contains UInt32s, the other Rational{Int64}s), so I don’t want to join them into a 2D array, to avoid changing dtypes.
For each i in I[2:end], I wish to sort the subarray a[I[i-1],I[i]-1] and apply the same permutation to the corresponding subarray b[I[i-1],I[i]-1]. My attempt at this is:
function sort!(a,b)
p=sortperm(a);
a[:], b[:] = a[p], b[p]
end
Threads.#threads for i in I[2:end]
sort!( a[I[i-1], I[i]-1], b[I[i-1], I[i]-1] )
end
However, already on a small example, I see that sort! does not alter the view of a subarray:
a, b = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(a,b); println(a,"\n",b) # works like it should
a, b = rand(1:10,10), rand(-1000:1000,10) .//1
sort!(a[1:5],b[1:5]); println(a,"\n",b) # does nothing!!!
Any help on how to create such function sort! (as efficient as possible) are welcome.
Background: I am dealing with data coming from sparse arrays:
using SparseArrays
n=10^6; x=sprand(n,n,1000/n); #random matrix with 1000 entries per column on average
x = SparseMatrixCSC(n,n,x.colptr,x.rowval,rand(-99:99,nnz(x)).//1); #chnging entries to rationals
U = randperm(n) #permutation of rows of matrix x
a, b, I = U[x.rowval], x.nzval, x.colptr;
Thus these a,b,I serve as good examples to my posted problem. What I am trying to do is sort the row indices (and corresponding matrix values) of entries in each column.
Note: I already asked this question on Julia discourse here, but received no replies nor comments. If I can improve on the quality of the question, don't hesitate to tell me.
The problem is that a[1:5] is not a view, it's just a copy. instead make the view like
function sort!(a,b)
p=sortperm(a);
a[:], b[:] = a[p], b[p]
end
Threads.#threads for i in I[2:end]
sort!(view(a, I[i-1]:I[i]-1), view(b, I[i-1]:I[i]-1))
end
is what you are looking for
ps.
the #view a[2:3], #view(a[2:3]) or the #views macro can help making thins more readable.
First of all, you shouldn't redefine Base.sort! like this. Now, sort! will shadow Base.sort! and you'll get errors if you call sort!(a).
Also, a[I[i-1], I[i]-1] and b[I[i-1], I[i]-1] are not slices, they are just single elements, so nothing should happen if you sort them either with views or not. And sorting arrays in a moving-window way like this is not correct.
What you want to do here, since your vectors are huge, is call p = partialsortperm(a[i:end], i:i+block_size-1) repeatedly in a loop, choosing a block_size that fits into memory, and modify both a and b according to p, then continue to the remaining part of a and find next p and repeat until nothing remains in a to be sorted. I'll leave the implementation as an exercise for you, but you can come back if you get stuck on something.
I'm trying to work with arrays, but I can't seem to make one that works for my data. I have 14 matrices I would like to put in an array, but I can't figure out the way to do it without manually writing c(m1,m2,m3...) to put in all of them
this is what i tried:
m_list <- mget(paste0("well_", 0:13)) ###to make a list of all my matrices
a <- array(c(m_list),
dim = c(7338, 15, 14))
but when I try to look at the array I created something is not right with it cause I try to call for one value, like this:
print(a[1,4,2])
but I get entire columns.
I assume the error is in the list of matrices. Please help
An answer to your question is that you should use do.call(c, m_list) instead of c(m_list). (Take a couple of small matrices and try to see what c(m_list) and c(m1, m2) return.)
Also you might want to think some more whether working with an array is better than working with a list and, more importantly, how you could avoid having multiple matrices in the first place and instead to directly read/define them as a list or an array.
You can simply use unlist inside your array function call instead of c.
a = array(unlist(m_list), dim = c(dim(m_list[[1]]), length(m_list)))
Some reproducible data:
m1 = matrix(1:5, 5, 5)
m2 = matrix(5:1, 5, 5)
m_list = list(m1, m2)
I'm quite new to MatLab and this problem really drives me insane:
I have a huge array of 2 column and about 31,000 rows. One of the two columns depicts a spatial coordinate on a grid the other one a dependent parameter. What I want to do is the following:
I. I need to split the array into smaller parts defined by the spatial column; let's say the spatial coordinate are ranging from 0 to 500 - I now want arrays that give me the two column values for spatial coordinate 0-10, then 10-20 and so on. This would result in 50 arrays of unequal size that cover a spatial range from 0 to 500.
II. Secondly, I would need to calculate the average values of the resulting columns of every single array so that I obtain per array one 2-dimensional point.
III. Thirdly, I could plot these points and I would be super happy.
Sadly, I'm super confused since I miserably fail at step I. - Maybe there is even an easier way than to split the giant array in so many small arrays - who knows..
I would be really really happy for any suggestion.
Thank you,
Arne
First of all, since you wish a data structure of array of different size you will need to place them in a cell array so you could try something like this:
res = arrayfun(#(x)arr(arr(:,1)==x,:), unique(arr(:,1)), 'UniformOutput', 0);
The previous code return a cell array with the array splitted according its first column with #(x)arr(arr(:,1)==x,:) you are doing a function on x and arrayfun(function, ..., 'UniformOutput', 0) applies function to each element in the following arguments (taken a single value of each argument to evaluate the function) but you must notice that arr must be numeric so if not you should map your values to numeric values or use another way to select this values.
In the same way you could do
uo = 'UniformOutput';
res = arrayfun(#(x){arr(arr(:,1)==x,:), mean(arr(arr(:,1)==x,2))), unique(arr(:,1)), uo, 0);
You will probably want to flat the returning value, check the function cat, you could do:
res = cat(1,res{:})
Plot your data depends on their format, so I can't help if i don't know how the data are, but you could try to plot inside a loop over your 'res' variable or something similar.
Step I indeed comes with some difficulties. Once these are solved, I guess steps II and III can easily be solved. Let me make some suggestions for step I:
You first define the maximum value (maxValue = 500;) and the step size (stepSize = 10;). Now it is possible to iterate through all steps and create your new vectors.
for k=1:maxValue/stepSize
...
end
As every resulting array will have different dimensions, I suggest you save the vectors in a cell array:
Y = cell(maxValue/stepSize,1);
Use the find function to find the rows of the entries for each matrix. At each step k, the range of values of interest will be (k-1)*stepSize to k*stepSize.
row = find( (k-1)*stepSize <= X(:,1) & X(:,1) < k*stepSize );
You can now create the matrix for a stepk by
Y{k,1} = X(row,:);
Putting everything together you should be able to create the cell array Y containing your matrices and continue with the other tasks. You could also save the average of each value range in a second column of the cell array Y:
Y{k,2} = mean( Y{k,1}(:,2) );
I hope this helps you with your task. Note that these are only suggestions and there may be different (maybe more appropriate) ways to handle this.
Hi I am stuck with one of these simple but time-consuming errors:
How can I populate an array with loops? I know I am on a C approach here
and R isn't C.
Data <-[SOMETHING HERE]
One <-200
Two <-100
array222 <- array(0,length(SomeLength))
for (i in 1:One)
{
for (j in 1:Two)
{
array222[i][j] = sample(Data,1)
}
I want to populate the array with random samples from another dataset but all
I get is this:
Warning in array222[i][j] = sample(Data, 1) :
number of items to replace is not a multiple of replacement length
First of all, you wouldn't use loops to do this in R. You'd just do
array222 <- matrix(sample(Data, One*Two, replace=T), nrow=One, ncol=Two)
But going back to your code, you fail to properly initialize your array222 variable. The matrix() syntax is probably easier for a 2-D array, but you could also use array(0, dim=c(One,Two)). You need to create it with the proper dimensions.
And additionally, the proper way to index a dimensional array is
array222[i,j] #NOT array222[i][j]
i want to use a 2D array to store all the values of img1, img2 and the compared vlaue of img1 and img2,
I want to achive the algorithm likes:
% read in the images from a folder one by one:
somefolder = 'folder';
filelist = dir([somefolder '/*.jpg']);
s=numel(filelist);
C = cell(length(filelist), 1);
for k=1:s
C{k}=imread([somefolder filelist(k).name]);
end
%choose any of the two images to compare
for t=1:(s-1)
for r=(t+1):s
img1=C{r};
img2=C{t};
ssim_value[num][1]=img1; % first img
ssim_value[num][2]=img2; % second img
ssim_value[num][3]=mssim; % ssim value of these two images
end
end
So, there is error about using the 2D array (ssim_value) that I used, what is the correct way of initialization it, and how to achieve the purpose that save the values that I want to store.
Could someone help me. Thanks in advance.
I'm assuming that "num" is a number that you will supply, like 5 or something. You cannot mix types in arrays as you do in Python. Also, as #Schorsch pointed out, you use parenthesis to index arrays in Matlab.
The two-dimensional array you are trying to form needs to be a 2-D cell array. For example:
a = {{"a",3},{"two",[1,2,3]};
In this case, a{1,2} = 3, and a{2,1} = "two".
You may not know in advance how many files are in the directory, so pre-initializing the cell array may not be possible in advance. In any case, Matlab arrays only need to be pre-initialized for performance reasons, and you can easily find information on initializing arrays in Matlab.
In light of this, I'm pretty sure what you are trying to accomplish is:
%choose any of the two images to compare
ssim_value = {};
for t=1:(s-1)
for r=(t+1):s
img1=C{r};
img2=C{t};
ssim_value{num,1}=img1; % first img
ssim_value{num,2}=img2; % second img
ssim_value{num,3}=mssim; % ssim value of these two images
end
end