I have a huge 3D matrix and there is a loop. In every iteration, I would like to extract some different parts (for example: 10000) out of it, then the convolution between those part and a patch is calculated.
I know it could easily be done using a loop but it is very time consuming.
Is there any alternative solution to work much faster than loop?
Let's suppose you have :
1) A row vector idx containing the row indexes of the top left corners of your parts.
2) A row vector idy containing the column indexes of the top left corners of your parts.
3) A row vector idz containing the indexes along the 3rd coordinate of the left corners of your parts.
We'll first have to create, from idx, idy and idz 3 vector containing ALL the indexes of the elements you need to extract from your matrix. Then we'll split the extracted matrix in blocks the same size of your patch using mat2cell, and then we'll apply the convn function to each block using cellfun.
Totidx=bsxfun(#plus,idx,[0:(size(patch,1)-1)]'); \\i-th column of this is the column vector idx(i):(idx(i)+size(patch,1)-1)
Totidx=reshape(Totidx,1,numel(Totidx)); \\ Creates the vector needed containing all indexes along first dimension.
Doing the same for idy and idz, we obtain 3 vectors Totidx, Totidy, Totidz containing all indexes needed.
Now we can extract the values from your initial matrix, say A :
ExtractedA=A(Totidx,Totidy,Totidz);
Apply mat2cell : NPatch denotes your number of extracted patches
B=mat2cell(ExtractedA,size(patch,1)*ones(1,NPatch),size(patch,2)*ones(1,NPatch),size(patch,3)*ones(1,NPatch));
Then you can apply your convn function to every cell of the cell array B : patch denotes the patch you want to convolute your extracted parts with
fun=#(M) convn(M,patch,'valid');
out=cellfun(fun,B,'uniformoutput',false);
Every cell of the cell array out is now one of the output you wanted
Related
I have an extension I am getting the data from, and I am referring to that extension in a formula with result of pre determined rows but I want more than 1 row in its result and that's when I have this problem, because I want to drag the formula to multiple rows but they overlap each other, for example if I wanted 3 rows in the result of the formula starting in row 1 and then drag it down from row 1 to row 3 the formula in row 1 and 2 will show an error because they're overlapped in each other I will put a picture in how it looks...
Is there a way to specify amount of rows as a space between each formula in a way that when I drag the formula down to more rows it will adjust to the "space" I specified?
This is the formula I am using, I am also referring to another sheet as you can see so it'll be great if you can use this formula to answer my question, if I can specify the "space" using another formula that is (also it's probably obvious but the pre determined rows in the formula is the "2d").
=CRYPTOFINANCE("KRAKEN:"&'crypto-track'!C4&"/USD", "price_history", "2d")
this is usually solved by constructing an array of formulae where you stack them up in the line like:
={CRYPTOFINANCE("KRAKEN:"&'crypto-track'!C4&"/USD", "price_history", "2d");
CRYPTOFINANCE("KRAKEN:"&'crypto-track'!C5&"/USD", "price_history", "2d");
CRYPTOFINANCE("KRAKEN:"&'crypto-track'!C6&"/USD", "price_history", "2d")}
this way the 2nd fx will pick up right after 1 fx ends
you can ease your pain of a "hand job" from constructing such an array - especially if that array needs to span over the larger range - by building a formula to generate a formula. for example: https://stackoverflow.com/a/68278101/5632629
also, make sure you obey the law of array constructs and successfully avoid all array errors - https://stackoverflow.com/a/58042211/5632629
In Google Sheets, running an array to transpose raw data into a readable format. The array forumula runs off of a row of data and am using the array to transpose the data into multiple rows. The issue is that for every additional row of raw data, I create five new rows of transposed data so the array formula breaks. Trying to make the array formula flex to added rows of data - any thought would be appreciated!
https://docs.google.com/spreadsheets/d/16GOsH-EUDm2IeRgUgEQ8LBEltQASfJ8hR6GXh-pqNcM/edit?usp=sharing
Rather than use an array formula, you can use an indirect formula and modulo arithmetic and integer division to do this. In k2 put the formula =indirect("A"&(3+FLOOR((row()-2)/5))) which will get the date from column A row 3 until you get down to row 7, when it starts taking from row 4. This formula can be dragged down and will jump rows every 5 as desired.
Similarly for L2 place in =indirect("B"&(3+FLOOR((row()-2)/5))) which can also be dragged down (copied) the column.
For column m we need to cycle through c2,d2,e2,f2, and g2, so we need modulo (clock) arithmentic. So in m2 place the formula =indirect(char(CODE("C")+(mod(ROW()-2,5)))&"2"). Finally for quantity we need to do both the column cycling and the incrementing rows every 5, so the formula for n2 is =indirect(char(CODE("C")+(mod(ROW()-2,5)))&(3+FLOOR((row()-2)/5))). That too can get copied on down. That should do it. There could definitely be more elegant ways.
I think not more elegant, but preserving the array formula would be =3+floor((row()-2)/5) in j2 and drag on down, and then adapt the array formula to: ={INDIRECT("A" & J2),INDIRECT("B"&J2),$C$2,INDIRECT("C"&J2);INDIRECT("A" & J2),INDIRECT("B"&J2),$D$2,INDIRECT("D"&J2);INDIRECT("A" & J2),INDIRECT("B"&J2),$E$2,INDIRECT("E"&J2);INDIRECT("A" & J2),INDIRECT("B"&J2),$F$2,INDIRECT("F"&J2);INDIRECT("A" & J2),INDIRECT("B"&J2),$G$2,INDIRECT("G"&J2)} which can be copied to 5 rows below it where the J2's become J7's and you are processing the next desired row.
I have approx. 100 rows by 60 columns with numbers which I'd need to be arranged in (same) bins categories. (12 bins)
In a new sheet, I used the frequency formula to look at the first row and return in an array in a column the distribution in bins. It worked for the first row of the source data but now I'd like that when I drag the formula to the next column, the data array in the frequency formula to move to the next row of the data source and return the distribution by bins.
Is it possible, can you please help with this one? Or is there any other way I can do such a bin arrangement? Need frequencies for A, B,C etc..and would like to drag the formula to the right, if that is possible.
partial data-distributed horizontally
Use the following formula for Column U:
=FREQUENCY(INDIRECT(ADDRESS(COLUMN()-19,3) & ":" & ADDRESS(COLUMN()-19,18)),$T$3:$T$17)
and drag/copy across as required. For details on INDIRECT function see this.
Note: INDIRECT is a volatile function and hence is recalculated every time anything changes in the worksheet.
See image for results based on data provided by you.
Find the max size of rectangular contiguous submatrix of unique (i.e. non repeated within a given submatrix) element.
How can I solve this?
You should set a maximum value to 0. Iterate the rows of the matrix and if they are not repeating (whatever that means), compare its size to the maximum. If it is bigger, then store the new maximum value and use that for further iterations. In case you found a new maximum, store whatever you need to store. So, the algorithm looks like this:
maximum <- 0
for all rows as row
if (row is not repeating) then
if (row rectangle size > maximum) then
maximum <- new maximum
store whatever you need to store
end if
end if
end for
Note, that if you do not have further information, then it is pointless to do a binary search, since you will have to check the size of each rectangle. If you have further knowledge about your rectangles, then the algorithm might be optimized.
A first idea (recursion): Maybe identify pairs in the whole array, this will identify constraints to respect. If there is a value v at both positions x0,y0 and x1,y1 then you cannot have a rectangle containing these positions, so this will let you construct some possible rectangles from these values and recurse on them?
Another one (dynamic programming): start with elementary arrays (size 1x1) and try to merge them respecting the constraint?
I am tracking particles into a 3D lattice. Each lattice element is labeled with an index corresponding to an unrolled 3D array
S = x + WIDTH * (y + DEPTH * z)
I am interested in the transition form cell S1 to cell S2. The resulting transition matrix M(S1,S2) is sparsely populated, because particles can reach only near by cells. Unfortunately using the indexing of an unrolled 3D array cells that are geometrically near might have big difference in their indexes. For instance, cells that are siting on top of each other (say at z and z+1) will have their indexes shifted by WIDTH*DEPTH. Therefore if I try accumulating the resulting 2D matrix M(S1,S2) , S1 and S2 will be very different, even dough the cells are adjacent. This is a significant problem, because I can't use the usual sparse matrix storage.
At the beginning I tried storing the matrix in coordinate format:
I , J VALUE
Unfortunately I need to loop the entire index set to find the proper S1,S2 and store the accumulated M(S1,S2).
Unusually sparse matrices have some underlying structure and therefore the indexing is quite straightforward. In this case however, I have some troubles figuring out how to index my cells.
I would appreciate your help
Thank you in advance,
There are several approaches. Which is best depends on operations that need to be performed on the matrix.
A good general purpose one is to use a hash table where the key is the index tuple, in your case (i,j).
If neighboring (in the Euclidean sense) matrix elements must be discoverable, then an alternate strategy is a balanced tree with a Morton Order key. The Morton order value of a key (i,j) is just the integers i and j with their bits interleaved. You should quickly see that index tuples close to each other in the index 2-space are also close in linear Morton order.
Of course if you are building the matrix all at once, after which it's immutable, then you can build the key-value pairs in an array rather than a hash table or balanced tree, sort them (lexicographically for (i,j) pairs and linearly for Morton keys) and then do reads with simple binary search.