how to copy a jagged array in awkward-array - awkward-array

In awkward0 I would like to separately persist various selections of a table
in pseudo code
X = awkward.Table(...)
one_jet = X[X.n_jet == 1]
two_jet = X[X.n_jet == 1]
but I notices the contents of any indexed jagged array doesn't change (only the starts stops are thinned), and so naively saving starts, stops this way would lear to a duplication of data on disk. Is there a way to "repack" jagged array so they become dense again?

I recently ran into this issue as well. The cleanest solution I found was to construct a new awkward array from the masked array, then save the new array.
X = awkward.Table(...)
one_jet = awkward.fromiter(X[X.n_jet == 1])
two_jet = awkward.fromiter(X[X.n_jet == 2]) # I assume you meant 2 here?


Array of sets in Matlab

Is there a way to create an array of sets in Matlab.
Eg: I have:
a = ones(10,1);
b = zeros(10,1);
I need c such that c = [(1,0); (1,0); ...], i.e. each set in c has first element from a and 2nd element from b with the corresponding index.
Also is there some way I can check if an unknown set (x,y) is in c.
Can you all please help me out? I am a Matlab noob. Thanks!
There are not sets in your understanding in MATLAB (I assume that you are thinking of tuples on Python...) But there are cells in MATLAB. That is a data type that can store pretty much anything (you may think of pointers if you are familiar with the concept). It is indicated by using { }.
Knowing this, you could come up with a cell of arrays and check them using cellfun
% create a cell of numeric arrays
C = {[1,0],[0,2],[99,-1]}
% check which input is equal to the array [1,0]
lg = cellfun(#(x)isequal(x,[1,0]),C)
Note that you access the address of a cell with () and the content of a cell with {}. [] always indicate arrays of something. We come to this in a moment.
OK, this was the part that you asked for; now there is a bonus:
That you use the term set makes me feel that they always have the same size. So why not create an array of arrays (or better an array of vectors, which is a matrix) and check this matrix column-wise?
% array of vectors (there is a way with less brackets but this way is clearer):
M = [[1;0],[0;2],[99;-1]]
% check at which column (direction ",1") all rows are equal to the proposed vector:
lg = all(M == [0;2],1)
This way is a bit clearer, better in terms of memory and faster.
Note that both variables lg are arrays of logicals. You can use them directly to index the original variable, i.e. M(:,lg) and C{lg} returns the set that you are looking for.
If you would like to get logical value regarding if p is in C, maybe you can try the code below
C = [1,2;
p1 = [1,2];
p2 = [1,-2];
>> any(sum((C-p1).^2,2)==0) # indicating p1 is in C
ans = 1
>> any(sum((C-p2).^2,2)==0) # indicating p2 is not in C
ans = 0
>> any(all(C-p1==0,2))
ans = 1
>> any(all(C-p2==0,2))
ans = 0

How would one access each element in a mulidimensional array without using #flatten?

I already flunked an assessment a little while ago but this question is still bugging me. The idea is to add all the elements of a multidimensional array like this: [1,[2,3],4,[5,6,[7]]]] and add them up as if they were all one-line, like 1+2+3+4+5+6+7, without being able to use #flatten. I got about as far as removing all the integers (like 1 and 4) and adding them into its own variable and then trying to reduce(:+) the 2nd level arrays( [2,3], etc), but I can't figure out how to write a loop that will delve deeper into the dimensions, like the [7]. How would one do that?
Code I've tried thus far
def multi_array_sum(arrays)
#need to add up all array elements
num = 0
arr = {}
arrays.each {|int| num = num + int if int.class == Fixnum; arrays.delete(int)} do |numz|
num = num + numz.reduce(:+)

Accessing data in a nested structure

Okay so I've found a better way of accessing my files but I'm still a bit stuck.
My code so far:
clc % clear window
clear %clear workspace
numfiles = 21;
data = cell(1, numfiles);
obsdata = dir('*.mat');
numfiles = length(obsdata);
data = cell(1, numfiles);
for k = 1:numfiles
data{k} = load(obsdata(k).name);
This sorts my data out.
There are 21 cells that contains the J6 - files as shown (the list of the files can be seen on the left):
Clicking each cell brings me to a structure:
Each of which contains data that I want to access.
I'm unsure as to how to go about writing my code so that I can store the data in the last part into two arrays (wavelength and intensity)
Try this. I used deal to copy out each of the's into a cell array called names. (It's probably a poor choice of name for a variable since you already have one called name, but anyway...)
obsdata = dir('*.mat');
numfiles = length(obsdata);
data = cell(1, numfiles);
names = cell(1,numfiles);
names = cell(1,numfiles);
[names{:}] = deal(;
for k = 1:numfiles
data{k} = load(names{k})
It'd help to hear a little bit more about how you want to store the data, but generally the work will be done inside the loop you use to load the files. A good starting point might be to get the data out of structs and into cell arrays:
for k = 1:numfiles
data{k} = struct2cell(load(obsdata(k).name));
I believe (it's been a while, and I don't have access to matlab anymore) that each of the 21 cells will now contain a cell array, storing the matrices you're interested in. Perhaps this is enough? From this point I think you can access data like this:
where x and y are the indices within the matrices that used to be stored as fields in a struct.
If you want to concatenate each of these matrices so that you get 21 cells, each with a single mx2 matrix, you can modify the loop:
for k = 1:numfiles
tmp = load(obsdata(k).name);
data{k} = vertcat(tmp{:});
With more information about how you want to structure the data we can refine the answer.

Is there a more efficient way of choosing an element from 1 array and to every element of the same array

I want to, for every element of an array ZAbs, compare it for equality to every element of the array itself and put them into another distinct array. I want the distinct array's elements to have the same index as the ZAbs array.
I did this by creating 4 nested for loops:
for pAbs2 = 1:400
for qAbs2 = 1:300
zAbsCompare = ZAbs(qAbs2, pAbs2);
for pAbs3 = 1:400
for qAbs3 = 1:300
zAbsCompare2 = ZAbs(qAbs3, pAbs3);
if (zAbsCompare == zAbsCompare2)
InitialZModEqualsImag(pAbs2,qAbs2) = InitialZImag(qAbs2, pAbs2);
InitialZModEqualsReal(pAbs2,qAbs2) = InitialZReal(qAbs2, pAbs2);
However, this runs really slowly. I can't think of a better way to do this, but as I'm inexperienced with MATLAB there's probably something I'm overlooking here. Any help?
EDIT: Fixed an error and restated the question.
You can do the comparison (not sure that's what you want) efficently with bsxfun:
comp = bsxfun(#eq, X, shiftdim(X,-2));
The result comp(m,n,p,q) is 1 if X(m,n) == X(p,q), and 0 otherwise.

How to set the fields simultaneously in a large structure array in Matlab?

I have a large structure array.
I would like to perform a sensitivity analysis on a function that processes this array.
So, say my structure array has name 's', 10,000 elements, and field names 'x' and 'y'.
I'd like to do the following:
xs = [s(:).x];
xs = xs + 5*randn(size(xs));
s(:).x = xs;
Sadly, the last step is not valid matlab. Any thoughts? Was hoping to avoid a loop.
From this answer and after playing around with deal. I think I have what you are looking for but it requires converting xs into a cell array using num2cell:
xs_cell = num2cell(xs); % convert matrix to cell array.
[S(:).X]=xs_cell{:}; % update the values in the field X