Lua code
local var = {}
for i = 1, 10000, 1 do
table.insert ( var, i )
end
var[5] = { some = false, another = 2 }
var[888] = #userdata
var[10000] = {}
var[1] = 1
var[1] = false
1 ) After this, is var still only with array-part, or it has hash-part too?
2 ) Does code
var[10001] = 1
add hash-part to var or it only forces table rehash without adding hash-part ?
3) How it affects on size of table?
Thanks!
The table will only have an array part after both 1 and 2. The reason is that you have a contiguous set of indices. Specifically, you created entries from 1 to 10,001 and Lua will have allocated space for them.
If for example, you had created 1 to 1000 then added 10001, it would have added the last one in the hash part rather than create nil entries for all the entries between.
It doesn't matter what type of data you put in as the value for the entries, Lua is only interested in the indices when deciding between array and hash. The exception here is setting values to nil. This can get a bit complicated but if table space is your primary concern, I don't believe Lua ever reduces the array an hash parts if you nil sets of them out. I could be mistaken on this.
As to the size, Lua uses a doubling strategy. So, after you hit entry 8192, Lua added another 8192 so there was no extra array space created between 10000 and 10001.
BTW Lua doesn't rehash every table addition. When it adds buckets, it gives itself some headroom. I believe it doubles there too. Note, if your data is sparse i.e. you aren't going to fill most of the indices between 1 and your max, then this approach to hashing can be very beneficial for space even if your indices are numbers. The main downside is it means you can't use ipairs across all your entries.
Related
I have an existing .py file that prints a classifier.predict for a SVC model. I would like to loop through each row in the X feature set to return a prediction.
I am currently trying to define the element from which to iterate over so as to allow for definition of the test statistic feature set X.
The test statistic feature set X is written in code as:
X_1 = xspace.iloc[testval-1:testval, 0:5]
testval is the element name used in the for loop in the above line:
for testval in X.T.iterrows():
print(testval)
I am having trouble returning a basic set of index values for X (X is the pandas dataframe)
I have tested the following with no success.
for index in X.T.iterrows():
print(index)
for index in X.T.iteritems():
print(index)
I am looking for the set of index values, with base 1 if possible, like 1,2,3,4,5,6,7,8,9,10...n
seemingly simple stuff...i haven't located an existing question via stackoverflow or google.
ALSO, the individual dataframes I used as the basis for X were refined with the line:
df1.set_index('Date', inplace = True)
Because dates were used as the basis for the concatenation of the individual dataframes the loops as written above are returning date values rather than
location values as I would prefer hence:
X_1 = xspace.iloc[testval-1:testval, 0:5]
where iloc, location is noted
please ask for additional code if you'd like to see more
the loops i've done thus far are returning date values, I would like to return index values of the location of the rows to accommodate the line:
X_1 = xspace.iloc[testval-1:testval, 0:5]
The loop structure below seems to be working for my application.
i = 1
j = list(range(1, len(X),1)
for i in j:
I'm quite new to MatLab and this problem really drives me insane:
I have a huge array of 2 column and about 31,000 rows. One of the two columns depicts a spatial coordinate on a grid the other one a dependent parameter. What I want to do is the following:
I. I need to split the array into smaller parts defined by the spatial column; let's say the spatial coordinate are ranging from 0 to 500 - I now want arrays that give me the two column values for spatial coordinate 0-10, then 10-20 and so on. This would result in 50 arrays of unequal size that cover a spatial range from 0 to 500.
II. Secondly, I would need to calculate the average values of the resulting columns of every single array so that I obtain per array one 2-dimensional point.
III. Thirdly, I could plot these points and I would be super happy.
Sadly, I'm super confused since I miserably fail at step I. - Maybe there is even an easier way than to split the giant array in so many small arrays - who knows..
I would be really really happy for any suggestion.
Thank you,
Arne
First of all, since you wish a data structure of array of different size you will need to place them in a cell array so you could try something like this:
res = arrayfun(#(x)arr(arr(:,1)==x,:), unique(arr(:,1)), 'UniformOutput', 0);
The previous code return a cell array with the array splitted according its first column with #(x)arr(arr(:,1)==x,:) you are doing a function on x and arrayfun(function, ..., 'UniformOutput', 0) applies function to each element in the following arguments (taken a single value of each argument to evaluate the function) but you must notice that arr must be numeric so if not you should map your values to numeric values or use another way to select this values.
In the same way you could do
uo = 'UniformOutput';
res = arrayfun(#(x){arr(arr(:,1)==x,:), mean(arr(arr(:,1)==x,2))), unique(arr(:,1)), uo, 0);
You will probably want to flat the returning value, check the function cat, you could do:
res = cat(1,res{:})
Plot your data depends on their format, so I can't help if i don't know how the data are, but you could try to plot inside a loop over your 'res' variable or something similar.
Step I indeed comes with some difficulties. Once these are solved, I guess steps II and III can easily be solved. Let me make some suggestions for step I:
You first define the maximum value (maxValue = 500;) and the step size (stepSize = 10;). Now it is possible to iterate through all steps and create your new vectors.
for k=1:maxValue/stepSize
...
end
As every resulting array will have different dimensions, I suggest you save the vectors in a cell array:
Y = cell(maxValue/stepSize,1);
Use the find function to find the rows of the entries for each matrix. At each step k, the range of values of interest will be (k-1)*stepSize to k*stepSize.
row = find( (k-1)*stepSize <= X(:,1) & X(:,1) < k*stepSize );
You can now create the matrix for a stepk by
Y{k,1} = X(row,:);
Putting everything together you should be able to create the cell array Y containing your matrices and continue with the other tasks. You could also save the average of each value range in a second column of the cell array Y:
Y{k,2} = mean( Y{k,1}(:,2) );
I hope this helps you with your task. Note that these are only suggestions and there may be different (maybe more appropriate) ways to handle this.
In my program I need to work with arrays roughly 500x500 to 1500x1500 within a function that is looped over 1000's of times. In each iteration, I need to start with an array that has the same form (whose dimensions are fixed across all iterations). The initial values will be:
[0 0 0 ... 1]
[0 0 0 ... 1]
....
However, the contents of the array will be modified within the loop. What is the most efficient way to "reset" the array to this format so I can pass the same array to the function every time without having to allocate a new set of memory every time? (I know the range of rows that were modified)
I have tried:
a[first_row_modified:last_row_modified,:] = 0.
a[first_row_modified:last_row_modified,:-1] = 1.
but it takes roughly the same amount of time as just creating a new array every time with the following:
a = zeros((sizeArray, sizeArray))
a[:,-1] = 1.
Is there a faster way to effectively "erase" the array and change the last column to ones? I think this is similar to this question, clearing elements of numpy array , although my array doesn't change sizes and i didn't see the definitive answer to the previously asked question.
No; I think the way you are doing it is about as fast as it gets.
I've embedded Lua into my C application, and am trying to figure out why a table created in my C code via:
lua_createtable(L, 0, numObjects);
and returned to Lua, will produce a result of zero when I call the following:
print("Num entries", table.getn(data))
(Where "data" is the table created by lua_createtable above)
There's clearly data in the table, as I can walk over each entry (string : userdata) pair via:
for key, val in pairs(data) do
...
end
But why does table.getn(data) return zero? Do I need to insert something into the meta of the table when I create it with lua_createtable? I've been looking at examples of lua_createtable use, and I haven't seen this done anywhere....
table.getn (which you shouldn't be using in Lua 5.1+. Use the length operator #) returns the number of elements in the array part of the table.
The array part is every key that starts with the number 1 and increases up until the first value that is nil (not present). If all of your keys are strings, then the size of the array part of your table is 0.
Although it's a costly (O(n) vs O(1) for simple lists), you can also add a method to count the elements of your map :
>> function table.map_length(t)
local c = 0
for k,v in pairs(t) do
c = c+1
end
return c
end
>> a = {spam="data1",egg='data2'}
>> table.map_length(a)
2
If you have such requirements, and if your environment allows you to do so think about using penlight that provides that kind of features and much more.
the # operator (and table.getn) effectivly return the size of the array section (though when you have a holey table the semantics are more complex)
It does not count anything in the hash part of the table (eg, string keys)
for k,v in pairs(tbl) do count = count + 1 end
I'm trying to check if my arrays are returning nonsense by accessing out of bounds elements, in fortran. And I want to check these values are less than one, and if they are, change them to one.
This is the piece of my code causing issues:
lastNeighLabel=(/clusterLabel(jj-1,kk,ll), clusterLabel(jj,kk-1,ll), clusterLabel(jj,kk,ll-1)/)
LastNeighLabel contains the cluster label (between 1 and n, where n isthe total number of unique seperate clusters found) for the last neighbour in the x,y,z direction respectively.
When jj or kk or ll are 1, they try and access the 0th element in the array, and as FORTRAN counts from 1 in arrays, it tries to destroy the universe. I'm currently in a tangled mess of about 8 if/elseif statements trying to code for every eventuality. But I was hoping there was a way of operating on each element. So basically I'd like to say where((/jj-1,kk-1,ll-1/).lt.1) do clusterLabel(jj-1,kk,ll)=0 etc depending on which element is causing the problem.
But I can't think of a way to do that because where will only manipulate the variables passed to it, not a different array at the same index. Or am I wrong?
Will gladly edit if this doesn't make sense.
It is not obligatory that Fortran accesses arrays starting from one. Any starting value is allowed. If it more convenient to you to have a zero indexed array, declare the array as:
real, dimension (0:N-1, 0:M-1) :: array
Or
real, dimension (0:N, 0:M) :: array
and have the 0 indices be extra to catch special cases.
This might be another solution to your problem, since zero index values would be legal.
Another possible way to approach this, is to create an extended cluster label array (with index bounds starting at 0), which is equal to the cluster label array with a layer of zeroes tacked on the outside. You can then let your loop run safely over all values of jj, kk, and ll. It depends on the size of the array if this is a feasible solution.
integer :: extended_cluster_label(0:size(cluster_label,1), &
0:size(cluster_label,2), &
0:size(cluster_label,3) &
)
extended_cluster_label(0,:,:) = 0
extended_cluster_label(:,0,:) = 0
extended_cluster_label(:,:,0) = 0
extended_cluster_label(1:, 1:, 1:) = cluster_label
Maybe you could use a function?
real function f(A,i,j,k)
real :: A(:,:,:)
integer :: i,j,k
if (i==0.or.j==0.or.k==0) then
f=0
else
f=A(i,j,k)
endif
end function f
and then use f(clusterLabel,jj-1,kk,ll) etc.