i am using octave for my project as i cant afford a matlab license, however i have run into a significant roadblock and that is the lack of associative array data structures.
my problem is this:
i have some data in the form of cell arrays containing matrices with each cell array representing a potential solution to a problem. i also have a floating point number that represents the evaluated performance of that solution which i want to put into a map-like data structure with the floating point score as the key, in order to sort the solutions by their performance.
can anyone suggest a simple solution to this problem?
what i have thought about doing so far is making each element part of a two element cell array, with the evaluation score as the first element and the data as the second, and then put those arrays into another cell array, which i then apply some sorting algorithm to, sorting by array{i}{1}.. but this seems like a pretty clunky solution.
does octave have any functionality in this respect that i am just unaware of? or is my clunky solution the only way to achieve this?
any help would be greatly appreciated
thanks
How about keeping the cell array as it is, but create a matrix where the columns are the evaluation score and the index into the cell array. Then you can easily use sortrows on the evaluation score column and use the index to pull the solution from the cell array. I think this should be a simple solution that has the benefit of not rearranging your potentially large set of data.
Related
I've found answers to similar problems, but none of them exactly described my problem.
so on the risk of being down-voted to hell I was wondering if there is a standard method to solve my problem. Further, there's a chance that I'm asking the wrong question. Maybe the problem can be solved more efficiently another way.
So here's some background:
I'm looping through a list of particles. Each particle has a list of it's neighboring particles. Now I need to create a list of unique particle pairs of mutual neightbours.
Each particle can be identified by an integer number.
Should I just build a list of all the pair's including duplicates and use some kind of sort & comparator to eliminate duplicates or should I try to avoid adding duplicates into my list in the first place?
Performance is really important to me. I guess most of the loops may be vectorized and threaded. On average each particle has around 15 neighbours and I expect, that there will be 1e6 particles at most.
I do have some ideas, but I'm not an experienced coder and I don't want to waste 1 week to test every single method by benchmarking different situations just to find out that there's already a standard meyjod for my problem.
Any suggestions?
BTW: I'm using C.
Some pseudo-code
for i in nparticles
particle=particles[i]; //just an array containing the "index" of each particle
//each particle has a neightbor-list
for k in neighlist[i] //looping through all the neighbors
//k represent the index of the neighbor of particle "i"
if the pair (i,k) or (k,i) is not already in the pair-list, add it. otherwise don't
Sorting the elements each iteration is not a good idea since comparison sort is O(n log n) complex.
The next best thing would be to store the items in a search tree, better yet binary search tree, and better yet self equalizing binary search tree, you can find implementations on GitHub.
Even better solution would give an access time of O(1), you can achieve this in 2 different ways one is a simple identity array, where at each slot you would save say a pointer to item if there is on at this id or some flag defining that current id is empty. This is very fast but wasteful. You'll need O(N) memory.
The best solution in my opinion would be to use a set or a has-map. Which are basically the same because sets can be implemented using hash-map.
Here is a github project with c hash-map implementation.
And stack overflow answer to a similar question.
Edited...
Thanks for every one to try to help me!!!
i am trying to make a Finite Element Analysis in Mathemetica.... We can obtain all the local stiffness matrices that has 8x8 dimensions. I mean there are 2000 matrices they are similar but not same. every local stiffness matrix shown like a function that name is KK. For example KK[1] is first element local stiffness matrix
i am trying to assemble all the local matrices to make global stiffness matrix. To make it easy:
Do[K[e][i][j]=KK[[e]][[i]][[j]],{e,2000},{i,8},{j,8}]....edited
Here is my question.... this equality can affect the analysis time...If yes what can i do to improve this...
in matlab this is named as 3d array but i don't know what is called in Mathematica
what are the advantages and disadvantages of this explanation type in Mathematica...is t faster or is it easy way
Thanks for your help...
It is difficult to understand what your question is, so you might want to reformulate it.
As others have mentioned, there is no advantage to be expected from a switch from a 3D array to DownValues or SubValues. In fact you will then move from accessing data-structures to pattern matching, which is powerful and the real strength of Mathematica but not very efficient for what you plan to do, so I would strongly suggest to stay in the realm of ordinary arrays.
There is another thing that might not be clear for someone more familiar with matlab than with Mathematica: In Mathematica the "default" for arrays behave a lot like cell arrays in matlab: each entry can contain arbitrary content and they don't need to be rectangular (as High Performance Mark has mentioned they are just expressions with a head List and can roughly be compared to matlab cell arrays). But if such a nested list is a rectangular array and every element of it is of the same type such arrays can be converted to so called PackedArrays. PackedArrays are much more memory efficient and will also speed up many calculations, they behave in many respect like regular ("not-cell") arrays in matlab. This conversion is often done implicitly from functions like Table, which will oten return a packed array automatically. But if you are interested in efficiency it is a good idea to check with Developer`PackedArrayQ and convert explicitly with Developer`ToPackedArray if necessary. If you are working with PackedArrays speed and memory efficiency of many operations are much better and usually comparable to verctorized operations on normal matlab arrays. Unfortunately it can happen that packed arrays get "unpacked" by some operations, so if calculations become slow it is usually a good idea to check if that has happend.
Neither "normal" arrays nor PackedArrays are restricted in the rank (called Depth in Mathematica) they can have, so you can of course create and use "3D arrays" just as you can in matlab. I have never experienced or would know of any efficiency penalties when doing so.
It probably is of interest that newer versions of Mathematica (>= 10) bring the finite element method as one of the solver methods for NDSolve, so if you are not doing this as an exercise you might want to have a look what is available already, there is quite excessive documentation about it.
A final remark is that you can instead of kk[[e]][[i]][[j]] use the much more readable form kk[[e,i,j]] which is also easier and less error prone to type...
extended comment i guess, but
KK[e][[i]][[j]]
is not the (e,i,j) element of a "3d array". Note the single
brackets on the e. When you use the single brackets you are not denoting an array or list element but a DownValue, which is quite different from a list element.
If you do for example,
f[1]=0
f[2]=2
...
the resulting f appears similar to an array, but is actually more akin to an overloaded function in some other language. It is convenient because the indices need not be contiguous or even integers, but there is a significant performance drawback if you ever want to operate on the structure as a list.
Your 'do' loop example would almost certainly be better written as:
kk = Table[ k[e][i][j] ,{e,2000},{i,8},{j,8} ]
( Your loop wont even work as-is unless you previously "initialized" each of the kk[e] as an 8x8 array. )
Note now the list elements are all double bracketed, ie kk[[e]][[i]][[j]] or kk[[e,i,j]]
I'm trying to create a program that will select the fastest sorting algorithm for a particular array of integers. I'm trying to check off the condition "is almost sorted," and was wondering what common practice to find this in the industry is.
Assume that there is a sorted array available to the coder. The two possible solutions I can think of are:
Loop through both lists simultaneously. Compare values at the index, find the percentage of correctly placed values. I understand that this is pretty quick (just O(N)), but it can be wildly inaccurate... what if everything is shifted by one space? This algorithm will give 0, but insertion sort will take a single run to do order this.
Find how far something is shifted from it's correct position in either direction (w/ wraparound). This seems to be a better solution, but could be pretty slow (O(N^2), since we might have to loop through a sorted list for every unsorted object, which could be corrected A BIT by comparing the value in a while loop).
Are there others? If not, which do I pick?
Thanks!
i knew its a weird Idea to think about, i want to know if it is possible to reverse the mechanism of sorting, (i dont want to reverse the order).
for example lets say i have a random array of integers, then i sorted the array with quicksort method, now i want to go back and un-sort the array and get it back to the it was.
you may suggest i save a copy of the array, thats not what i want, think of it as a time line and have the ability to go backward or forward of sorting method.
and if it is possible please consider showing me the best way to do it with Delphi XE.
Thanks in advance.
It is not possible to unsort. You have to either:
create a separate array that holds a copy of the values and then sort that array so that you can preserve the original array.
create a separate array that holds pointers/indexes to the values in the original array, and then sort the second array using the values it refers to.
I usually use a function that given an array returns an array of the sorted indexes.
This way you will always have the original data and you will be able to access the data in a sorted way using something like:
for jIndex in ASortedIndexesArray do
ShowMessage(AOriginalArray[jIndex]);
Hope this helps.
Mirko
If you want to think of it as a time line and have the ability to go backward or forward of sorting method then organize it like a timeline - with records in a file. Save each step and you will be able to reproduce it.
If the array is of integers the indices will not help you as one index (a pointer) takes the same memory as one array element. If you lack RAM use a file to store and retrieve the array. If you use larger data structures you can create, store and retrieve indices as David suggested.
You could log the swapping actions that quicksort (or whatever sorting algorithm you use) does to a list and then go forward and backward in that list to undo/redo these actions. Not simple to implement, but doable.
I have a 3D array of int values and I want to search just through one of the subarrays for a specific value. While I could for-loop my way through every possible combination of the below code.
array[numberIwant][1-255[1-255];
That seems like overkill. I've come across the foreach type of for and thought that might be the answer to my quest but either it's not or I don't understand it well enough to get it to work. Could anyone suggest the way this should be done?
The foreach and the for loop will have almost identical processing time for an array of that size.
Although it might seem like overkill it is not, you will need to do a triple nested for loop then have an if statement seeing if the number you want was found.