Delphi dynamic array with tlist - arrays

I'm creating a simple interpreter in Delphi/Lazarus for a language similar to BASIC.
I already achieved a lot of functionalities. At this moment, I'm trying to create a DIM like command in order to handle multidimensional numeric arrays.
My idea is to use TList to simulate multidimensional arrays limited only by available memory. For example, when a declare in my interpreter a command like the following:
DIM num_arr[3,3,3]
I would like to create a three dimensional array of double, with each index varying from 0 to 2.
So far I have only the function to create the "TList array". I'm using two TList objects to keep the array dimensions and the data items list, I have a third one that holds the indexes for store/retrieve data. What I can't figure is how to associate a list of indexes to a specific entry in the TList. It's simple when the array is up to two dimensions, I can convert each pair of indexes to a numeric sequence, but no success to three and more dimensions.
Is there any algorithm I could use to solve that? It's really hard to find something related to that matter.
Any suggestion about how to implement something similar is welcome. Below I'm publishing part of the code I developed so far:
//Creates a "n" dimensional array
function array_allocate(Dim: TList; var DataArray: TList): integer;
var
i: integer;
s: string;
begin
Result := 1;
//Simple way to find the array length
//For example. An array with dimensions [3,3,3] will handle 27 items
for i := 0 to Dim.Count-1 do
begin
Result := Result * Integer(Dim[i]);
end;
Result := Result;
DataArray.Capacity := Result; //DataArray now handles 27 items
//************************************
//Every line below is just for testing
//************************************
fmMain.Memo1.Lines.Add('Allocating size for array with '+IntToStr(Dim.Count)+' dimension(s).');
s := '';
for i := 0 to Dim.Count-1 do
s := s + '['+IntToStr(Integer(Dim[i]))+']';
fmMain.Memo1.Lines.Add('DIM Sizes: '+s);
fmMain.Memo1.Lines.Add('Manage: '+IntToStr(Result)+' Items.');
end;
{*************************************}
{NOT functional, this is the challenge}
{*************************************}
function calculate_offset(Dim, Indexes: TList; var DataArray: TList; BaseAddr: integer): integer;
var
i, Depth, dimIdx: Integer;
k,index,sizeProduct: integer;
begin
for Depth := 0 to Dim.Count-1 do
for dimIdx := 0 to Integer(Dim[Depth])-1 do
fmMain.mmOut.Lines.Add('Dim: '+IntToStr(Depth)+' ,Index: '+IntToStr(dimIdx));
result := 0;
end;
procedure TfmMain.FormShow(Sender: TObject);
var
dataList: TList; //keep the data
dimList: TList; //keep the dimensions
indexList: TList; //keep the indexes
offset: integer;
begin
dimList := TList.Create; //create the dim array
//simulate the creation of an array with dimension [3,3,3]
//something like DIM myVar[3,3,3]
dimList.Add(Pointer(3));
dimList.Add(Pointer(3));
dimList.Add(Pointer(3));
dataList := TList.Create; //create the data list
array_allocate(dimList, dataList); //allocates memory
//indexList is the way to indicate which index to retrieve/store data
indexList := TList.Create;
indexList.Add(Pointer(1));
indexList.Add(Pointer(1));
indexList.Add(Pointer(1));
indexList.Add(Pointer(1));
//The idea is to relate indexes like [0,0,0], [0,0,1], [0,1,1] to
//a numeric sequence between 0 to 26 in this case (DIM = [3,3,3])
offset := calculate_offset(dimList, indexList, dataList, 1, 0);
indexList.Free;
dimList.Free;
dataList.Free;
end;

As I read your question, you want to convert from a tuple of indices to a linear index, assuming row major storage.
Suppose the dimensions are held in a dynamic array, dim and the indices are in a dynamic array, idx. Then the linear index is calculated like this:
function LinearIndex(const dim, idx: array of Integer): Integer;
var
i: Integer;
begin
Assert(Length(dim)=Length(idx));
Assert(Length(dim)>0);
Result := idx[0];
for i := 1 to high(dim) do
Result := Result*dim[i-1] + idx[i];
end;

The LinearIndex function described above was not functional to certain unbalanced arrays. For example, if I properly allocate "dim" to hold the value [2,3], the function would return 2 as the index for both contents of "idx": [0,2], [1,0].
I created a Pascal function based on another post I found here at StackOverflow and this one seems to work.
function LinearIndex2(const dim, idx: array of Integer): Integer;
var
i,index,mult: Integer;
begin
Assert(Length(dim)=Length(idx));
Assert(Length(dim)>0);
index := 0; mult := 1;
for i := 0 to High(dim) do
begin
index := index + idx[i] * mult;
mult := mult * dim[i];
end;
result := index;
end;

Related

Delphi: How to reference an array from within a class

I work with arrays and have tested the functionality within the context of a class, like:
Ttest = class
values : array of integer;
procedure doStuff;
end;
The methods, like doStuff all operate on the values array without the need to pass the array as a parameter. This suits me and it is fast.
Now I want to use this class to work with an external array, like Ttest.create(myValues) In the constructor I could copy myValues to the internal values but that would be quite a waste and moreover, at the end the copy would have to be reversed to pass the updated values back.
My question is how can I extend this class so that it can efficiently work with an external array. In pseudo code like this:
constructor create(var myValues : array of integer);
begin
address of values := address of myValues;
doSTuff;
end;
Lesson 1
In Delphi, dynamic arrays are reference types. A variable of dynamic array type contains only a pointer to the actual dynamic array heap object, and in an assignment,
A := B
where A and B are dynamic arrays of the same type, the dynamic array heap object isn't copied. The only thing that happens is that A and B will point to the same dynamic array heap object (that is, the 32- or 64-bit B pointer is copied to A) and that the reference count of the heap object is increased by 1.
Lesson 2
When you write
constructor Create(var AValues: array of Integer);
you need to realise that this, despite the appearance, isn't a dynamic array parameter, but an open array parameter.
If you explicitly want a dynamic array parameter, you need to use such a type explicitly:
constructor Create(AValues: TArray<Integer>);
By definition, TArray<Integer> = array of Integer is a dynamic array of Integers.
Please note that the language only has two types of arrays – static and dynamic; the open array concept is only about function parameters.
If you want to work with dynamic arrays, taking advantage of their nature as reference types, I would suggest you use dynamic array parameters. Then the only thing that is passed to the function (the constructor in this case) is the pointer to the heap object. And the heap object's reference count is increased, of course.
Lesson 3 – An example
var
Arr: TArray<Integer>;
procedure Test(A: TArray<Integer>);
var
i: Integer;
begin
for i := Low(A) to High(A) do
A[i] := 2*A[i];
end;
procedure TForm1.FormCreate(Sender: TObject);
var
i: Integer;
begin
SetLength(Arr, 10);
for i := 0 to High(Arr) do
Arr[i] := i;
Test(Arr);
for i := 0 to High(Arr) do
ShowMessage(Arr[i].ToString);
end;
After SetLength, Arr points to a dynamic array heap object of reference count 1. You can see it in your RAM (press Ctrl+Alt+E, then Ctrl+G and goto Arr[0]). When you enter Test, the reference count is increased to 2 because both Arr and A refer to it. When you leave Test, the reference count is reduced back to 1 because A goes out of scope: now again only Arr refers to it.
Lesson 4
Now, a “gotcha”: if you change the number of elements of a dynamic array, it needs to be reallocated (*). Hence, a new dynamic array heap object is created with a reference count of 1, and the old object has its reference count decreased by 1 (and removed if it becomes zero).
Thus, while the previous example works as expected, the following will not:
var
Arr: TArray<Integer>;
procedure Test(A: TArray<Integer>);
var
i: Integer;
begin
SetLength(A, 5);
for i := Low(A) to High(A) do
A[i] := 2*A[i];
end;
procedure TForm1.FormCreate(Sender: TObject);
var
i: Integer;
begin
SetLength(Arr, 10);
for i := 0 to High(Arr) do
Arr[i] := i;
Test(Arr);
for i := 0 to High(Arr) do
ShowMessage(Arr[i].ToString);
end;
The SetLength will create a new dynamic array heap object with refcount 1 and copy half of the old array into this, put the new address in the local A parameter, and Test will transform this new array, not touching the old one which the global Arr variable points to. The reference count of the original heap object is decreased by one.
However, if you use a var parameter,
procedure Test(var A: TArray<Integer>);
var
i: Integer;
begin
SetLength(A, 5);
for i := Low(A) to High(A) do
A[i] := 2*A[i];
end;
it will work as before. This effectively works with Arr. If we had increased the number of elements, the array would probably have been reallocated and the global Arr variable would have been updated with the new address.
Conclusion
As long as you don't need to reallocate the memory (change the number of elements), Delphi already gives you what you want, because dynamic arrays are reference types. If you do need to reallocate, at least now you know enough of the technical details in order to reason about it.
Update: Hence, the suggestion is to do
type
TTest = class
FData: TArray<Integer>;
constructor Create(AData: TArray<Integer>);
procedure Enlarge;
procedure Shrink;
procedure ShowSum;
end;
{ TTest }
constructor TTest.Create(AData: TArray<Integer>);
begin
FData := AData; // will NOT copy the array, since dynamic arrays are reference types
end;
procedure TTest.Enlarge;
var
i: Integer;
begin
for i := 0 to High(FData) do
FData[i] := 2*FData[i];
end;
procedure TTest.ShowSum;
var
s: Integer;
i: Integer;
begin
s := 0;
for i := 0 to High(FData) do
Inc(s, FData[i]);
ShowMessage(s.ToString);
end;
procedure TTest.Shrink;
var
i: Integer;
begin
for i := 0 to High(FData) do
FData[i] := FData[i] div 2;
end;
To test it:
procedure TForm1.FormCreate(Sender: TObject);
var
MyArray: TArray<Integer>;
t: TTest;
begin
MyArray := [1, 2, 3, 4, 5];
t := TTest.Create(MyArray);
try
t.ShowSum;
t.Enlarge;
t.ShowSum;
t.Shrink;
t.ShowSum;
finally
t.Free;
end;
end;
Footnotes
If you (1) decrease the number of elements and (2) the reference count is 1, typically the data isn't moved in memory. If the reference count is > 1, the data is always moved, because SetLength guarantees that the reference count of its argument is 1 when it returns.

How to refactor copying/moving data between arrays?

I have a process that reads data into 150+ temp arrays, process the data and copies from temp arrays into work arrays. Work arrays are in one global array so I can import multiple data, means I can repeat the same process up to 100 times and end up with set of on big array that holds 100x work data I can work with, compare and do stuff.
I have 150+ arrays, so 150 times:
// for each array
SetLength(myData[Idx].WorkNames,Length(tmpNames)); // <- prepare to copy
for i := 0 to High(tmpNames) do // <- copy
myData[Idx].WorkNames[i]:=tmpNames[i];
SetLength(tmpNames,0); // <- clear tmp array
4 lines of code for each array - 150x4 = 600 loc + initial + empty lines - around 900 loc.
Here is the example of what I do:
type
TName = record
NameID:integer;
Description:string;
end;
TItem = record
ItemID:integer;
Description:string;
Active:boolean;
end;
TWorkData = record
WorkDataType:string;
WorkNames:array of TName;
WorkItems:array of TItem;
end;
var
AllWorkData:array of TWorkData; // <- global array that has all work data - up to 100x sets of work data
tmpNames:array of TName; // <- tmp arrays before saving to work array
tmpItems:array of TItem; //
procedure TForm1.Button1Click(Sender: TObject);
var i,Idx:integer;
begin
// 1. read data into tmp arrays
ReadDataIntoTmpArrays;
ProcessTmpData;
// 2. copy tmp arrays into work data
Idx:=GetWorkDataIdx; // <- work data sequence number; start with 0
AllWorkData[Idx].WorkDataType:=GetWorkDataName(Idx);
SetLength(AllWorkData[Idx].WorkNames,Length(tmpNames));
SetLength(AllWorkData[Idx].WorkItems,Length(tmpItems));
for i := 0 to High(tmpNames) do
AllWorkData[Idx].WorkNames[i]:=tmpNames[i];
for i := 0 to High(tmpItems) do
AllWorkData[Idx].WorkItems[i]:=tmpItems[i];
// 3. clear tmp arrays
SetLength(tmpNames,0);
SetLength(tmpItems,0);
end;
Question: Is there something I can do that is easier to maintain, refactor the code?
If you really do want to copy, then do so using generics. You could derive from the TArray class of static class methods declared in System.Generics.Collections. For instance:
type
TArray = class(Generics.Collections.TArray)
public
class function Copy<T>(const Source: array of T; Index, Count: Integer): TArray<T>; overload; static;
class function Copy<T>(const Source: array of T): TArray<T>; overload; static;
end;
....
class function TArray.Copy<T>(const Source: array of T; Index, Count: Integer): TArray<T>;
var
i: Integer;
begin
SetLength(Result, Count);
for i := 0 to high(Result) do begin
Result[i] := Source[i+Index];
end;
end;
class function TArray.Copy<T>(const Source: array of T): TArray<T>;
var
i: Integer;
begin
SetLength(Result, Length(Source));
for i := 0 to high(Result) do begin
Result[i] := Source[i];
end;
end;
Note that all of the above requires you to stop using array of TMyType and instead start using the generic dynamic array TArray<TMyType>.
In your case though you are overcomplicating. Replace:
SetLength(myData[Idx].WorkNames,Length(tmpNames)); // <- prepare to copy
for i := 0 to High(tmpNames) do // <- copy
myData[Idx].WorkNames[i]:=tmpNames[i];
SetLength(tmpNames,0); // <- clear tmp array
with:
myData[Idx].WorkNames := tmpNames;
tmpNames := nil;
If you are prepared to let tmpNames simply leave scope then you can use a single line:
myData[Idx].WorkNames := tmpNames;
Although, if tmpNames is re-used for a different array, then the nil assignment is needed.
Then again, as far as I can see from the code in the question you don't need the temporary arrays at all. Why not operate directly on the long lived data structures.
These array assignments are only permissible if the source and target of the assignment are assignment compatible. Your types are not because you have used distinct types. Switch to TArray<T> to avoid that. See this question for more: Why are two seemingly identical dynamic array types deemed not assignment compatible?
Remember that dynamic arrays are reference types. In the usage shown here, all you need to do is copy the reference. You only need one instance of the actual array. So copying is not necessary at all.

Searching for a one dimensional array in multi dim. array

Please what would be the fastest (and most 'elegant') way to search for FaDirection array in the multi dim. FdVSubs array?
TaDirection = array[0..7] of TSubRect; //TSubRect = class(TObject)
Multi dim array:
FdVSubs: array[0..15] of TaDirection;
Array:
FaDirection : TaDirection;
I need to resolve if FaDirection is already stored in FdVSubs or not.
Thank You.
First and most important, you need to decide what defines two arrays as being equal. Then you'll have to adjust my code to match that definition. For the answer I'll assume arrays are equal if and only if they have identical object references at each corresponding position in the arrays.
function DirectionsAreEqual(const ADirection1, ADirection2: TaDirection): Boolean;
var
LoopI: Integer;
begin
Result := True;
for LoopI := Low(TaDirection) to High(TaDirection) do
begin
if (ADirection1[LoopI] <> ADirection2[LoopI]) then
begin
Result := False;
Exit;
end;
end;
end;
function DirectionExistsIn(const ADirection: TaDirection;
const ADirectionList: array of TaDirection): Boolean;
var
LoopI: Integer;
begin
Result := False; //Caters for empty DirectionList
for LoopI := Low(ADirectionList) to High(ADirectionList) do
begin
Result := DirectionsAreEqual(ADirection, ADirectionList[LoopI]);
if (Result) then
begin
Exit;
end;
end;
end;
//Then you can simply do the test as follows:
if DirectionExistsIn(FaDirection, FdVSubs) then ...
NOTES
DirectionExistsIn uses an Open Array, so it doesn't matter how many elements are defined for the FdVSubs array.
A small change to DirectionExistsIn can allow you to return the position of the duplicate.

Converting a Variant Array to a Dynamic Array

I'm trying to convert a variant array (of doubles, but it could be anything I guess) to a dynamic array. I usually use the DynArrayFromVariant and DynArrayToVariant procedures, but in this case my variant arrays are 1 based. These two functions only seem to work on 0 based arrays. Any idea how I could do what I need to do?
If you know the type of your array elements you can write more efficient (while less generic) code:
function DoubleDynArrayFromVarArray(const V: Variant): TDoubleDynArray;
var
P: Pointer;
Count: Integer;
begin
Result := nil;
if not VarIsArray(V) or (VarType(V) and varTypeMask <> varDouble) or
(VarArrayDimCount(V) <> 1) then
raise EVariantInvalidArgError.Create(SVarInvalid);
Count := VarArrayHighBound(V, 1) - VarArrayLowBound(V, 1) + 1;
if Count = 0 then
Exit;
P := VarArrayLock(V);
try
SetLength(Result, Count);
Move(P^, Result[0], Count * SizeOf(Double));
finally
VarArrayUnlock(V);
end;
end;

How to swap two rows of a two-dimensional array, and why would it work?

Summarization:
Please check the comments below from David, Uwe, and other experts.
================================================================================
The following code swaps two rows in a two-dimensional, dynamic array of double values. I am wondering: (1) whether the following code is a best practice of swapping two rows of a two-dimensional array? If not, then what is the best practice to do this kind of job? (2) why would the following code work? I mean, isn't two-dimensional array a continuous contiguous section of memory? Does the following code work only by luck? Any suggestion is appreciated!
unit Unit5;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs;
type
TAADouble = array of array of Double;
TForm5 = class(TForm)
procedure FormShow(Sender: TObject);
private
{ Private declarations }
public
{ Public declarations }
end;
var
Form5: TForm5;
procedure SwapRows(arr: TAADouble; row0, row1: Integer);
implementation
{$R *.dfm}
procedure SwapRows(arr: TAADouble; row0, row1: Integer);
var
Tmp: Integer;
begin
{$IFDEF FPC}
Tmp := PtrUInt(arr[row0]);
PtrUInt(arr[row0]) := PtrUInt(arr[row1]);
PtrUInt(arr[row1]) := Tmp;
{$ELSE}
Tmp := Integer(arr[row0]);
Integer(arr[row0]) := Integer(arr[row1]);
Integer(arr[row1]) := Tmp;
{$ENDIF}
end;
procedure TForm5.FormShow(Sender: TObject);
var
tmpArray: TAADouble;
I, J: Integer;
rowStr: string;
begin
SetLength(tmpArray, 10, 10);
rowStr := '';
for I := 0 to 9 do
for J := 0 to 9 do
tmpArray[I][J] := I * J;
for I := 0 to 9 do
begin
rowStr := '';
for J := 0 to 9 do
rowStr := rowStr + FloatToStr(tmpArray[I][J]) + ' ';
OutputDebugString(PWideChar(rowStr));
end;
SwapRows(tmpArray, 3, 4);
for I := 0 to 9 do
begin
rowStr := '';
for J := 0 to 9 do
rowStr := rowStr + FloatToStr(tmpArray[I][J]) + ' ';
OutputDebugString(PWideChar(rowStr));
end;
end;
end.
You ask:
Does the following code work only by
luck?
Well, yes, you are relying on implementation specific details.
In fact the correct way to write it is perfectly natural and simple:
type
TDoubleArray = array of Double;
TDoubleMatrix = array of TDoubleArray;
procedure SwapRows(M: TDoubleMatrix; Row1, Row2: Integer);
var
Temp: TDoubleArray;
begin
Temp := M[Row1];
M[Row1] := M[Row2];
M[Row2] := Temp;
end;
You need to declare an intermediate type for the row, TDoubleArray, so that you can perform the assignment to Temp in the swap routine.
A 2D constant size array
array [1..M] of array [1..N] of TMyType
is a contiguous block of memory.
A 2D dynamically size array as you have is not. Indeed it can even be ragged in the sense that the rows have different numbers of columns. So you can have, say, a triangular matrix.
A dynamic array is implemented as a pointer to a memory block representing that array. So a two-dimensional dynamic array is actually a pointer to an array of pointers. Thats why swapping the row(-pointer)s actually works.
See David's answer for a cleaner approach.
Update:
If you are allowed to use generics you might as well do this:
procedure <SomeClassOrRecord>.SwapRows<T>(var arr: TArray<T>; row0, row1: Integer);
var
Tmp: T;
begin
Tmp := arr[row0];
arr[row0] := arr[row1];
arr[row1] := Tmp;
end;

Resources