I'm trying to call a Delphi DLL from Excel and return a column of variant data values. I have the dll returning a single shortstring, and that appears in a cell ok. Now I am trying to return a column of variable values. It gets into my code ok, but the array in excel is all 0.
Any ideas greatly appreciated.
Here is the macro registration in Excel: =REGISTER("c:\projects\test\delphixl.dll","GetPolicyData","KDD","GetPolicyData","Co,Pol",1,"Delphi")
I'm not sure what the 1 is after the parameters; I cant find full documentation anywhere.
the range in Excel has: {=GetPolicyData(C1,D1)}
Below is the code in D7. According to the excel doc for the register functions, this appears to be OK.
K Data Type
The K data type uses a pointer to a variable-size FP structure. You must define this structure in the DLL or code resource as follows:
typedef struct _FP
{
unsigned short int rows;
unsigned short int columns;
double array[1]; /* Actually, array[rows][columns] */
} FP;
Type
pPolicyData = ^tPolicyData;
tPolicyData = Record
Rows: word;
Cols: word;
data: variant;
End;
var
pd: tPolicyData;
Function GetPolicyData(co: pShortString; pol: pShortString): pPolicyData; Stdcall;
Var
polc: tpolcmst;
Begin
lpro := tlifepro.create;
lpro.opendatabases;
polc := tpolcmst.create(lpro);
Try
polc.read(co^, pol^);
pd.Rows := 2;
pd.Cols := 1;
pd.data := VarArrayCreate([0, 1], varVariant);
pd.data[0] := datetostr(polc.issuedate);
pd.data[1] := format('%.2f', [polc.modepremium]);
result := addr(pd);
Finally
lpro.closedatabases;
freeit(lpro);
End;
End;
You have declared the data member of the FP record (TPolicyData) as a Variant. It should be declared Double, and it should be an array of the required dimensions.
In this case, with Rows = 2 and Cols = 1 :
TPolicyData = record
Rows: word;
Cols: word;
data: array[0..1] of Double;
End;
If the Rows and Cols are dynamic then you will need to calculate the memory required for the FP record and allocate this dynamically. The required memory will be:
2 bytes for "Rows" word
+ 2 bytes for "Cols" word
+ Rows x Cols x 8 bytes for "data" array of Doubles
With Delphi 7 you will need to use pointer offsets into the data memory to assign the values to the individual Double elements in the "array".
However, since you have not mentioned the need to address variable rows and columns, I shall not go into further detail on that as it may be superfluous.
Related
I hava an array of strings. I increment this array when it is necessary with the function:
function insertMessageAction(list: TMessagesActions; message: String): TMessagesActions;
var
lenght: integer;
begin
if message <> '' then begin
lenght := GetArrayLength(list);
SetArrayLength(list, lenght +1);
if GetArrayLength(list) > lenght then begin
list[lenght] := message
end;
end;
result := list;
end;
If de increment is of 0 to 2, no problems, by when I increment to 3 lenght, the array is corrupted and 'value' of list is: "Acces violation at address 00403498. Read of address 0000006A".
It is impossible to create more long arrays of 2 items (strings)? There is limit of characters?
Thanks.
First of all, do not use array for your task. Memory reallocation that happens when changing the size of your string array is an expensive operation. What's more, you were trying to return a copy of that input array (in a wrong way), which would be unecessarily inefficient too.
I strongly suggest you to use the TStringList which is intended to be used for a collection of strings. By using TStringList, the whole code from your question would become:
StringList.Add(Message);
But to your question. It is possible, but you need to comply with a few things; at least:
Do not attempt to return an input array passed by reference:
Result := list;
If you need to have an array as a return type for some reason, allocate the size of this output array and copy all the elements from the input array:
InputLen := GetArrayLength(List);
SetArrayLength(Result, InputLen + 1);
for I := 0 to InputLen - 1 do
Result[I] := List[I];
Result[InputLen] := Message;
If you'd still like to stay by array, better pass it by variable parameter:
[Code]
type
TMessagesActions = TArrayOfString;
procedure InsertMessageAction(var AList: TMessagesActions;
const AMessage: String);
var
ArrayLen: Integer;
begin
ArrayLen := GetArrayLength(AList);
SetArrayLength(AList, ArrayLen + 1);
AList[ArrayLen] := AMessage;
end;
I am trying to write an application that reads RPM files. The start of each block has a Magic char of [4]byte.
Here is my struct
type Lead struct {
Magic [4]byte
Major, Minor byte
Type uint16
Arch uint16
Name string
OS uint16
SigType uint16
}
I am trying to do the following:
lead := Lead{}
lead.Magic = buffer[0:4]
I am searching online and not sure how to go from a slice to an array (without copying). I can always make the Magic []byte (or even uint64), but I was more curious on how would I go from type []byte to [4]byte if needed to?
The built in method copy will only copy a slice to a slice NOT a slice to an array.
You must trick copy into thinking the array is a slice
copy(varLead.Magic[:], someSlice[0:4])
Or use a for loop to do the copy:
for index, b := range someSlice {
varLead.Magic[index] = b
}
Or do as zupa has done using literals. I have added onto their working example.
Go Playground
You have allocated four bytes inside that struct and want to assign a value to that four byte section. There is no conceptual way to do that without copying.
Look at the copy built-in for how to do that.
Try this:
copy(lead.Magic[:], buf[0:4])
Tapir Liui (auteur de Go101) twitte:
Go 1.18 1.19 1.20 will support conversions from slice to array: golang/go issues 46505.
So, since Go 1.18,the slice copy2 implementation could be written as:
*(*[N]T)(d) = [N]T(s)
or, even simpler if the conversion is allowed to present as L-values:
[N]T(d) = [N]T(s)
Without copy, you can convert, with the next Go 1.17 (Q3 2021) a slice to an array pointer.
This is called "un-slicing", giving you back a pointer to the underlying array of a slice, again, without any copy/allocation needed:
See golang/go issue 395: spec: convert slice x into array pointer, now implemented with CL 216424/, and commit 1c26843
Converting a slice to an array pointer yields a pointer to the underlying array of the slice.
If the length of the slice is less than the length of the array,
a run-time panic occurs.
s := make([]byte, 2, 4)
s0 := (*[0]byte)(s) // s0 != nil
s2 := (*[2]byte)(s) // &s2[0] == &s[0]
s4 := (*[4]byte)(s) // panics: len([4]byte) > len(s)
var t []string
t0 := (*[0]string)(t) // t0 == nil
t1 := (*[1]string)(t) // panics: len([1]string) > len(s)
So in your case, provided Magic type is *[4]byte:
lead.Magic = (*[4]byte)(buffer)
Note: type aliasing will work too:
type A [4]int
var s = (*A)([]int{1, 2, 3, 4})
Why convert to an array pointer? As explained in issue 395:
One motivation for doing this is that using an array pointer allows the compiler to range check constant indices at compile time.
A function like this:
func foo(a []int) int
{
return a[0] + a[1] + a[2] + a[3];
}
could be turned into:
func foo(a []int) int
{
b := (*[4]int)(a)
return b[0] + b[1] + b[2] + b[3];
}
allowing the compiler to check all the bounds once only and give compile-time errors about out of range indices.
Also:
One well-used example is making classes as small as possible for tree nodes or linked list nodes so you can cram as many of them into L1 cache lines as possible.
This is done by each node having a single pointer to a left sub-node, and the right sub-node being accessed by the pointer to the left sub-node + 1.
This saves the 8-bytes for the right-node pointer.
To do this you have to pre-allocate all the nodes in a vector or array so they're laid out in memory sequentially, but it's worth it when you need it for performance.
(This also has the added benefit of the prefetchers being able to help things along performance-wise - at least in the linked list case)
You can almost do this in Go with:
type node struct {
value int
children *[2]node
}
except that there's no way of getting a *[2]node from the underlying slice.
Go 1.20 (Q1 2023): this is addressed with CL 430415, 428938 (type), 430475 (reflect) and 429315 (spec).
Go 1.20
You can convert from a slice to an array directly with the usual conversion syntax T(x). The array's length can't be greater than the slice's length:
func main() {
slice := []int64{10, 20, 30, 40}
array := [4]int64(slice)
fmt.Printf("%T\n", array) // [4]int64
}
Go 1.17
Starting from Go 1.17 you can directly convert a slice to an array pointer. With Go's type conversion syntax T(x) you can do this:
slice := make([]byte, 4)
arrptr := (*[4]byte)(slice)
Keep in mind that the length of the array must not be greater than the length of the slice, otherwise the conversion will panic.
bad := (*[5]byte)(slice) // panics: slice len < array len
This conversion has the advantage of not making any copy, because it simply yields a pointer to the underlying array.
Of course you can dereference the array pointer to obtain a non-pointer array variable, so the following also works:
slice := make([]byte, 4)
var arr [4]byte = *(*[4]byte)(slice)
However dereferencing and assigning will subtly make a copy, since the arr variable is now initialized to the value that results from the conversion expression. To be clear (using ints for simplicity):
v := []int{10,20}
a := (*[2]int)(v)
a[0] = 500
fmt.Println(v) // [500 20] (changed, both point to the same backing array)
w := []int{10,20}
b := *(*[2]int)(w)
b[0] = 500
fmt.Println(w) // [10 20] (unchanged, b holds a copy)
One might wonder why the conversion checks the slice length and not the capacity (I did). Consider the following program:
func main() {
a := []int{1,2,3,4,5,6}
fmt.Println(cap(a)) // 6
b := a[:3]
fmt.Println(cap(a)) // still 6
c := (*[3]int)(b)
ptr := uintptr(unsafe.Pointer(&c[0]))
ptr += 3 * unsafe.Sizeof(int(0))
i := (*int)(unsafe.Pointer(ptr))
fmt.Println(*i) // 4
}
The program shows that the conversion might happen after reslicing. The original backing array with six elements is still there, so one might wonder why a runtime panic occurs with (*[6]int)(b) where cap(b) == 6.
This has actually been brought up. It's worth to remember that, unlike slices, an array has fixed size, therefore it needs no notion of capacity, only length:
a := [4]int{1,2,3,4}
fmt.Println(len(a) == cap(a)) // true
You might be able to do the whole thing with one read, instead of reading individually into each field. If the fields are fixed-length, then you can do:
lead := Lead{}
// make a reader to dispense bytes so you don't have to keep track of where you are in buffer
reader := bytes.NewReader(buffer)
// read into each field in Lead, so Magic becomes buffer[0:4],
// Major becomes buffer[5], Minor is buffer[6], and so on...
binary.Read(reader, binary.LittleEndian, &lead)
Don't. Slice itself is suffice for all purpose. Array in go lang should be regarded as the underlying structure of slice. In every single case, use only slice. You don't have to array yourself. You just do everything by slice syntax. Array is only for computer. In most cases, slice is better, clear in code. Even in other cases, slice still is sufficient to reflex your idea.
I would like to make a procedure that take array of shortstring as argument
procedure f(const a, b: Array of shortstring);
I would like to call this with arrays of known length and shortstrings of known length e.g.
var
A, B: array[1..2] of string[5];
C, D: array[1..40] of string[12];
begin
f(A,B);
f(C,D);
end;
This result in an compiler error E2008 Incompatible types.
Why is that? Can I write a procedure that can take arrays of shortstring (any length of arrays/strings)?
Why use shortstring?
The shortstings are fields in an existing record. There are alot of these record with thousand of shortstrings. In an effort to migrate data from turbo power B-Tree Filer to SQL databases one step is to convert the record to a dataset, and the back to a record, to confirm all fields are converted correctly both directions. I have been using CompareMem on the records to check this, but it does not provide enough information as to which field a conversion error is in. Thus a small program was created, which from the record definition can generate code to compare the two records. It was for this code generator I needed a function to compare shortstrings. It ended up using CompareMem on the shortstrings.
A ShortString is 0 to 255 characters long. The length of a ShortString can change dynamically, but memory is a statically allocated 256 bytes, the first byte stores the length of the string, and the remaining 255 bytes are available for characters, whilist string[5] declared in this way allocate only as much memory as the type requires (5 byte + 1 byte for length).
you could use type
type
MyString = string[5];
...
procedure f(const a, b: Array of MyString);
...
var
A, B: array[1..2] of MyString;
begin
f(A,B);
end;
In a similar situation I've used the following:
type
TOpenArrayOfOpenString = record
strict private
FSizeOfString: Integer;
FpStart: PChar;
FArrayLength: Integer;
function GetItemPtr(AIndex: Integer): PShortString;
public
constructor Init(var AFirstString: Openstring; AArrayLength: Integer);
function Equals(const AArray: TOpenArrayOfOpenString): Boolean;
property SizeOfString: Integer read FSizeOfString;
property pStart: PChar read FpStart;
property ArrayLength: Integer read FArrayLength;
property ItemPtrs[AIndex: Integer]: PShortString read GetItemPtr; default;
end;
{ TOpenArrayOfOpenString }
constructor TOpenArrayOfOpenString.Init(var AFirstString: Openstring; AArrayLength: Integer);
begin
FSizeOfString := SizeOf(AFirstString);
FpStart := #AFirstString[0]; // incl. length byte!
FArrayLength := AArrayLength;
end;
function TOpenArrayOfOpenString.Equals(const AArray: TOpenArrayOfOpenString): Boolean;
begin
Result := CompareMem(pStart, AArray.pStart, SizeOfString * ArrayLength);
end;
function TOpenArrayOfOpenString.GetItemPtr(AIndex: Integer): PShortString;
begin
Result := PShortString(pStart + AIndex * SizeOfString);
end;
You could use it like this:
procedure f(const a: TOpenArrayOfOpenString);
var
i: Integer;
begin
for i := 0 to Pred(a.ArrayLength) do
Writeln(a[i]^);
end;
procedure Test;
var
A: array[1..2] of string[5];
C: array[1..40] of string[12];
begin
f(TOpenArrayOfOpenString.Init(A[1], Length(A)));
f(TOpenArrayOfOpenString.Init(C[1], Length(C)));
end;
It's not as elegant as a solution built into the language could be and it is a bit hacky as it relies on the fact/hope/... that the strings in the array are laid out contiguously. But it worked for me for some time now.
type
shortStrings =array[1..2] of string[5];
...
a,b : shortString;
..
procedure rock(a,b : shortStrings);
..
You are combining two different kinds of open array.
First, there is the classic Turbo Pascal "string" (also called "openstring" in Delphi IIRC) which is essentially string[255]. As string[255] is a superset of all shortstrings, the open array aspect simply converts all shortstring types to it.
The "array of xx" syntax is the Delphi (4+?) open array. It is an open array of any type, not just strings, and the syntax to call it is f(nonarrayparam,[arrayelement0,arrayelement1]);
Somehow you seem to mix both syntaxes, and even aggrevate it by adding CONST which sollicits pass by reference and excludes conversions.
I think you assume shortstring has an performance advantage. It has, in some cases. Open array is not one of those cases. Not even in TP :-)
I am writing a windows application in Lazarus/FreePascal (like Delphi). I have a TDataset object that is populated by 5000 rows, 2 columns of numerical values.
I need to pass this data to a C function that I am importing statically from a .dll library.
Here is an excerpt from the library's manual that explains what format its parameters should be in:
flan_index_t flann_build_index(float* dataset,
int rows,
int cols,
float* speedup,
struct FLANNParameters* flann_params);
This function builds an index and return a reference to it. The
arguments expected by this function are as follows: dataset, rows and
cols - are used to specify the input dataset of points:
dataset is a pointer to a rows cols matrix stored in row-major
order (one feature on each row)
Can I simply pass the TDataSet object? Do I have to do something to it first so that the pointer is in the right form?
Obviously you can't pass the TDataSet object. It is a FreePascal object, and the function seems to expect a pointer to a float (which is probably a pointer to a Single in FreePascal). It probably expects a two-dimensional array of float's. You have to pass another pointer to float and a pointer to a FLANNParameters structure as well.
Move() won't work either. A TDataSet is not an array.
I guess you'll have to declare an array like Uwe said, populate it using your dataset and pass the array:
type
PMyFloatArray = ^TFloatArray;
TMyFloatArray = array[0..4999, 0..1] of Single;
var
MyArray: PMyFloatArray;
idx: flan_index_t;
begin
New(MyArray);
try
// Fill array using your TDataSet...
// set up other parameters...
idx := flann_build_index(MyArray, 5000, 2, &speedup, etc...);
// ...
finally
Dispose(MyArray);
end;
end;
Shameless plug
Please read my Pitfalls of Conversion article about converting function declarations from C to Delphi (and probably FreePascal on Win32). Now I'm at it, you may want to read my article Addressing Pointers too.
No, you can't pass the dataset directly. The naming "dataset" might imply that, but the meaning is totally different. You have to pass a pointer to a float matrix to the function. To implement this you should declare an array[0..4999, 0..1] of float (probably double) and fill it from the dataset.
Using Rudy's solution as a base (Thanks by the way!) I came up with this:
with Datasource1.DataSet do
begin
Open;
First;
field_count := FieldCount;
record_count := RecordCount;
row := 0;
while not EOF do
begin
for col := 0 to field_count - 1 do
MyArray[row, col] := Fields.Fields[col].AsFloat;
row := row + 1; //Shift to next row in array
Next; //Shift to next row in dataset
end;
end;
Seems to work great; and a lot faster than I expected.
I have 2 or more dynamic string array that fill with some huge data , i want to merge this 2 array to one array , i know i can do it with a for loop like this :
var
Arr1, Arr2, MergedArr: Array of string;
I: Integer;
begin
// Arr1:= 5000000 records
// Arr2:= 5000000 records
// Fill MergedArr by Arr1
MergedArr:= Arr1;
// Set length of MergedArr to length of ( Arra1 + Arr2 )+ 2
SetLength(MergedArr, High(Arr1)+ High(Arr2)+2);
// Add Arr2 to MergedArr
for I := Low(Arr2)+1 to High(Arr2)+1 do
MergedArr[High(Arr1)+ i]:= Arr2[i-1];
end;
but it is slow on huge data , is there faster way like copy array memory data ?
First of all string is special, so it should be treated specially: Don't try outsmarting the compiler, keep your code unchanged. String is special because it's reference counted. Every time you copy a string from one place to an other it's reference count is incremented. When the reference count reaches 0, the string is destroyed. Your code plays nice because it lets the compiler know what you're doing, and in turn the compiler gets the chance to properly increment all reference counts.
Sure, you can play all sorts of tricks as suggested in the comments to gabr's answer, like filling the old arrays with zero's so the reference count in the new array remains valid, but you can't do that if you actually need the old arrays as well. And this is a bit of a hack (albeit one that will probably be valid for the foreseeable future). (and to be noted, I actually like this hack).
Anyway, and this is the important part of my answer, your code is most likely not slow in the copying of the strings from one array to the other, it's most likely slowly somewhere else. Here's a short console application that creates two arrays, each with 5M random strings, then merges the two arrays into a third and displays the time it took to create the merge. Merging only takes about 300 milliseconds on my machine. Filling the array takes much longer, but I'm not timing that:
program Project26;
{$APPTYPE CONSOLE}
uses SysUtils, Windows;
var a, b, c: array of string;
i: Integer;
Freq: Int64;
Start, Stop: Int64;
Ticks: Cardinal;
const count = 5000000;
begin
SetLength(a,count);
SetLength(b,count);
for i:=0 to count-1 do
begin
a[i] := IntToStr(Random(1));
b[i] := IntToStr(Random(1));
end;
WriteLn('Moving');
QueryPerformanceFrequency(Freq);
QueryPerformanceCounter(Start);
SetLength(c, Length(a) + Length(b));
for i:=0 to High(a) do
c[i] := a[i];
for i:=0 to High(b) do
c[i+Length(a)] := b[i];
QueryPerformanceCounter(Stop);
WriteLn((Stop - Start) div (Freq div 1000), ' milliseconds');
ReadLn;
end.
You can use built-in Move function which moves a block of memory to another location. Parameters are source and target memory blocks and size of data to be moved.
Because you are copying strings, source arrays must be destroyed after the merging by filling them with zeroes. Otherwise refcounts for strings will be all wrong causing havoc and destruction later in the program.
var
Arr1, Arr2, MergedArr: Array of string;
I: Integer;
begin
SetLength(Arr1, 5000000);
for I := Low(Arr1) to High(Arr1) do
Arr1[I] := IntToStr(I);
SetLength(Arr2, 5000000);
for I := Low(Arr2) to High(Arr2) do
Arr2[I] := IntToStr(I);
// Set length of MergedArr to length of ( Arra1 + Arr2 )+ 2
SetLength(MergedArr, High(Arr1)+ High(Arr2)+2);
// Add Arr1 to MergedArr
Move(Arr1[Low(Arr1)], MergedArr[Low(MergedArr)], Length(Arr1)*SizeOf(Arr1[0]));
// Add Arr2 to MergedArr
Move(Arr2[Low(Arr2)], MergedArr[High(Arr1)+1], Length(Arr2)*SizeOf(Arr2[0]));
// Cleanup Arr1 and Arr2 without touching string refcount.
FillChar(Arr1[Low(Arr1)], Length(Arr1)*SizeOf(Arr1[0]), 0);
FillChar(Arr2[Low(Arr2)], Length(Arr2)*SizeOf(Arr2[0]), 0);
// Test
for I := Low(Arr1) to High(Arr1) do begin
Assert(MergedArr[I] = IntToStr(I));
Assert(MergedArr[I] = MergedArr[Length(Arr1) + I]);
end;
// Clear the array to see if something is wrong with refcounts
for I := Low(MergedArr) to High(MergedArr) do
MergedArr[I] := '';
end;
An excellent maxim is that the fastest code is that which never runs. Since copying is expensive you should look to avoid the cost of copying.
You can do this with a virtual array. Create a class which holds an array of array of string. In your example the outer array would hold two string arrays.
Add a Count property that returns the total number of strings in all of the arrays.
Add a default indexed property that operates by working out which of the outer arrays the index refers to and then returns the appropriate value from the inner array.
For extra points implement an enumerator to make for in work.