ADOQuery.Locate Slow, Create Index - sql-server

I have following line to Locate a row in a Query .
if Query.Locate('Line;Hour;Minute',VarArrayOf([Line-400,AHour,minuteof(Start)]),[]) = true then
This is slow, now I remember that one can add Indexes to the Query so that the Locate itself is multiple times faster. Unfortunately I cannot seem to find the example.
Can someone please help me out?
Regards
Robert

Interesting q.
Update please see update below.
I set up some test data on my SS2014 Sql Server to run some tests using code like this:
ID := 1;
for Line := 1 to 1000 do begin
for AHour := 1 to 24 do begin
for AMinute := 1 to 60 do begin
AdoQuery1.InsertRecord([ID, Line, AHour, AMinute]);
Inc(ID);
end;
end;
end;
end;
Then, I ran some tests like this one
procedure TForm1.LocateTest1(DisableControls, UseSort : Boolean);
var
T1 : Integer;
Line,
AHour,
AMinute : Integer;
begin
AdoQuery1.Sql.Text := 'select * from linetest order by line, ahour, aminute';
AdoQuery1.CursorLocation := clUseClient;
AdoQuery1.Open;
T1 := GettickCount;
if DisableControls then
AdoQuery1.DisableControls;
if UseSort then
AdoQuery1.Recordset.Sort := 'Line,AHour,AMinute';
Line := 1000;
AHour := 23;
for AMinute := 60 downto 1 do begin
if not AdoQuery1.Locate('Line;AHour;AMinute', VarArrayOf([Line, AHour, AMinute]), []) then
Caption := Format('Locate failed %d %d %d', [Line, AHour, AMinute]);
end;
Memo1.Lines.Add('Test1 : ' + IntToStr(GetTickCount - T1));
if DisableControls then
AdoQuery1.EnableControls;
AdoQuery1.Close;
end;
The reason for involving Disable/EnableControls was because of the results I reported
here Why does scrolling through ADOTable get slower and slower?
, that calling DisableControls has a huge impact on scrolling speed even if there are no db-aware
controls involved.
However, it seems that scrolling does not make a major impact on executing Locate() on a TAdoQuery, because calling DisableControls only took about 1.5 seconds of the recorded time of around
26 seconds. Obviously, TAdoQuery.Locate doesn't perform at all well with a large number of rows.
The idea of the UseSort parameter was to see if sorting the RecordSet behind the
AdoQuery made any difference to the speed, but it didn't, the reason being that Locate calls
TCustomAdoDataSet.LocateRecord which uses Sort anyway.
You mentioned adding indexes. Unfortunately, TAdoQuery only supports use of server-side indexes
in executing thq SQL query, not locating records in the retrieved result set. You can add
client-side indexes to TAdoTable, but according to a test similar to the one above, rather to my surprise,
they make
virtually no difference to the speed of Locate().
So, given my results so far, it would seem very possibly quicker to use a parameterised
SELECT to retrieve only the row currently of interest, rather than trying to Locate it
in a large tesult set. Alterantaivel, you could retrieve the result set into a ClientDataSer via a DatasetProvider or into a FireDAC FDMemTable, etc. Ymmv, it depends on what you're doing exactly ...
Update Since posting my original answer, I have a couple of further updates
that it might be useful to include.
One concerns a way of mimicking Locate using calls to the AdoQuery's RecordSet's Find and Filter methods that is significantly faster (around 15 secs) than doing AdoQuery1.Locate repeatedly. I am still probling this and will post another update in a day or two.
The other is to briefly mention doing the Locates doing a FireDAC FDQuery instead of an AdoQuery. This seems to do the same set of Locates as takes around 25 seconds with the AdoQuery in under 9 seconds, using the following code:
Using FDQuery.Locate
procedure TForm2.LocateTest;
var
T1 : Integer;
Line,
AHour,
AMinute : Integer;
begin
FDQuery1.Sql.Text := 'select * from linetest order by line, ahour desc, aminute desc';
//FDQuery1.CursorLocation := clUseClient;
FDQuery1.CursorKind := ckForwardOnly;
FDQuery1.Open;
T1 := GettickCount;
Line := 1000;
AHour := 1;
for AMinute := 1 to 60 do begin
if not FDQuery1.Locate('Line;AHour;AMinute', VarArrayOf([Line, AHour, AMinute]), []) then
Caption := Format('Locate failed %d %d %d', [Line, AHour, AMinute]);
end;
Memo1.Lines.Add('Test1 : ' + IntToStr(GetTickCount - T1));
FDQuery1.Close;
end;

The documentation says that Locate move the cursor to the first row matching a specific search criteria.
If your table have many records, Locate is slow.
Locate is usually used in local databases, but in client-server RDBMs is better thar you use a SQL with WHERE for minimize the search time and minimize the data traffic.

Related

Delete records using TFDQuery in Delphi

Help me to fix this problem to delete records with TFDQuery.
When this value of record is choosed by me with Edit.Text or DBEdit.Text, I try like this but it is not working:
FDQuery.SQL.Text := 'delete * FROM MyTable column_name =:KEY1 ';
FDQuery.ParamByName('KEY1').AsString = 'dbedit.text';
FDQuery.Open;
fdquery.SQL.Text := 'DELETE FROM MyTable WHERE column_name = :KEY1';
fdquery.ParamByName('KEY1').AsString := dbedit.Text;
fdquery.Execute();
You could also use TFDCommand rather than TFDQuery as you are not expecting to read the result:
fdcommand.CommandText := 'DELETE FROM MyTable WHERE column_name = :KEY1';
fdcommand.ParamByName('KEY1').AsString := dbedit.Text;
fdcommand.Execute();
If this is a command you expect to re-use you could put the SQL statement into the command at design time, with the parameter name, and then at run time you would only need to do:
fdcommand.ParamByName('KEY1').AsString := dbedit.Text;
fdcommand.Execute();
Depending on the underlying database you are using have commands pre-populated can allow the query to be prepared in advance. For complex queries (unlike this one) this means that the execution plan is built only once.

Delphi Database Connection Using ACCESS and ADO connections

Okay so basically I've been working on my computing project for a while now and I've got 90% of it working however I'm having a problem with Delphi where is says that my database is not connected/ there is a problem connecting however I've already tried writing the information to the screen and this showed me that the items I was looking to pick up where in fact being picked up so the failure is when the items are being input in to the database. This however shouldn't be happening as the System already has database information displayed from that table and the user can physically select things from the database tables within the program however when trying to store the information back into the database it just breaks. Me and my computing teacher can not work it out, any help would be appreciated.
The problem appears on the new orders page. If you'd rather look at the system then you can download it from here https://drive.google.com/folderview?id=0B_iRfwwM9QpHVXJnSkx4U1FjMlk&usp=sharing
procedure Tform1.btnSaveClick(Sender: TObject);
var orderID:integer;
count:integer;
begin
try
//save into the order table first
tblOrder.Open;
tblOrder.Insert;
tblOrder.FieldByName('CustomerID').value:= strtoint(cboCustomer.Text);
tblOrder.Close;
tblOrder.Open;
tblOrder.Last;
orderID:=tblOrder.FieldByName('OrderID').Value;
showmessage(inttostr(orderID));
for count := 1 to nextFree-1 do
begin
if itemOrdered[count,1]<>0 then
begin
tblOrderLine.Open;
tblOrderLine.AppendRecord([orderID, itemOrdered[count,1],itemOrdered[count,2]]);
end;
end;
showmessage('The order has been saved');
except
showmessage('There was a problem connecting to the database');
end;
end;
You're doing far too much open, do something, close, open. Don't do that, because it's almost certain that is the cause of your problem. If the data is already being displayed, the database is open already. If you want it to keep being displayed, the database has to remain open.
I also removed your try..except. You can put it back in if you'd like; I personally like to allow the exception to occur so that I can find out why the database operation failed from the exception message, rather than hide it and have no clue what caused it not to work.
procedure Tform1.btnSaveClick(Sender: TObject);
var
orderID: integer;
count: integer;
begin
//save into the order table first
tblOrder.Insert;
tblOrder.FieldByName('CustomerID').value:= strtoint(cboCustomer.Text);
tblOrder.Post;
orderID:=tblOrder.FieldByName('OrderID').Value;
showmessage(inttostr(orderID));
for count := 1 to nextFree-1 do
begin
if itemOrdered[count, 1] <> 0 then
begin
tblOrderLine.AppendRecord([orderID, itemOrdered[count,1],itemOrdered[count,2]]);
tblOrderLine.Post;
end;
end;
showmessage('The order has been saved');
end;

Why doesn't my query return any results when I add a WHERE clause?

My Delphi application is connected to SQLite successfully.
procedure TForm1.Button1Click(Sender: TObject);
begin
ZQuery1.Close;
ZQuery1.SQL.Clear;
ZQuery1.SQL.Text := 'SELECT Name FROM city;';
ZQuery1.Open;
while not ZQuery1.EOF do
begin
Memo1.Lines.Add(ZQuery1.FieldValues['name']);
ZQuery1.Next;
end;
end;
The above code works fine and loads contents of field name from table city.
However,
procedure TForm1.Button1Click(Sender: TObject);
begin
ZQuery1.Close;
ZQuery1.SQL.Clear;
ZQuery1.SQL.Text := 'Select name from city WHERE district = :aField';
ZQuery1.Params.ParamByName('aField').Value := 'kabol';
ZQuery1.Open;
while not ZQuery1.EOF do
begin
Memo1.Lines.Add(ZQuery1.FieldValues['name']);
ZQuery1.Next;
end;
end;
Surprisingly, when I add a where clause, the query returns nothing! Could anyone suggest what is wrong in my code?
Here is an image of the data in my table:
You probably don't have any data that has a district of kabol. The addition of the WHERE clause would then result in no rows being returned, meaning that ZQuery1.Eof is immediately true, and your while not ZQuery1.Eof do loop never gets entered.
You can check this by changing your first query (the one that works) to something like this:
ZQuery1.SQL.Text := 'SELECT Name, District FROM City';
Then change the output to
Memo1.Lines.Add(ZQuery1.FieldValues['name'] + #9 +
ZQuery1.FieldValues['district']);
If you don't see at least one line in the memo that contains kabol in the rightmost column, you don't have any rows that match your WHERE criteria. (Note that most databases are case-sensitive, so kabol is not equal to Kabol; the first would match your WHERE, but the second would not.)
Your screenshot shows one database row where district is 'Kabol' (uppercase K), but your SQL query is looking for 'kabol' (lowercase k) instead. Assuming the query is comparing strins case-sensitively, that would explain why no row is found. So either fix the case in your query input, or else perform a case-insensitive query instead.

SDAC -RecordCount and FetchAll

I am using SDAC components to query a SQL Server 2008 database. It has a recordcountproperty as all datasets do and it also has the FetchAll property (which I think it is called packedrecords on clientdatasets). Said that, I got a few questions:
1 - If I set FetchAll = True the recordcount property returns ok. But in this case, when I have a large database and my query returns a lot of lines, sometimes the memory grows a lot (because it is fetching all data to get the recordcount of course).
2 - If I set FetchAll = False, the recordcount returns -1 and the memory does not grow. But I really need the recordcount. And I also wanna create a generic function for this, so I dont have to change all my existent queries.
What can I do to have the recordcount working and the memory usage of the application low in this case?
Please, do not post that I dont need recordcount (or that I should use EOF and BOF) because I really do and this is not the question.
I thought about using a query to determine the recordcount, but it has some problems since my query is going to be executed twice (1 for recordcount, 1 for data)
EDIT
#Johan pointed out a good solution, and it seems to work. Can anybody confirm this? I am using 1 TMSCconnection for every TMSQuery (because i am using threads), so I dont think this will be a problem, will it?
MSQuery1.FetchAll := False;
MSQuery1.FetchRows := 10;
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
ShowMessage(IntToStr(MSQuery1.RecordCount)); //returns 10
MSQuery1.Close;
MSQuery2.SQL.Text := 'SELECT ##rowcount AS num_of_rows';
MSQuery2.Open;
ShowMessage(MSQuery2.FieldByName('num_of_rows').AsString); //returns 289
EDIT 2*
MSQuery1 must be closed, or MSQuery2 will not return the num_of_rows. Why is that?
MSQuery1.FetchAll := False;
MSQuery1.FetchRows := 10;
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
ShowMessage(IntToStr(MSQuery1.RecordCount)); //returns 10
//MSQuery1.Close; <<commented
MSQuery2.SQL.Text := 'SELECT ##rowcount AS num_of_rows';
MSQuery2.Open;
ShowMessage(MSQuery2.FieldByName('num_of_rows').AsString); //returns 0
Run your query as normal, than close the query
MSQuery1.SQL.Text := 'select * from cidade';
MSQuery1.Open;
MSQuery1.Close;
You need the close otherwise SQL-server has not closed the cursor yet, and will not register the query as 'completed'.
and run the following query right afterwards:
SELECT ##rowcount AS num_of_rows
This will select the total number of rows your last select read.
It will also select the number of rows your update/delete/insert statement affected.
See: http://technet.microsoft.com/en-us/library/ms187316.aspx
Note that this variable is per connection, so queries in other connections do not affect you.
I use ODAC and I believe SDAC inherits from the same base classes and works the same way as ODAC. In ODAC, there is an option called QueryRecCount under Options in your query component. Look for TCustomDADataSet.Options.QueryRecCount in your help file.
Setting QueryRecCount = True and FetchAll = False will reduce your memory usage and give you the record count. But SDAC will run a second query in the background to get the record count so it does add a little bit of extra time to your query.
Take a look at the Devart forum entry at http://www.devart.com/forums/viewtopic.php?t=8143.

Delphi: "Parameter object is improperly defined. Inconsistent or incomplete information was provided."

I'm trying to insert a record into a table in a 3-tier database setup, and the middle-tier server generates the error message above as an OLE exception when it tries to add the first parameter to the query.
I've Googled this error, and I find the same result consistently: it comes from having a colon in a string somewhere in your query, which b0rks ADO's SQL parser. This is not the case here. There are no spurious colons anywhere. I've checked and rechecked the object definition against the schema for the table I'm trying to insert into. Everything checks out, and this has my coworkers stumped. Does anyone know what else could be causing this? I'm at my wits' end here.
I'm using Delphi 2007 and SQL Server 2005.
I can get this error, using Delphi 2007 and MSSQL Server 2008, and I found a workaround. (which is pretty crappy IMHO, but maybe its useful to you if yours is caused by the same thing.)
code to produce the error:
with TADOQuery.Create(nil)
do try
Connection := ADOConnection;
SQL.Text := ' (SELECT * FROM Stock WHERE InvCode = :InvCode ) '
+' (SELECT * FROM Stock WHERE InvCode = :InvCode ) ';
Prepared := true;
Parameters.ParamByName('InvCode').Value := 1;
Open; // <<<<< I get the "parameter object is...etc. error here.
finally
Free;
end;
I found two ways to fix it:
1) remove the brackets from the SQL, ie:
SQL.Text := ' SELECT * FROM Stock WHERE InvCode = :InvCode '
+' SELECT * FROM Stock WHERE InvCode = :InvCode ';
2) use two parameters instead of one:
with TADOQuery.Create(nil)
do try
Connection := ADOConnection;
SQL.Text := ' (SELECT * FROM Stock WHERE InvCode = :InvCode1 ) '
+' (SELECT * FROM Stock WHERE InvCode = :InvCode2 ) ';
Prepared := true;
Parameters.ParamByName('InvCode1').Value := 1;
Parameters.ParamByName('InvCode2').Value := 1;
Open; // <<<<< no error now.
finally
Free;
end;
I found this thread while searching the previously mentioned Exception message. In my case, the cause was an attempt to embed a SQL comment /* foo */ into my query.sql.text.
(I thought it would have been handy to see a comment go floating past in my profiler window.)
Anyhow - Delphi7 hated that one.
Here a late reply. In my case it was something completely different.
I tried to add a stored procedure to the database.
Query.SQL.Text :=
'create procedure [dbo].[test]' + #13#10 +
'#param int ' + #13#10 +
'as' + #13#10 +
'-- For the parameter you can pick two values:' + #13#10 +
'-- 1: Value one' + #13#10 +
'-- 2: Value two';
When I removed the colon (:) it worked. As it saw the colon as a parameter.
I just encountered this error myself. I'm using Delphi 7 to write to a 2003 MS Access database using a TAdoQuery component. (old code) My query worked fine directly in MS Access, but fails in Delphi through the TAdoQuery object. My error came from a colon (apologies to the original poster) from a date/time value.
As I understand it, Jet SQL date/time format is #mm/dd/yyyy hh:nn:ss# (0 left-padding is not required).
If the TAdoQuery.ParamCheck property is True then this format fails. (Thank you posters!) Two work-arounds are: a) set ParamCheck to False, or b) use a different date/time format, namely "mm/dd/yyyy hh:nn:ss" (WITH the double quotes).
I tested both of these options and they both worked.
Even though that double-quoted date/time format isn't the Jet date/time format, Access is pretty good at being flexible on these date/time formats. I also suspect it has something to do with the BDE/LocalSQL/Paradox (Delphi 7's native SQL and database engine) date/time format (uses double quotes, as above). The parser is probably designed to ignore quoted strings (double quotes are the string value delimiter in BDE LocalSQL), but may stumble somewhat on other non-native date/time formats.
SQL Server uses single quotes to delimit strings, so that might work instead of double quotes when writing to SQL Server tables (not tested). Or maybe the Delphi TAdoQuery object will still stumble. Turning off ParamCheck in that case may be the only option. If you plan to toggle the ParamCheck property value in code, you'll save some processing time by ensuring the SQL property is empty before enabling it, if you're not planning on parsing the current SQL.
I'm facing the same error described in your question. I've traced the error into ADODB.pas -> procedure TParameters.AppendParameters; ParameterCollection.Append(Items[I].ParameterObject).
By using breakpoints, the error was raised, in my case, by a parameter which should fill a DateTime field in the database and I've never filled up the parameter. Setting up the parameter().value:='' resolved the issue (I've tried also with varNull, but there is a problem - instead of sending Null in the database, query is sending 1 - the integer value of varNull).
PS: I know is a 'late late late' answer, but maybe somebody will reach at the same error.
If I remember well, you have to explicit put NULL value to the parameter. If you are using a TAdoStoredProc component, you should do this in design time.
Are you using any threading? I seem to remember getting this error when a timer event started a query while the ADO connection was being used for another synchronous query. (The timer was checking a "system available" flag every minute).
Have you set the DataType of the parameter or did you leave it as ftUnknown?
I have also had the same problem, but with a dynamic command (e.g. an Update statement).
Some of the parameters could be NULL.
The only way i could get it working, was setting the parameter.DataType := ftString and parameter.Size := 1 and not setting the value.
cmdUpdate := TADOCommand.Create(Self);
try
cmdUpdate.Connection := '**Conections String**';
cmdUpdate.CommandText := 'UPDATE xx SET yy = :Param1 WHERE zz = :Param2';
cmdUpdate.Parameters.ParamByName('Param2').Value := WhereClause;
if VarIsNull(SetValue) then
begin
cmdUpdate.Parameters.ParamByName('Param1').DataType := ftString;
cmdUpdate.Parameters.ParamByName('Param1').Size := 1;
end else cmdUpdate.Parameters.ParamByName('Param1').Value := SetValue;
cmdUpdate.Execute;
finally
cmdUpdate.Free;
end;
I just ran into this error today on a TADOQuery which has ParamCheck := False and has no colons in the SQL.
Somehow passing the OLECMDEXECOPT_DODEFAULT parameter to TWebBrowser.ExecWB() was causing this for me:
This shows the problem:
pvaIn := EmptyParam;
pvaOut := EmptyParam;
TWebBrowser1.ExecWB(OLECMDID_COPY, OLECMDEXECOPT_DODEFAULT, pvaIn, pvaOut);
This does not show the problem:
pvaIn := EmptyParam;
pvaOut := EmptyParam;
TWebBrowser1.ExecWB(OLECMDID_COPY, OLECMDEXECOPT_DONTPROMPTUSER, pvaIn, pvaOut);
A single double quote in the query can also raise this error from what I just experienced and I am not using parameters at all ...
You can get this error when attempting to use a time value in the SQL and forget to wrap it with QuotedStr().
I got the same error. Turned out, that it is because a parameter of the stored procedure was declared as varchar(max). Made it varchar(4000) and error disappeared.

Resources