I've been working with a Legacy application which interacts with a database through ADODB, and most of the changes to records follow a fairly straightforward pattern of:
Create a Recordset from a query
Make various change to the recordset
call .Update on the recordset.
What I'm wondering is, with ADODB recordsets, is there anyway to extract the 'changes'. The logic which changes the recordset is scattered about, and all I need is the changes, not how it was changed...
Any suggestions for tracking changes in a recordset (in code, a trigger on the DB or similar is no use here)
I personally have never used this functionality, but the documentation states you can set rs.Filter property to adFilterPendingRecords to show records that have been changed but not sent to the server yet (only applies to batch update mode).
Or, you could iterate through all records in a recordset, and if the .Status property has the adRecModified flag set, then you can compare .Value and .UnderlyingValue of each of the fields to see whether they are different.
Related
Background: I have few models which are materialized as 'Table'. This tables are populated with wipe(Truncate) and Load. Now I want to protect my existing data in the Table if the query used to populate data is returning empty result set. How can I make sure an empty result set is not replacing my existing data in table.
My table lies in Snowflake and using dbt to model the output table.
Nutshell: Commit the transaction only when SQL statement used is returning Not empty result set.
Have you tried using dbt ref() function, which allows us to reference one model within another?
https://docs.getdbt.com/reference/dbt-jinja-functions/ref
If you are loading data in a way that is not controlled via dbt and then you are using this table - this is called a source. You can read more about this in here.
dbt does not control what you load into a source, everything else that is the T in the ELT is controlled where you reference a model via ref() function. A great example if you have a source that changes and you load it into a table and make sure that incoming data does not "drop" already recorded data is "incremental" materialization. I suggest you read more in here.
Thinking incremental takes time and practise, also it is recommended every now and then to do a --full-refresh.
You can have pre-hooks and post-hooks that can check your sources with clever macros and add dbt tests. We would really need a little bit more context of what you have and what you wish to achieve to suggest a real response.
Access front end, SQL Server back end.
Simple update query
PARAMETERS ParamTransactionID Long, ParamVoidFlag Short;
UPDATE tblTransaction
SET tblTransaction.VoidInProgressFlag = [ParamVoidFlag]
WHERE (tblTransaction.TransactionID=[ParamTransactionID]);
using the query here
Set qdef = CurrentDb.QueryDefs("qUPD-tblTransaction_VoidInProgress")
qdef.Parameters![ParamVoidFlag] = VoidFlag
qdef.Parameters![ParamTransactionID] = TransactionID
qdef.Execute dbFailOnError + dbSeeChanges
qdef.Close
gives
[Microsoft][ODBC SQL Server Driver]Query timeout expired
ODBC--update on a linked table 'tblTransaction' failed.
Editing the table directly works.
Opening the query and giving parameters works.
From the app still doesn't.
UPDATE
Deleted the view, no affect.
The old version is now getting the same failure, so it seems like it is not a code issue.
The only thing in common is the table, so it might be a minor change I made there.
I'll check and see if it is just that table or the entire database.
But seems odd that I can make the change by running the query directly, but get different results running it from code.
UPDATE 2
I thought that perhaps it was something to do with the entire database being read-only somehow, and this is just the first place it is getting hit. But no. Other forms can update their tables with no issues.
So it looks related to the specific table. But it still seems odd that I can update perfectly fine by just running the Update Query.
UPDATE 3
To make testing easier, I am running the queryfrom the main menu form, instead of going through all the forms to get to the point where it fails.
Running against the original DB schema worked. Made the same changes again, replacing NTEXT with VARCHAR(MAX), and it still works.
Back to the original table, still works.
Go back through all the forms, fails.
So the problem seems to be related to one of the forms that is open.
I'll go back through that sequence again.
Also, this explains why it works from the query and not from the form.
Sadly, I can't get to the query to run that while the form is open.
Ok, the first question is why/how did you wind up with "Short" data type for the parameter?
It should be:
PARAMETERS ParamTransactionID Long, ParamVoidFlag Bit;
UPDATE tblTransaction
SET tblTransaction.VoidInProgressFlag = [ParamVoidFlag]
WHERE (tblTransaction.TransactionID=[ParamTransactionID]);
You also state that this query works when you run it from the UI.
So, in code, then you have this:
Make sure that ALL code modules have option Explicit at the start like this:
Option Compare Database
Option Explicit
So, your code should now be:
Dim qdef as DAO.QueryDef
Set qdef = CurrentDb.QueryDefs("qUPD-tblTransaction_VoidInProgress")
qdef.Parameters![ParamVoidFlag] = VoidFlag
qdef.Parameters![ParamTransactionID] = TransactionID
qdef.Execute dbFailOnError + dbSeeChanges
So correct the data type you have for ParmaVoidFlag.
Also, check the table name in the left nave pane - is it
dbo_tblTransaction
or
tblTransaction.
Also, OPEN UP the linked table in design view - (ignore read only message). Look at the data types. You MUST have a PK defined - so check for a PK.
Next up, on sql server side, true/false fields MUST NOT have a default of null. If they do then updates will fail. So in sql server, make sure the table in question has a default of 0 set.
Last but not least?
If the query still errors out, then you need to add a timestamp (not date time) to the sql server table and re-link. This will/is required if any columns are floating point.
After you add the option explicit to the start of the code module, then also from menu do a debug->compile - make sure code compiles.
It turned out that the query failed if a particular form was open.
That form queried the same table, but with Snapshot instead of Dynaset.
I don't know why that locks the table. It has a proper key and index. But Dynaset fixes it.
Summary
In a Microsoft Access 2010 database (accdb), I have a form that dynamically loads other forms into a subform object on the main form. The forms used in the subform object are bound to ODBC pass-through queries that execute stored procedures to return recordsets. I can't figure out why I can use one sproc and it works perfectly fine, but if I bind the form to another sproc, it fails to load the subform.
Technical Walkthrough
I have two pass-through ODBC queries. qryGood and qryBad. They use identical ODBC connection strings (ODBC;DRIVER=SQL Server;SERVER=MyServer;UID=MyUser;Trusted_Connection=Yes;DATABASE=MyDatabase), and the SQL behind them is identical, but pointing to two different SQL stored procedures on the SQL 2012 database server.
qryGood source: exec spGoodProc 123456
qryBad source: exec spBadProc 123456
The SQL behind the sproc is very simple. Return records from a single table, filtering by the ID passed as a parameter. (Some will do more complex things, but I am just focusing on a simplified example here that demonstrates the problem.)
The RecordSource property of frmMySubform is set to qryBad.
The subform SourceObject is set via VBA code: sfrmMain.SourceObject = "frmMySubform" No errors are thrown at this point. While the SourceObject property now returns frmMySubform, the .Form object does not seem to be set.
I then try to reference a property on the subform: Debug.Print sfrmMain.Form.Name This fails with error 2467: The expression you entered refers to an object that is closed or doesn't exist.
I can then open frmMySubform in design view, change the RecordSource property to qryGood and it works just fine. This seems to point to a problem with spBadProc that only manifests itself when used as the RecordSource on a subform.
What I Have Tried
In an effort to troubleshoot this problem, I have used the process of elimination to narrow this down as far as I can, but I am still not understanding why the one sproc works and the other doesn't. Both return records just fine in SQL and when running the pass-through query directly. Both work fine when opening the form directly. It only becomes a problem when the form is set as a SourceObject in a subform control.
I have used sp_procedure_params_rowset to compare the parameters in the sprocs, and they are identical. I have compared the data types of the columns in SQL and there is nothing new or different in tblBad that isn't in tblGood. I have also tried profiling the SQL server while setting the form, and it seems to call the sproc just fine. I didn't see any clues when comparing the trace between the bad and the good calls.
Setting the RecordSet directly to an ODBC link to tblBad works just fine (and I presume a view would be fine as well) but having the simple stored procedure wrapper somehow triggers the error.
I have also compared the security, properties and extended properties for spGoodProc and spBadProc and they are identical.
My Question
What can I do on the troubleshooting side to reduce this down further? Has anyone out there encountered similar issues with bound sprocs on subforms? I am working on a very complex database with hundreds of forms, tables and queries, so I would really like to understand why this is occurring before I go too far down this path.
Thanks in advance for any insight you are able to share on this perplexing problem. :-)
Found it!
After tracing it back to something with the specific table, I removed all constraints, keys, and then columns from a copy of the table, systematically testing to see if I could pinpoint the problem. Sure enough, it was a specific column name in the stored procedure!
Simply aliasing this column to a different name solved the problem. (See below for expanded details)
Update after Further Testing
After additional testing to further pinpoint the issue, I think I now understand why this was occurring. When you link an ODBC table and specify a unique (key) column, Access will automatically attempt to set the LinkMasterFields and LinkChildFields to the key column name when a subform is loaded and the subform has a column with the same name. While this works fine with linked tables or views, it does not work when the RecordSource of the subform is set to a stored procedure.
If you attempt to do this by manually adding the subform, you will see the following notification:
However, if you set the subform target through VBA code, you don't get any warning or error message. It simply doesn't (fully) load the subform. #Albert D. Kallal, you were right on about this being related to a master/child fields issue!
I was able to consistently reproduce the issue in a test database file in both Access 2010 and Access 2016. If you would like to see this for yourself, you can use the following steps to reproduce it:
Create a SQL table with a PrimaryID column.
CREATE TABLE [dbo].[tblBugTest](
[PrimaryID] [int] NOT NULL,
[TestColumn] [nchar](10) NULL
) ON [PRIMARY]
Add a couple test records to the table you just created.
Create a Stored Procedure to return the records.
CREATE PROCEDURE [dbo].[spBugExample]
AS SELECT * FROM tblBugTest
Create a blank Microsoft Access database (accdb).
Using ODBC, create a linked table to tblBugTest.
Important: Select PrimaryID as the unique column.
Create a pass-through query named qryPassThrough using ODBC to the same database, and set the SQL to exec spBugExample.
Open the query to verify that it returns records.
Create three blank forms in the database. frmMain, frmSubForm, and frmBlank.
Add frmBlank as a subform to frmMain. Name the subform sfrmSubform.
Set the RecordSource of frmMain to the linked table.
Add a button to frmMain to switch the subform from frmBlank to frmSubForm.
Private Sub cmdShowBug_Click()
With Me.sfrmSubform
.SourceObject = "frmSubForm"
Debug.Print .LinkMasterFields
Debug.Print .LinkChildFields
Debug.Print .Form.Name
.SourceObject = "frmBlank"
End With
End Sub
Set the RecordSource of frmSubForm to qryPassThrough.
Drop a couple bound controls onto frmSubForm.
Test frmSubForm by itself. It should load a record from tblBugTest.
Open frmMain and click the button. It should throw an error.
If you step through the code, you will see that before setting the SourceObject, the LinkMasterFields property is blank. After setting SourceObject, you can hover over LinkMasterFields and see that it is now set to the PrimaryID column.
Workarounds
Changing any of the following will work around the error by avoiding the problematic auto-linking of the master/child fields.
Delete and relink the linked table, this time not specifying a unique column.
Alias the column in the Stored Procedure to a different name than the unique column.
Clear the RecordSource property of the parent form.
Clear the subform RecordSource property and set the RecordSet after loading the subform.
Use a view or linked table instead of a stored procedure in the subform.
Keep in mind that the subform data source will be attempted to be loaded BEFORE the main form loads. What this suggests is that on the main form's load event, you will
First setup the PT query.
Then set the OBJECT source of the sub form.
In other words, the source object of the sub form control should be blank.
You code then to set up the PT query will be:
With currentdb.queryDefs("qryGood")
.SQL = "EXEC spGoodProc " & 123456
end with
Of course you can replace the 123456 with a varible, or even a value from a text box (from the main form).
Now that the PT query is setup, you THEN are to set the form that the sub-form is to load.
So, after above code, we then have:
me.mySubForm.SourceObject = "name of subform goes here"
So, it should be about a total of 4 lines of code. And as above shows, you don't even need any connection string stuff in your VBA code.
So, just keep in mind:
Setup the PT query as per above. You can then launch a report, or even a form, or in this case set the form that the sub-form control is to load. This also suggests/hints that you need to remove the source object of the sub form control (leave it blank).
You can have the sub-from source object set, but then this would suggest that you setup the PT query source as per above BEFORE you launch the main form with the sub form based on the PT query. As noted, this set of steps is required since the sub form actually load and resolves it data source BEFORE the main form displays and renders. So, by leaving the source-object blank for the sub form, then you the developer re-gains complete control over the order of loading.
I have an Access 2010 database that's linked to a SQL Azure backend (yes, I know this isn't ideal, and it's being slowly phased out). On the backend, I have a stored procedure that I want to use to populate a read-only subform each time a new record is loaded. I'm attempting to do this by generating a recordset in VBA, then setting the subform's RecordSet property. It actually works, but with a nasty side-effect.
When I set the RecordSet property, it also seems to be setting the RecordSource property of the subform. That RecordSource is something Access can't parse, because it's meant to be a call to the backend. If I try using a DAO passthrough query to generate the recordset, the RecordSource looks like:
EXEC dbo.GetDuplicateAddressesByManufacturer N'...', N'...', N'...'
If I try using an ADO command to generate the recordset, it looks like:
{ call dbo.GetDuplicateAddressesByManufacturer ?, ?, ? }
As soon as I try to move to the next record, Access throws an error because it tries to load a new record for the subform first, and it can't open what it sees as the subform's RecordSource. If I'm trying the DAO route, it tells me "Invalid SQL Statement", and if I'm using ADO, it tells me "Data provider could not be initialized."
Anyone have an idea how I can get around this? Isn't there a way to set the RecordSet property without also setting the RecordSource? I could swear I've done that before, but maybe I've just never noticed that the RecordSource gets set too.
Failing that, is there a way I can interject some code before the Form_Current event to remove the subform's RecordSource? The code I'm using to set the RecordSet each time works great -- the problem is the error that's raised before my code works. Once I dismiss the error message, everything works fine, but obviously I don't want users to get an error message each time they change records.
If all else fails, I guess I could always use the query to populate a local temp table, but it seems like a lot of overhead to do every single time someone moves to a new record.
Why bother with a stored procedure? Just link the sub form to the table, and setup the link master/child settings. You only pull down the required records.
If the sub form is a complex query with multiple tables, then you certainly want the data joins etc. to occur server side and AGAIN simply create a view and again set the sub form source to that view (and again the link master/child settings will do all the dirty work for you).
And there no reason why you cannot create a pass-through query and SIMPLE assign that to the forms recordSource.
It does not matter what “junk” you place inside of the query, and that includes RAW T-SQL.
And while you can load up the DAO reocrdset with this pass-thought, you really don’t need to. I suppose for some happy reason you are doing this, and at least if you must, then recordSoruce becomes the name of the pass-though and NOT your raw T-SQL anyway.
However, really, just dump all that recordset junk, and just go:
Me.MySubForm.Form.RecordSource = "my pass though query".
Thus above is only one line.
You doing all these hand stands to increase performance then at the end of the day why does your main form allow navigation? You should build a search screen, display the results, let user pick a row and THEN launch the main form to edit/display the ONE record along with the sub form data.
When they close, they are back to searching for the next customer etc. This approach also thus solves your messy navigation issues. It also why the web and most software works this way (it reduces bandwidth issues).
However, if you must have navigation, and for some reason CANNOT use a view and cannot let Access use the link master/child settings to do your dirty work?
Then in the forms on-current event you can modify the pass-though and simply re-assign it to the sub form.
Eg:
With CurrentDb.QueryDefs("qPass")
.SQL = "select * from FaxBook3 where id = 3"
End With
'
Me.RecordSource = "qpass"
Now how in the above I am using RAW T-SQL in the pass-though query, and then simply assign the pass-though to the forms recordsource (in your case you assign to the sub form.
Me.MySubForm.Form.RecordSource = above
And there NO reason why the above .SQL cannot be your stored procedure
.SQL = "Exec your-storedProcedure " & strVbaStringParmater
And again assign the form (or sub form) recordSource.
So you REALLY do not need to create some reocrdset in code and it not yield you any performance increase, but will cause you to write more code and have problems as you outlined in your post.
I have an access application that has a form that allows the user to enter case notes. The main field of this form is tied to a SQL Server varchar(MAX) field in the source table. Since the users switched to Access 2007, their program keeps crashing when they are on the case notes form. As a possible solution to this problem, I would like to try unbinding this form and re-building it as an unbound form.
This form needs to be able to add and update records into my SQL Server database. It also needs to be able to browse between records. I guess I am at a loss as to where to start. Any suggestions/code snippets is appreciated.
As a starting point, try Google on "unbound form in Access". Don't be distracted by PacMan! ;)
Anyway, the basic idea of an unbound form is to load the data into the unbound controls from a recordset, then to save it back when edits are done. This means you need these things:
controls to select the needed record, some kind of find functionality.
code to open the recordset and write the data from the fields into the corresponding controls on the form.
controls to save the record back to the database, which will use a SQL update to write the values in the unbound controls back to the database. I prefer not to update fields that have not changed (because I do lots of replicated Jet apps, and multiple updates can lead to unnecessary replication conflicts). You can compare the data in the undound controls to the data in the original recordset (if you open it as a snapshot type recordset, it won't reflect any updates since it was opened), and write your SQL UPDATE for only the fields where the values don't match. You'll have to account for Nulls.
The common practice is to name the controls exactly the same as the fields they correspond with so you can loop the recordset's fields collection and load the data into the controls:
For Each fld In rs.Fields
Me.Controls(fld.Name) = fld.Value
Next fld
You can do likewise for saving the data and checking the control values against the original recordset values.
I don't know if this works with SQL Server VarChar() fields or not, but you could also try what I call a "semi-bound" form, where you load the recordset with the form's RecordSource property but don't bind the fields to controls. Thus, the form is bound, but the controls are not. I very often do this with a fully bound form where I make the memo fields unbound (to avoid the danger of memo field pointer corruption in Jet/ACE back ends). In that case, with a bound form recordset and an unbound textbox for editing, you'd do this:
in the form's OnCurrent event, load the unbound field(s) data into the corresponding unbound textbox(es).
in the AfterUpdate event(s) of the unbound control(s), write the data in the unbound textbox(es) back to the recordsource.
Those two steps would basically look something like this:
Private Sub Form_Current()
Me!txtMemo = Me!Memo
End Sub
Private Sub txtMemo_AfterUpdate()
Me!Memo = Me!txtMemo
Me.Dirty = False
End Sub
With a Jet/ACE back end, you'd want to save the record immediately after you write the memo field value because otherwise you haven't avoided the danger of corrupting the memo field pointer. With a SQL Server back end, you may or may not need to do that, since the issues are completely different. Saving will release the write lock, but you might not need to avoid that.
Also, I'm assuming that the VarChar() data can be read from the form's underlying Recordsource and written into the textbox. You'll have to see if that works.