Speeding up an Access Database - database

I have an Access database to report on event statistics gathered from a mainframe system. The mainframe scheduler (ZEKE) doesn't have robust reporting features, so I export daily event data to report on.
A master listing from a separate source (a static list that will not change on a regular basis) lists the individual applications, including the application code (which is the naming standard for production runs) and the name of the programmer, coordinator, manager, business unit, etc. for that application.
The user can search by any field, application code, programmer, coordinator, etc.
Choose the production center to search in (there are 5) or default to all, and choose all dates, a single date, or a date range.
The query takes the search parameters and starting with either the application code, or the person, searches the table for applications and copies records to a temp table for reporting.
For example, to see how many failures the application coordinator John Doe had for the past week for all of the applications he is responsible for, the query would move all application records listing John Doe as the coordinator to the temp table.
From there, it moves through the temp table for each application and searches the event data for events under that application code which meet the criteria entered for date, production center and event type (success, failure or both).
This is moved to a temp table for the final report.
The table for event data is currently 2.5 million lines (this is 15 days worth of data) and is growing daily.
I put the back end onto a newly created NAS drive on our network.
A report that took two minutes when the back end and front end were on the same machine now takes 29 minutes.
Any suggestions to streamline the queries over a network?
Code which is run from the report criteria selection form and runs the report.
'this macro will generate a report based on multiple input criteria.
'this report allows the user to slect:
' date range, single date or all dates
' type of events: Abends, Successes or both
' centers to pull data from: OCC,QCC,BCC,ITS,DAIN, or ALL centers
' The type of data to report on: App code, App Coordinator, Custodian, L3, L4 or L5
'Once the user has selected all of the required data and fields, the report will be generated
'based on the selection criteria.
'we begin by defining the active database as the currently open database
Dim db As DAO.Database
Set db = DBEngine(0)(0)
On Error GoTo ErrorHandler
'Now we designate the variables which will be used in this macro
Dim strSQ1 As String
Dim strSQ2 As String
Dim strSQ3 As String
Dim strSQ4 As String
Dim appl As String
Dim evstatus As String
Dim appletype As String
Dim fullapp As String
Dim length As Long
Dim iipmname As String
Dim iipmcoor As String
Dim fullappnm As String
Dim fullappcoor As String
Dim kinddate As String
Dim coor As String
Dim cust As String
Dim appL3 As String
Dim appL4 As String
Dim appL5 As String
Dim ctrOCC As String
Dim ctrMTL As String
Dim ctrBCC As String
Dim ctrITS As String
Dim ctrDAIN As String
'We will start by setting some default values
'We will ste the default values for center selection.
'We start by searching for terms we know are not there, then change them to
'valid search terms if the center is selected.
ctrOCC = "notOCC"
ctrMTL = "notMTL"
ctrBCC = "notBCC"
ctrITS = "notITS"
ctrDAIN = "notUSWM"
fullapp = "*"
'First we determine which event types the user wants to look for
state = Me![opt-status].Value
If state = 1 Then
evstatus = " [ev-status] = 'AEOJ'"
ElseIf state = 2 Then
evstatus = " [ev-status] = 'EOJ'"
ElseIf state = 3 Then
evstatus = " ([ev-status] = 'EOJ' OR [ev-status] = 'AEOJ')"
End If
'MsgBox "Event status pulled is:.. " & evstatus & "."
' Next up we will configure the date parameters based on the user input
If [grp-datesel] = 1 Then
Sdte = "1"
Edte = "9999999"
kinddate = "[ev-date] >= " & Sdte & " AND [ev-date] <= " & Edte & " "
End If
If [grp-datesel] = 2 Then
'error handling
If IsNull(Me.[sel-onedate]) Then
MsgBox "You have not entered a date to search....please try again."
Me.[sel-onedate] = Null
Me.[sel-onedate].SetFocus
Exit Sub
End If
'end of error handling
Dim currdte As Date
currdte = Me![sel-onedate].Value
currjul = Format(currdte, "yyyyy")
daycurr = CDbl(currjul)
Sdte = daycurr
Edte = daycurr
kinddate = "[ev-date] >= " & Sdte & " AND [ev-date] <= " & Edte & " "
End If
If [grp-datesel] = 3 Then
'error handling
If IsNull(Me.[sel-Sdate]) Or IsNull(Me.[sel-Edate]) Then
MsgBox "You Must enter a start and end date for the search....please try again."
Me.[sel-Sdate] = Null
Me.[sel-Edate] = Null
Me.[sel-Sdate].SetFocus
Exit Sub
End If
'end of error handling
Dim startdte As Date
Dim enddte As Date
startdte = Me.[sel-Sdate].Value
enddte = Me.[sel-Edate].Value
startjul = Format(startdte, "yyyyy")
endjul = Format(enddte, "yyyyy")
Sday = CDbl(startjul)
Eday = CDbl(endjul)
Sdte = Sday
Edte = Eday
'MsgBox "start date is " & Sdte & " and end date is " & Edte & "."
'check that dates are in proper chronological order
If Sdte > Edte Then
MsgBox "The start Date you entered is after the end date....please try again."
Me.[sel-Sdate] = Null
Me.[sel-Edate] = Null
Me.[sel-Sdate].SetFocus
Exit Sub
End If
'keep going if it's all good
kinddate = "[ev-date] >= " & Sdte & " AND [ev-date] <= " & Edte & " "
End If
MsgBox "Date used is:.. " & kinddate & "."
'Now lets look at center selection
If [chk-allctr].Value = True Then
ctrOCC = "OCC"
ctrMTL = "MTL"
ctrBCC = "BCC"
ctrITS = "ITS"
ctrDAIN = "USWM"
End If
If [chk-OCC].Value = True Then
ctrOCC = "OCC"
End If
If [chk-MTL].Value = True Then
ctrMTL = "MTL"
End If
If [chk-BCC].Value = True Then
ctrBCC = "BCC"
End If
If [chk-RTF].Value = True Then
ctrITS = "ITS"
End If
If [chk-DAIN].Value = True Then
ctrDAIN = "DAIN"
End If
'Error handling if no center is selected
If [chk-OCC].Value = Flase Then
If [chk-MTL].Value = Flase Then
If [chk-BCC].Value = Flase Then
If [chk-RTF].Value = Flase Then
If [chk-DAIN].Value = Flase Then
MsgBox "You have not selected a center to search search....please try again."
Me.[chk-allctr].SetFocus
Exit Sub
End If
End If
End If
End If
End If
'end of error handling
'MsgBox "centers used are: Chr(10) " & ctrOCC & " Chr(10) " & ctrBCC & " Chr(10) " & ctrMTL & " Chr(10) " & ctrITS & " Chr(10) " & ctrDAIN & " For this run"
'All good so far, now we will parse the application code if an
'application code report is selected
appl = "*"
If [opt-criteria].Value = 1 Then
'error handling
If IsNull(Me.[sel-appcode]) Then
MsgBox "You have not entered an application code to search....please try again."
Me.[sel-appcode] = Null
Me.[sel-appcode].SetFocus
Exit Sub
End If
'end of error handling
End If
If [opt-criteria].Value = 1 Then
appl = Me![sel-appcode].Value
End If
'trust = "no"
'If Mid(appl, 3, 2) = "RT" Then trust = "yes"
'length = Len(appl)
'If length = 2 Then appltype = "short"
'If length = 3 Then appltype = "long"
'If appltype = "short" Then fullapp = "" & appl & "00"
'If appltype = "long" Then fullapp = "" & appl & "0"
'If trust = "yes" Then fullapp = appl
'End If
fullapp = appl
'MsgBox "App to use is: " & appl & " fullapp code is " & fullapp & "."
'Now we set values if names are used
coor = "*"
cust = "*"
appL3 = "*"
appL4 = "*"
appL5 = "*"
If [opt-criteria].Value = 2 Then
'error handling
If IsNull(Me.[sel-coor]) Then
MsgBox "You have not entered a Coordinator to search....please try again."
Me.[sel-coor] = Null
Me.[sel-coor].SetFocus
Exit Sub
End If
'end of error handling
coor = Me![sel-coor].Value
'MsgBox "Coordinator report selected for: " & coor & "."
End If
If [opt-criteria].Value = 3 Then
'error handling
If IsNull(Me.[sel-custodian]) Then
MsgBox "You have not entered a Custodian to search....please try again."
Me.[sel-custodian] = Null
Me.[sel-custodian].SetFocus
Exit Sub
End If
'end of error handling
cust = Me![sel-custodian].Value
'MsgBox "Custodian report selected for: " & cust & "."
End If
If [opt-criteria].Value = 4 Then
'error handling
If IsNull(Me.[sel-L3]) Then
MsgBox "You have not entered an L3 to search....please try again."
Me.[sel-L3] = Null
Me.[sel-L3].SetFocus
Exit Sub
End If
'end of error handling
appL3 = Me![sel-L3].Value
'MsgBox "L3 report selected for: " & appL3 & "."
End If
If [opt-criteria].Value = 5 Then
'error handling
If IsNull(Me.[sel-L4]) Then
MsgBox "You have not entered an L4 to search....please try again."
Me.[sel-L4] = Null
Me.[sel-L4].SetFocus
Exit Sub
End If
'end of error handling
appL4 = Me![sel-L4].Value
'MsgBox "L4 report selected for: " & appL4 & "."
End If
If [opt-criteria].Value = 6 Then
'error handling
If IsNull(Me.[sel-L5]) Then
MsgBox "You have not entered an L5 to search....please try again."
Me.[sel-L5] = Null
Me.[sel-L5].SetFocus
Exit Sub
End If
'end of error handling
appL5 = Me![sel-L5].Value
'MsgBox "L5 report selected for: " & appL5 & "."
End If
'Most of these reports take a while to build with this macro, so to make sure the user
'knows that the macro is still working, we didsplay a splash screen. It's cute and has
'hamsters, cause everyone loves hamsters.
DoCmd.OpenForm "PlsWaitFrm", acWindowNormal
[Forms]![PlsWaitFrm].Repaint
'All of out criteria values are now selected. We can move on to pulling data from the tables.
'We start by populating the IIPM table with the information that we require for applications.
strSQ1 = "DELETE * from [tbl-RPT-IIPM] "
db.Execute strSQ1
strSQ2 = "INSERT INTO [tbl-RPT-IIPM] " & _
"SELECT * FROM [tbl-IIPM] " & _
"WHERE (([AppCode] like '" & fullapp & "')" & _
"AND ([AppCoordinator] like '" & coor & "') " & _
"AND ([AppCustodian] like '" & cust & "') " & _
"AND ([L3] like '" & appL3 & "') " & _
"AND ([L4] like '" & appL4 & "') " & _
"AND ([L5] like '" & appL5 & "')) "
db.Execute strSQ2
'MsgBox "made it past the populate of rpt-iipm"
'Now we have populated the IIPM report table, it's time to populate the event report table.
'We will loop through all fields in the IIPM report table and pull information for each
'application code.
strSQ3 = "DELETE * from [tbl-EVENTREPORT] "
db.Execute strSQ3
Dim rs As DAO.Recordset
Set db = CurrentDb
Set rs = db.OpenRecordset("tbl-RPT-IIPM") 'this opens the IIPM report table just populated
'populate the table
rs.MoveLast
rs.MoveFirst
Do While Not rs.EOF
'we will execute these action against the selected record.
'first step - parse the application code to display the full application code
appl = rs![AppCode].Value
length = Len(appl)
If length = 1 Then appl = "" & appl & "00"
rptdelin = Mid(appl, 3, 1)
rptcode = Mid(appl, 1, 3)
If rptdelin = "0" Then rptcode = Mid(appl, 1, 2)
If rptdelin = "R" Then rptcode = "RT" & Mid(appl, 1, 2) & ""
'MsgBox "searching for: " & rptcode & "."
applist = applist & "," & appl
strSQ4 = "INSERT INTO [tbl-EVENTREPORT] " & _
"SELECT * FROM [tbl-EVENT DATA] " & _
"WHERE (([ev-jobname] LIKE '?" & rptcode & "*') " & _
"AND (([ev-ctr] = '" & ctrOCC & "')" & _
"OR ([ev-ctr] = '" & ctrMTL & "')" & _
"OR ([ev-ctr] = '" & ctrBCC & "')" & _
"OR ([ev-ctr] = '" & ctrITS & "')" & _
"OR ([ev-ctr] = '" & ctrDAIN & "'))" & _
"AND (" & kinddate & ") " & _
"AND " & evstatus & ")"
db.Execute strSQ4
'now we're done with this report, we move on to the next
rs.MoveNext 'press Ctrl+G to see debuG window beneath
Loop
'END OF LOOPING CODE
'MsgBox "made it past the looping"
'Now we have completed populating the table that the report will be based on.
'Next step is to gather master statistics to produce abend and success percentages.
totfail = DCount("[ev-status]", "tbl-EVENTREPORT", "[ev-status] = 'AEOJ'")
totsucc = DCount("[ev-status]", "tbl-EVENTREPORT", "[ev-status] = 'EOJ'")
Dim allabend As Long
Dim allsucc As Long
allabend = DCount("[ev-status]", "[tbl-EVENT DATA]", "[ev-status] = 'AEOJ' AND ([ev-date] >= " & Sdte & " AND [ev-date] <= " & Edte & ")")
allsucc = DCount("[ev-status]", "[tbl-EVENT DATA]", "[ev-status] = 'EOJ' AND ([ev-date] >= " & Sdte & " AND [ev-date] <= " & Edte & ")")
Dim pctabend As Long
Dim pctsucc As Long
pctabend = (totfail / allabend) * 100
pctsucc = (totsucc / allsucc) * 100
'Now we will generate the reports for display based on what type of report was selected
'by the user in the initial form.
'Before we open the report, we will close the splash screen
DoCmd.Close acForm, "PlsWaitFrm", acSaveNo
'Now we open the report
If [opt-criteria].Value = 1 Then
fullappnm = DLookup("AppName", "tbl-RPT-IIPM", "AppCode = '" & fullapp & "' ")
fullappcoor = DLookup("AppCoordinator", "tbl-RPT-IIPM", "AppCode = '" & fullapp & "' ")
DoCmd.OpenReport "rpt-APPLREPORT", acViewReport
[Reports]![rpt-APPLREPORT]![rpt-appcode].Value = fullapp
[Reports]![rpt-APPLREPORT]![rpt-appname].Value = fullappnm
[Reports]![rpt-APPLREPORT]![rpt-appcoor].Value = fullappcoor
[Reports]![rpt-APPLREPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-APPLREPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-APPLREPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-APPLREPORT]![rpt-succpct].Value = pctsucc
End If
If [opt-criteria].Value = 2 Then
DoCmd.OpenReport "rpt-COORREPORT", acViewReport
[Reports]![rpt-COORREPORT]![rpt-appcode].Value = applist
[Reports]![rpt-COORREPORT]![rpt-appcoor].Value = coor
[Reports]![rpt-COORREPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-COORREPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-COORREPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-COORREPORT]![rpt-succpct].Value = pctsucc
End If
If [opt-criteria].Value = 3 Then
DoCmd.OpenReport "rpt-CUSTREPORT", acViewReport
[Reports]![rpt-CUSTREPORT]![rpt-appcode].Value = applist
[Reports]![rpt-CUSTREPORT]![rpt-appcoor].Value = cust
[Reports]![rpt-CUSTREPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-CUSTREPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-CUSTREPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-CUSTREPORT]![rpt-succpct].Value = pctsucc
End If
If [opt-criteria].Value = 4 Then
DoCmd.OpenReport "rpt-L3REPORT", acViewReport
[Reports]![rpt-L3REPORT]![rpt-appcode].Value = applist
[Reports]![rpt-L3REPORT]![rpt-appcoor].Value = appL3
[Reports]![rpt-L3REPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-L3REPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-L3REPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-L3REPORT]![rpt-succpct].Value = pctsucc
End If
If [opt-criteria].Value = 5 Then
DoCmd.OpenReport "rpt-L4REPORT", acViewReport
[Reports]![rpt-L4REPORT]![rpt-appcode].Value = applist
[Reports]![rpt-L4REPORT]![rpt-appcoor].Value = appL4
[Reports]![rpt-L4REPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-L4REPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-L4REPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-L4REPORT]![rpt-succpct].Value = pctsucc
End If
If [opt-criteria].Value = 6 Then
DoCmd.OpenReport "rpt-L5REPORT", acViewReport
[Reports]![rpt-L5REPORT]![rpt-appcode].Value = applist
[Reports]![rpt-L5REPORT]![rpt-appcoor].Value = appL5
[Reports]![rpt-L5REPORT]![rpt-abendtot].Value = totfail
[Reports]![rpt-L5REPORT]![rpt-succtot].Value = totsucc
[Reports]![rpt-L5REPORT]![rpt-abdpct].Value = pctabend
[Reports]![rpt-L5REPORT]![rpt-succpct].Value = pctsucc
End If
ErrorHandler:
If Err.Number = 7874 Then
Resume Next 'Tried to delete a non-existing table, resume
End If
End Sub
'''

Firstly, you need to work out where the bottlenecks are, so I would suggest putting some Debug.Print Now statements throughout the code to give you an idea of what is causing the issue.
I would guess that two of the processes that take most of the time are the DELETE/INSERT statements that you are doing.
I would suggest that rather than doing this, you look at normalizing your database, and then creating a query that provides the information that you need.
Also, by running the report directly from a query rather than a temporary table means that you don't have to worry about the deletes/inserts creating database bloat.
If you really insist on keeping this process, then consider deleting the table [tbl-RPT-IIPM] and then recreating it, rather than deleting the records. And consider removing the indexes before the insert, and then adding them back afterwards, as indexes splow down inserts, but obviously speed up searches and joins.
Also, when you are inserting data into [tbl-RPT-IIPM], you are using ([L3] like '" & appL3 & "'), which is the same as ([L3]='" & appL3 & "'), but slower.
When you are inserting data into [tbl-EVENTREPORT], you are doing it when looping through a recordset - it may be faster to use an INSERT SQL statement.
Regards,

OK, with some more information, some more answers that may (or may not!!) help. Again, you will need to run timing tests to see which works best for you.
Try adding a "Yes/No" field to the table [tbl-EVENT DATA]. You can then use an UPDATE statement to indicate which fields to include in the report, rather than using the slow INSERT query.
Another thing to try would be to replace the INSERT statement with several, each using a different value for [ev-ctr]. Or else rather than using OR try using IN:
strSQ4 = "INSERT INTO [tbl-EVENTREPORT] " & _
"SELECT * FROM [tbl-EVENT DATA] " & _
"WHERE [ev-jobname] LIKE '?" & rptcode & "*' " & _
"AND [ev-ctr] IN('" & ctrOCC & "','" & ctrMTL & "','" & ctrBCC & "','" & ctrITS & "','" & ctrDAIN & "')" & _
"AND " & kinddate & _
"AND " & evstatus
Also, I notice that kinddate is set to effectively include all dates in one instance, and that evstatus is set to include both "EOJ" and "AEOJ" in one instance. Rather than including these fields as criteria in these cases, you may wish to not include them at all:
If state = 1 Then
evstatus = " AND [ev-status] = 'AEOJ'"
ElseIf state = 2 Then
evstatus = " AND [ev-status] = 'EOJ'"
ElseIf state = 3 Then
evstatus = " "
End If
And then you would rewrite " AND " & evstatus to & evstatus in the SQL statement.
A final thing to look at is to actually run the INSERT directly in the backend, rather than operating on linked tables in the frontend, as Access will be dragging vast amounts of data across the network and then sending it back. As a basic guide, something like this:
Sub sUpdateQuery()
Dim objAccess As New Access.Application
objAccess.OpenCurrentDatabase "J:\downloads\test.accdb"
objAccess.DoCmd.RunSQL "UPDATE test2 SET Field1=UCASE(Field1);"
objAccess.CloseCurrentDatabase
Set objAccess = Nothing
End Sub
Regards,

Applecore, Firstly, let me thank you for your insights. Unfortunately due to the nature of the way the data is processed, some of them I'm not sure I can implement. I have used debug.print statements to get a better idea of the timing.
You are correct, the INSERT statement is causing me the most problems, and only the second one. The deletes fly through almost instantly, no issues there. It's the second insert from the event data that is slowing it down.
I have been thinking about this since inception of how to nromalize more efficiently and create better relationships, but I'm stymied. My issue is, the data between the event table and the event table are related "in the world" but in no clear way in terms of data. There is no way to determine the relationship without a complex calculation. For example, the unique part of the application data is the application code. They are always unique. A single Application coordinator can have dozens of codes assigned to them, as can custodians, L3,L4, etc. Each event is related to an application, however, there is no specific field that is exported that tells the application code, it is obtained by parsing the event name (And yes, that is as archaic as it sounds). The event naming standards are standard mainframe 8 character names: .
For example PGRD1234 - Production job, GRD application, 1234 as the designator. So to determine what application the job is related to, I take the application code, and select LIKE with wildcards. It's not 100% accurate I am well aware, but to use wildcards, I seem to be stuck using LIKE. I haven't been able to make '=' work with wildcards. Can you?
You also mentioned "When you are inserting data into [tbl-EVENTREPORT], you are doing it when looping through a recordset - it may be faster to use an INSERT SQL statement." which I'm not sure what you are saying.. My apologies. I don't think I am understanding it. I think that is what I am doing now. I use the IIPM table to obtain the listing of the application codes I need to pull, then loop through that recordset to pull all of the event data for only those applications. As there is no direct correlation between the data, I Can't think of another way to do it.

Related

Doing stuff for each record in a recordset

I'm working on some code that checks a database for a specific value and returns a string under certain conditions.
I cannot get this code to iterate to the next row in my database results. It appears that my code only checks the first record of the recordset and then continues.
Sometimes my recordset may only have one row, but sometimes it may have many rows. If any of the rows of this recordset have a certain value I want to throw a MsgBox. Not just the first or last record.
Here's my code:
'Database Connection Strings.
strServerName = "string"
strDatabase = "string"
strUserName = "string"
strPassword = "string"
'Connection string for SQL Server.
strConn = "Driver={SQL Server};Server=" & strServerName & ";Database=" & strDatabase & ";Uid=" & strUsername & ";Pwd=" & strPassword & ";"
'Create required objects for the session.
Set WShell = CreateObject("WScript.Shell")
Set db = CreateObject("ADODB.Connection")
Set rs = CreateObject("ADODB.Recordset")
'Create the array of reciever lines and count them.
Set arrayLine = LINES.Value
intNumOfLines = arrayLine.Count
intLineNum = Cint(intNumOfLines)-1
cleanPN2 = array()
skipArray = array("-", " ", "PLATE", "HEAT-TREAT", "PAINT", "MACHINE", "WELD", "MPI")
result = MsgBox ("Scan PO for cert requirements?", vbYesNo, "PO Requirement Scanner")
Select Case result
Case vbYes
'Iterate through the reciever lines and look for part numbers.
For intLineNum = 0 To intNumOfLines
If intLineNum = intNumOfLines Then
Exit For
End If
Set arrayLine = LINES(intLineNum).Value
strPN = arrayLine("VENDOR_PART_ID")
cleanPN = split(strPN, " ")
For Each iteration in cleanPN
iteration = LTrim(RTrim(iteration))
ReDim Preserve cleanPN2(UBound(cleanPN2) + 1)
cleanPN2(UBound(cleanPN2)) = iteration
Next
Next
'Take any part numbers that were found and search the WO Master for operations that require certs.
For Each cleanPN3 In cleanPN2
strSQL = "SELECT USER_3 FROM OPERATION WHERE WORKORDER_BASE_ID = " & "'" & cleanPN3 & "';"
db.Open strConn, db
rs.Open strSQL, db
If Not rs.EOF And Not rs.BOF Then
strUSER3 = rs("USER_3")
Do While rs("USER_3") = Null
strUSER3 = rs("USER_3").MoveNext
Loop
If (strUSER3 <> Null) Or (strUSER3 <> "") Then
MsgBox "Certifications are required for part number " & cleanPN3 & "!", vbOKOnly
End If
End If
rs.Close
db.Close
Next
MsgBox "PO Scan Complete!"
Case vbNo
MsgBox("PO Scan Cancelled!")
End Select
Assuming everything else is working as you wish
rs.Open strSQL, db
If Not rs.BOF Then
rs.MoveFirst
Do While Not rs.EOF
If Len(Trim(Cstr(rs.fields("USER_3").value)) > 0 Then
MsgBox "Certifications are required for part number " & cleanPN3 & "!", vbOKOnly
End If
rs.MoveNext
Loop
End If
rs.Close

create an array from access table vba

I am trying to pull a list of names and email addresses from a tables in access and use that information to send out personalized emails. So I am trying to call 2 columns from access into an array and then using a loop to go through each row. I think this will be the best way to do it but i am open to other ideas. I have been able to generate code that produces one email with the first record, but i have no idea how to go about getting the rest of the records:
Private Sub SMT()
Application.Run "VariablesForRanges"
strDB1 = ThisWorkbook.Path & "\database.accdb"
strqry1 = "SELECT table1.name, table2.Email"
strqry1 = strqry1 & " FROM table2 inner join table1 on table1.FULL_NAME=table2.Name"
strqry1 = strqry1 & " group by table1.name, table2.Email;"
Set cn1 = New ADODB.Connection
cn1.Open "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & strDB1 & ";"
Set rs1 = New ADODB.Recordset
With rs1
Set .ActiveConnection = cn1
.Open strqry1
End With
If rs1.BOF = True Then
MsgBox "There Are No Records To Import", vbInformation
Else
Set OutApp = CreateObject("Outlook.Application")
Set OutMail = OutApp.CreateItem(0)
strbody = "Hi " & rs1("name") & "," & vbNewLine & vbNewLine & _
"test" & vbNewLine & vbNewLine & _
"Best regards," & vbNewLine & _
On Error Resume Next
With OutMail
.To = rs1("Email")
End With
On Error GoTo 0
Set OutMail = Nothing
Set OutApp = Nothing
End If
End Sub
Any help with creating an array and loop would be appreciated. Thanks.
Why not just loop through the record set?
Do While Not rs1.EOF
' create emails
rs1.MoveNext
Loop

How to write a SQL command to move all old data from access table into another database/table?

I have a table in MS Access 2003 in which I want to archive all old data.
The criteria is that the creation data should be less than a specific date.
I can write a SQL statement to select them, but I don't know how to move them to another database/table? Assuming that the archive database/table is already created and data structure matches current table.
Also how I can make sure that all data which is moved to archive table is removed from current table?
I want to write VBA code to run the command check that data is archived correctly.
You want to 1) move data meeting certain criteria from one table to another, existing table with the same format. 2) You want to "make sure that all data which is moved to archive table is removed from current table." And 3) you "want to write VBA code to run the command check that data is archived correctly."
Contrary to popular opinion, Access does support transactions (the claim that Access SQL does not support transactions is true, but we can still use transactions in VBA code). So modifying code in this post to use transactions in a workspace, I believe this would do the trick (tested in Access 2010 using DAO).
The code to lock, get counts and unlock is not really necessary, and may increase the difficulty of implementing the archive, since it will require that no one be writing to the table while you're updating it. And if it did find a problem, Access does not support transaction logging, so you would have a very short list of options as to how to fix it. But it sounded like you wanted to be absolutely sure the counts were correct, so this adds another level, arguably unnecessary, of checking.
Option Compare Database
Option Explicit
Sub ArchiveOldRecords()
Dim nSourceCount As Long, nMoveCount As Long, nDestCount As Long
Dim strSQL As String, sMsg As String
Dim rsLock As DAO.Recordset
Dim rsBefore As DAO.Recordset, rsAfter As DAO.Recordset
Dim wrk As Workspace, db As DAO.Database
Const strcTableSource As String = "t_TestWithDate" ' Move records FROM table
Const strcTableArch As String = "t_ArchiveTestWithDate" ' Move records TO table
Const strcWHERE As String = " WHERE field2 < " _
& "DATEADD(""yyyy"", -1, Date())" ' Select date field and DATEADD params
Const strcCount As String = "SELECT COUNT(*) As "
On Error GoTo TrapError
Set db = CurrentDb
Set wrk = DBEngine.Workspaces(0)
' Lock table - so no one can add/delete records until count is verified
Set rsLock = db.OpenRecordset(strcTableSource, dbOpenTable, dbDenyWrite)
' Get initial table counts
Set rsBefore = db.OpenRecordset( _
strcCount & "SourceCount, " _
& "(SELECT COUNT(*) FROM " & strcTableSource _
& strcWHERE & ") As MoveCount, " _
& "(SELECT COUNT(*) FROM " & strcTableArch & ") As DestCount " _
& "FROM " & strcTableSource & ";", dbOpenForwardOnly)
nSourceCount = rsBefore!SourceCount
nMoveCount = rsBefore!MoveCount
nDestCount = rsBefore!DestCount
rsBefore.Close
wrk.BeginTrans
' Copy records
strSQL = "INSERT INTO " & strcTableArch _
& " SELECT * FROM " & strcTableSource & " " & strcWHERE & ";"
db.Execute strSQL, dbFailOnError
' Unlock table - only needed for counts
rsLock.Close
Set rsLock = Nothing
' Delete copied records
strSQL = "DELETE * FROM " & strcTableSource & " " & strcWHERE & ";"
db.Execute strSQL, dbDenyWrite + dbFailOnError
' Lock table - only needed for counts
Set rsLock = db.OpenRecordset(strcTableSource, dbOpenTable, dbDenyWrite)
wrk.CommitTrans
' Get final table counts
Set rsAfter = db.OpenRecordset( _
strcCount & "SourceCount, " _
& "(SELECT COUNT(*) FROM " & strcTableSource _
& strcWHERE & ") As MoveCount, " _
& "(SELECT COUNT(*) FROM " & strcTableArch & ") As DestCount " _
& "FROM " & strcTableSource & ";", dbOpenForwardOnly)
' Double-check counts
If (rsAfter!SourceCount <> nSourceCount - nMoveCount) _
Or (rsAfter!DestCount <> nDestCount + nMoveCount) _
Or (rsAfter!MoveCount > 0) Then
sMsg = vbNewLine
sMsg = sMsg & "Records in " & strcTableSource & " before: "
sMsg = sMsg & nSourceCount
sMsg = sMsg & vbTab & "after: "
sMsg = sMsg & rsAfter!SourceCount
sMsg = sMsg & vbNewLine
sMsg = sMsg & "Records to archive from " & strcTableSource & ": "
sMsg = sMsg & nMoveCount
sMsg = sMsg & vbTab & "after: "
sMsg = sMsg & rsAfter!MoveCount
sMsg = sMsg & vbNewLine
sMsg = sMsg & "Records in " & strcTableArch & " before: "
sMsg = sMsg & nDestCount
sMsg = sMsg & vbTab & "after: "
sMsg = sMsg & rsAfter!DestCount
MsgBox "Count double-check failed" & sMsg
End If
Exit_Sub:
On Error Resume Next
' Unlock table and close recordsets
rsLock.Close
rsBefore.Close
rsAfter.Close
Set rsBefore = Nothing
Set rsAfter = Nothing
Set rsLock = Nothing
Set db = Nothing
Set wrk = Nothing
Exit Sub
TrapError:
MsgBox "Failed: " & Err.Description
wrk.Rollback
Err.Clear
Resume Exit_Sub
End Sub
There is no MOVE command but you can copy the records across to the target and then use a similar query to remove them from the source when you are sure you have no Paste Errors.
INSERT INTO MyArchive (fld1, fld2, fld3, fld4) SELECT fld1, fld2, fld3, fld4 FROM MyTable WHERE fld4 < DATEADD("y", -5, Date())
That copies across everything older than 5 years. After confirming the transfer,
DELETE * FROM MyTable WHERE fld4 < DATEADD("y", -5, Date())
That's off the top of my head and I transition between T-SQL and MS Access a fair bit but I think that is pretty solid Access query code. Your own field lists will vary accordingly.

random function not working properly

I am having trouble with an asp page that contains a random function to display a random number of random records. The way I have the code now, as shown below, nothing appears on the page even though there are several records that match the criteria.
Out of 500 records I have in my database, about 70 match the criteria, but form some reason, they never seem to fall within the plist variable, to be used in the query.
When I change the max number to a lower number, say 10, I get an error message that the EOF has been found or no more records are available. Using debugging code, I couldnt find anything out of the ordinary, just the fact that from all the 100 records input into the plist, none of them match the records from the main criteria.
I am posting the entire code here. Maybe someone can catch what may be causing to not work properly.
Sub GetRandomDomains
dim conn, maxrecords, count, webname
dim randomrs, sql
'If (conn.State = adStateClosed) Then
OpenDB conn
'End If
count=0
maxrecords=100
KeywordColumnGetRandomKeywords conn, maxrecords, sql
If sql="" then
conn.close
set conn=nothing
Response.Write(" NOT AVAILABLE")
exit sub
end if
set randomrs=conn.execute(sql)
Response.Write("<ul id='catnav'>")
do While Not randomrs.EOF and count<maxrecords
If isnull(randomrs("sitename")) Then
webname=randomrs("domain")
Else
webname=randomrs("sitename")
End if
Response.Write "<li> " & webname &"</li>"
count=count+1
randomrs.movenext
loop
Response.Write("</ul>")
CloseSet randomrs
CloseDB conn
end sub
Sub KeywordColumnGetRandomKeywords (conn,maxrecords,sql)
dim i, id, index, plist, rs, sqlstr, sortstr
plist=""
Set rs=Server.CreateObject("ADODB.Recordset")
sqlstr="SELECT domainid FROM domains"
sqlstr=sqlstr
Debugwrite sqlstr, "sqlstr for random domains"
rs.Open sqlstr,conn,3,3
If rs.eof then
CloseSet rs
Response.Write(" EMPTY")
sql=""
exit sub
end if
Debugwrite rs("domainid"), "rs for random domains"
Dim arrData ' Array to Store Data
Dim arrSequence ' Array to Hold Random Sequence
Dim iArrayLooper ' Integer for Looping
Dim iarraysize ' Size of Data Array
If xdbasetype="Local" Then
iarraysize=cint(rs.recordcount)
else
iarraysize=cint(GetRecordcount (conn))
end if
Debugwrite GetRecordcount(conn), "getrecordcount for random domains array"
Debugwrite(IsArray(iarraysize)), "random domains count array"
'if (cint(GetRecordcount(conn)))= 0 or isnull(cint(GetRecordcount(conn))) then
'Exit Sub
'End if
redim arrdata(cint(iarraysize))
for i = 0 to iarraysize-1
arrData(i)=rs(0)
rs.movenext
next
rs.close
Set rs = Nothing
If iarraysize<maxrecords then
maxrecords=iarraysize
end if
' Get an array of numbers 0 to array size randomly sequenced.
arrSequence = Resequencearray(iArraySize)
for i = 0 to maxrecords-1
index=arrsequence(i)
id=arrdata(index)
if plist<>"" then
plist=plist & ","
end if
plist=plist & id
Next
sql="select domainid, domain, sitename,sitematch,altmatch from domains"
sql = sql & " WHERE restricted=0 and(sitematch like '%" & xsitematch & "%' or altmatch like '%" & xaltmatch & "%')"
sql = sql & " and domainid In (" & plist & ") "
Debugwrite sql, "first sql for random domains"
end sub
Function ResequenceArray(iArraySize)
Dim arrTemp()
Dim I
Dim iLowerBound, iUpperBound
Dim iRndNumber
Dim iTemp
' Set array size
ReDim arrTemp(iArraySize - 1)
Randomize
iLowerBound = LBound(arrTemp)
iUpperBound = UBound(arrTemp)
For I = iLowerBound To iUpperBound
arrTemp(I) = I
Next
' Loop through the array once, swapping each value
' with another in a random location within the array.
For I = iLowerBound to iUpperBound
iRndNumber = Int(Rnd * (iUpperBound - iLowerBound + 1))
' Swap Ith element with iRndNumberth element
iTemp = arrTemp(I)
arrTemp(I) = arrTemp(iRndNumber)
arrTemp(iRndNumber) = iTemp
Next 'I
' Return our array
ResequenceArray = arrTemp
End Function
'***********************************************************************
' get record count for mysql
'************************************************************************
Function GetrecordCount(conn)
dim sqlstr, rs, rcount
sqlstr="select count(domainid) FROM domains WHERE restricted=0 and (domaingroup='" & xdomaingroup & "' or altmatch like '%" & xaltmatch & "%')"
Debugwrite sqlstr, "sqlstr for random domains"
set rs=conn.execute(sqlstr)
if rs.eof then
rcount=0
else
rcount=rs(0)
end if
CloseSet rs
Getrecordcount=cint(rcount)
Debugwrite rcount, "getrecordcount for random domains"
End function
Okay. There may be a far more simple way of approaching this. Here's a piece of code that gets the data as an array - far simpler and we have much more control over what's in it...
'Constants relating to the following routines...
const C_NO_DATA = "NO_DATA" 'Used when no data is returned to a consuming routine
const C_ERROR = "ERROR" 'Used when an error is generated
'GetDataSet
' Returns a table of data based on the supplied SQL statement and connection string.
'Parameters:
' sqlString (string) - The SQL string to be sent.
' connString (string) - The database connection string.
'Usage:
' dataSet = GetDataSet(sqlString, connString)
'Description:
' This function generates a table of information in a 2 dimensional array. The first dimension represents the columns
' and the second the rows. If an error occurs while the routine is executing the array and the base index (0,0) is set
' to C_ERROR, (0,1) to the VBScript error index, and (0,2) to the VBScript error description.
function GetDataSet(sqlString, connString)
'Initialise...
dim returnVal, rsData
on error resume next
'Define and open the recordset object...
set rsData = Server.CreateObject("ADODB.RecordSet")
rsData.Open sqlString, connString, 0, 1, 1
'Initialise an empty value for the containing array...
redim returnVal(0,0)
returnVal(0,0) = C_NO_DATA
'Deal with any errors...
if not rsData.EOF and not rsData.BOF then
'Store the data...
returnVal = rsData.GetRows()
'Tidy up...
rsData.close
set rsData = nothing
select case err.number
case 3021 'No data returned
'Do nothing as the initial value will still exist (C_NO_DATA)
case 0 'No error
'Do nothing as data has been returned
case else
redim returnVal(4,0)
returnVal(0,0) = C_ERROR
returnVal(1,0) = err.number
returnVal(2,0) = err.description
returnVal(3,0) = sqlString
returnVal(4,0) = connString
end select
end if
on error goto 0
'Return the array...
GetDataSet = returnVal
end function
Okay, so we read the data we're after into an array. Note that I don't know where your xaltmatch and xsitematch variables come from so you'll need to supply these at some point...
Dim ds, sql
sql = _
"SELECT " & _
"domainid, " & _
"domain, " & _
"sitename, " & _
"sitematch, " & _
"altmatch " & _
"FROM " & _
"domains " & _
"WHERE " & _
"restricted=0 AND " & _
"(sitematch LIKE '%" & xsitematch & "%' OR " & _
"altmatch LIKE'%" & xaltmatch & "%') AND " & _
"domainid IN (" & plist & ") "
ds = GetDataSet(sql, conn)
The ds variable now contains an array of elements pulled from the database. All we need to do now is loop through the required number of times...
dim row, rows, used, randoms, col, cols, rc, cc
rows = UBound(ds, 2) 'Get the upper bound of the second array dimension
cols = UBound(ds, 1) 'Get the number of columns for writing to the page
randoms = 70 'Total number of randoms we need
used = ","
for rc = 1 to randoms
'Just in case we don't want to repeat the same row...
do
row = int(rnd(1)*rows) 'Zero based - we don't need to add 1
until instr(used, "," & row & ",")) = 0
'Add our random row to the list...
used = used & row & ","
'Write our output to the page...
response.write("<table>")
for cc = 0 to cols
response.write("<tr>")
response.write("<td>")
response.write(ds(cc, row))
response.write("</td>")
response.write("</tr>")
next 'cc
response.write("</table>")
next 'rc
The GetDataSet code is part of my stock functions, so I know that works, but I will hold my hand up and say that I haven't tested the rest of it.
Have a bash with this one, Luis, and let me know how you get on.

find the duplicate and write it in log file

I have craeted code which reads the acc no, rtn, name and amt from text file and stores in recordset. After that i created sql that stores recordset data into sql server 2005 table.
The problem is In that accno column is primary key. but i have some duplicate accno in my text file. While adding recordset to database, if it finds duplicate accno it is stopping there and not inserting any rows after that duplicate column.
Now i what i want to do is if there is any duplicate column, i want to store that column into log file and skip that column and insert remaining columns into databse. I dont know how to do it. Can anybody help me. like how to check the duplicate column and skip that and insert remaining.
' Write records to Database
frmDNELoad.lblStatus.Caption = "Loading data into database......"
Dim lngRecCount As Long
lngRecCount = 0
rcdDNE.MoveFirst
With cmdCommand
.ActiveConnection = objConn
.CommandText = "insert into t_DATA_DneFrc (RTN, AccountNbr, FirstName, MiddleName, LastName, Amount) values ('" & rcdDNE("RTN") & "', '" & rcdDNE("AccountNbr") & "', '" & rcdDNE("FirstName") & "', '" & rcdDNE("MiddleName") & "', '" & rcdDNE("LastName") & "', '" & rcdDNE("Amount") & "')"
.CommandType = adCmdText
End With
Set rcddnefrc = New ADODB.Recordset
With rcddnefrc
.ActiveConnection = objConn
.Source = "SELECT * FROM T_DATA_DNEFRC"
.CursorType = adOpenDynamic
.CursorLocation = adUseClient
.LockType = adLockOptimistic
.Open
End With
Do Until rcdDNE.EOF
lngRecCount = lngRecCount + 1
frmDNELoad.lblStatus.Caption = "Adding record " & lngRecCount & " of " & rcdDNE.RecordCount & " to database."
frmDNELoad.Refresh
DoEvents
Call CommitNew
rcdDNE.MoveNext
Loop
frmDNELoad.lblStatus.Caption = "DNE Processing Complete."
frmDNELoad.Refresh
End Function
Sub CommitNew()
' Add records to DneFrc table
With rcddnefrc
.Requery
.AddNew
.Fields![RTN] = rcdDNE.Fields![RTN]
.Fields![AccountNbr] = rcdDNE.Fields![AccountNbr]
.Fields![FirstName] = rcdDNE.Fields![FirstName]
.Fields![MiddleName] = rcdDNE.Fields![MiddleName]
.Fields![LastName] = rcdDNE.Fields![LastName]
.Fields![Amount] = rcdDNE.Fields![Amount]
.Update
End With
End Sub
More of a strategy then a specific answer but ...
When importing data from external sources we'll often insert the data into staging tables that do not have the same keys/contraints placed on them and then sanitize the data prior to insertion.
What is done during "sanitation" depends on your requirements (for example, when you have two of the same account numbers are the records the same or are the data fields different, if the fields are different, how do you choose which data to use?). And then insert/move it into the production table once sanitization is complete.
I ran into this problem and what I did is to make a collection that I stored the object and the key into the Key. If I try to add a duplicated key I get an error.
This is the easyest way I found to do this in vb6. in c# is dictionary.
My suggestion would be to add error handling to CommitNew to see if the row inserted would create a primary key violation, and if so then perform other handling.
Example:
Sub CommitNew()
''#Add records to DneFrc table
On Error GoTo CommitNew_Error
With rcddnefrc
.Requery
.AddNew
.Fields![RTN] = rcdDNE.Fields![RTN]
.Fields![AccountNbr] = rcdDNE.Fields![AccountNbr]
.Fields![FirstName] = rcdDNE.Fields![FirstName]
.Fields![MiddleName] = rcdDNE.Fields![MiddleName]
.Fields![LastName] = rcdDNE.Fields![LastName]
.Fields![Amount] = rcdDNE.Fields![Amount]
.Update
End With
Exit Sub ''# If no error, exit routine.
CommitNew_Error:
If Err.Number = -2147217873 Then
''# code here will only execute if the constraint violation occurs
Call WriteDuplicateAccountToFile()
Err.Clear() ''# This clears the error, since you handled it
Else
''# Do stuff with other errors.
''# If you're not sure, at least display what error its giving, like so
MsgBox "The following error was encountered when new record was saved:" & _
vbNewLine & CStr(Err.Number) & " - " & Err.Description & vbNewLine & _
"New record not saved.", vbOkOnly + vbCritical, "Error"
End If
End Sub

Resources