I am trying to find the best way to extract report from Access into Excel and updating Access, once data has been yieled by Questionnaire participants.
The data in Access is quite complex and large (contained in a single master table - 50+ columns & 200K+ rows) - which doesn't make it easy on the eyes.
I would like to extract reports from Access (essentially a subset of data - M columns out of N). The report will be updated by individuals (i.e. values updated) and will need to be fed back into Access.
I'm looking for the most user - friendly approach to achieve the above. Does anyone have any ideas?
Ps obvious answers welcome, I am a complete Access/Excel noob - but have a background in MySQL, SQL Server.
As far as i'm aware, there is no single tool to do this, if you want to create your own, i recommend you to use any .NET language, ADO.NET and Interop to read data from and into Excel. If you don't feel you want to do some .NET programming, there is a reporting tool called DBxtra that take Excel, Access and many databases as input and let you export data to Excel (either static or dynamic) and other formats. The route will be like this:
1.- Create your query from your Access database in DBxtra
2.- Export the results into Excel (either linked or static sheet)
3.- Let yout users update the data in Excel
4.- Use Excel macros to update the data in the Access database.
To update the access database you will need to put a macro like this into in your Excel file:
Sub appProb()
On Error GoTo 1
Dim XLTable As TableDef
Set Db = OpenDatabase(mdbFile)
Set XLTable = Db.CreateTableDef("tblProbTemp")
XLTable.Connect = "Excel 5.0;DATABASE=" & xlsFile
XLTable.SourceTableName = "tblXLProb"
Db.TableDefs.Append XLTable
strSQL = "Insert into tblProb Select * FROM tblProbTemp"
Db.Execute strSQL
1:
Err.Clear
On Error Resume Next
Db.TableDefs.Delete "tblProbTemp"
Db.Close
End Sub
The only thing worth to note is that tblXLProb is named range.
Note: This macro was copied from here: http://www.mrexcel.com/forum/microsoft-access/51157-update-access-records-excel.html
Related
I am trying to create an excel addin which has set of Functions to pull value from the database (I use MS SQL Server). So my query will return only one record set. I use something like below in my vba code.
Using Excel VBA to run SQL query
But the problem with this is if I have my custom function in 100 cells, the macro makes connection to the DB everytime and retrive data from the DB.
Is there a way where I can make one connection and use that connection to write as many queries as I want?
Simple, run all your 100 function/ loops to access the database. One you've finished then close the connection. Look at your modified code below...
Option Explicit
Sub ConnectSqlServer()
Dim conn As ADODB.Connection
Dim rs As ADODB.Recordset
Dim sConnString As String
' Create the connection string.
sConnString = "Provider=SQLOLEDB;Data Source=INSTANCE\SQLEXPRESS;" & _
"Initial Catalog=MyDatabaseName;" & _
"Integrated Security=SSPI;"
' Create the Connection and Recordset objects.
Set conn = New ADODB.Connection
Set rs = New ADODB.Recordset
' Open the connection and execute.
conn.Open sConnString
'>>>> run 100 loops
Dim i As Integer
For i = 1 To 100
Set rs = conn.Execute("SELECT * FROM Table" + 1 + ";")
' Check we have data.
If Not rs.EOF Then
' Transfer result.
' I assume that you've 100 Sheets
Sheets(i).Range("A1").CopyFromRecordset rs
' Close the recordset
rs.Close
Else
MsgBox "Error: No records returned.", vbCritical
End If
Next
' Clean up
If CBool(conn.State And adStateOpen) Then conn.Close
Set conn = Nothing
Set rs = Nothing
End Sub
I've add 100 loops and run it before the connection to database is closed.
Hope it useful.
In this kind of cases do not do db operations in loops. This is time consuming and not proper useage. Instead, in a loop create select, insert or whatsoever statement and then complate the loop operations and calculations then open the connection just one time and send request (created sql script) to db and get response from db then close connection. Thats all. db operations must be sparated from dailiy and recursive opeations.(antipattern) best regards.
Disclaimer: while this is not a direct solution to the problem described in the post I would like to add this approach as a much faster and easier solution to the problem described.
Step 1: create a (possibly hidden) sheet where you pull all the SQL data that you need in this Excel file. Pull the data into one table with all the necessary columns / dimensions to get the data afterwards from this table.
Here is what I mean by that. Let's assume that you need in this Excel file some data from the table Users on the SQL server as well as some information from a StockMarket table on the server. From the table Users you want the UserID, the first name, last name, and the job title. From the table StockMarket you will need the Stockmarket ID and the price for this particular share. As these prices are by date you also need the the quote date for the price.
Now, since you want all in one table you must think of a smart way to combine all this data into one table. One approach could be the following:
Afterwards you can get all the data from the above table with lookup functions like these:
=INDEX(SQLdata,MATCH(1,(SQLdata[Table]="Users")*(SQLdata[UserID]=25),0),4)
Note, that I named the table SQLdata to make it easier when looking at the formula and understanding it. Also, like this you can easily scan your Excel file for any reference to this table.
Another approach could be the following to make the table more concise:
Note, that this time I am mixing Strings with Numbers and also Strings with Dates (which is very poor design and for some people here even impossible to think of). Also, the column headers are now less descriptive. Yet, this works too:
=INDEX(SQLrev,MATCH(1,(SQLrev[Table]="Users")*(SQLrev[Dimension1]=25),0),5)
Note, that I called the table this time SQLrev.
Both solutions allow you also to aggregate data from the table. So, if you want (for example) the average price for Apple in 2017 then you can use the following formula to sum up the the quotes for this year and divide them by 3:
=SUM(IF("StockMarket"=SQLrev[Table];1;0)*IF("AAPL"=SQLrev[Dimension1];1;0)*SQLdata[Price])/3
The most notable advantage for this approach is that you merely have to update one table in the entire Excel file which means that there is only one SQL pull from the server.
The most notable disadvantage (apart from the SQL select which might get pretty complicated to write) is that you need to know of all the data that needs to reside in this table. If the data is not pulled into this table then none of the above formulas will be able to retrieve these values.
While this approach certainly has its downsides this is much easier to implement than the Excel AddIn you are aiming for.
All above formulas are array formulas and must be entered pressing Ctrl + Shift + Enter. For more information on array formulas please read the following: https://support.office.com/en-us/article/Guidelines-and-examples-of-array-formulas-7D94A64E-3FF3-4686-9372-ECFD5CAA57C7
I am in the process of converting an Access application to use a SQL Server backend while still using the Access front end forms. Sounds like fun I know.
This application needs data access to 2 SQL Server databases that are on the same server. There are numerous inline sql query strings that attempt to connect to both databases at the same time on a single ADODB connection. This is failing because I am expecting records but none are returned.
What is the best way to fix this? Is there any way to use these sql strings or must it all be converted to stored procedures? Thanks for any help.
Here is some code:
Dim conn As ADODB.Connection
Set conn = New ADODB.Connection
Dim rst As ADODB.Recordset
Set rst = New ADODB.Recordset
With conn
.Provider = "sqlncli11"
.ConnectionString = "Server=[MY_SERVER];Database=[MY_DATABASE];User Id=sa; Password=password;"
.Open
End With
Dim str As String
str = "SELECT TABLE_DB1.Parent_Item_No FROM TABLE_DB1 INNER JOIN [DB2].[dbo].TABLE_DB2 ON (TABLE_DB1.Comp_Item_No = " & _
"TABLE_DB2.item_no) AND (TABLE_DB1.Loc = TABLE_DB2.loc) " & _
"GROUP BY TABLE_DB1.Parent_Item_No " & _
"HAVING (((TABLE_DB1.Parent_Item_No)='" & str_Assembly & "'));"
With rst
.Open str, conn, adOpenKeyset, adLockOptimistic ' this fails to return records
If .RecordCount > 0 Then
'Do Stuff
Else
'Do Other Stuff
End If
End With
You're only checking RecordCount. Take a look at this: slxdeveloper.com/page.aspx?action=viewarticle&articleid=33 Some recordset types don't populate the RecordCount property (adOpenKeyset should though). What happens if you use While Not .EOF and .BOF instead? What is that actual value of RecordCount in your code?
Would it at all be possible to run queries from SQL which save to an access file? I have nothing but trouble pulling data directly into access. I do have success when I set up an ODBC database and go to data -> get external data -> from other sources -> From microsoft query
Another method I myself have been successful with is using the power query add-on from microsoft. https://www.microsoft.com/en-us/download/details.aspx?id=39379
Despite this, what I still mostly end up doing is using the SQL import/export tool. I don't have screenshots or specific instructions as I'm not at work right now, but this can write directly to text files, Access databases, everything. I love it too much. Getting the correct drivers was a loop for sure. If you're on 64-bit and having issues then this is the driver you need. http://www.microsoft.com/en-us/download/details.aspx?id=13255
What I do is:
Set up my source (SQL 11 native client), choose the database you're pulling from
Specify an outfile type and location (I think it has to already exist)
When prompted to specify whether to pull data from tables and views or write a query, select write a query.
Go through the rest of the importer, you can edit the sql statement later on when viewing conversion and specify whether the transfer fails or ignores errors etc.
I personally still use the import export tool for transfers of all sizes because it's just so difficult to get all the correct drivers and get SQL to like what I want. (and without admin rights I get tired of asking my boss).
I hope one of those solutions can help you!
I've outline a more proper fix and a quick fix...
The more proper fix is the Data Layer Pattern. There is a lot to this fix and it may require some application structural changes. This is discussed in depth in another question:
Data Access Layer design patterns
A very simple fix is to use Access Linked tables. A Linked Table works like a normal Access table except the data is stored and updated on the SQL Server. Its basically a built in Data Access Layer to SQL Server. Its not an elegant solution but it gets you up and running right away. More info can be found here:
https://support.office.com/en-us/article/Import-or-link-to-SQL-Server-data-a5a3b4eb-57b9-45a0-b732-77bc6089b84e#bm2
On thing to be aware of with Linked Tables are that some Access Queries and Forms retrieve all the records before filtering and can lock the table so you can end up with some performance headaches if you have lots of data and lots of users.
Consider using the SQL server SYNONYM feature to add aliases for objects in one database to the other. Then just update all your queries to use one database.
Also, you could merge the two databases with each one, or one of them, going into a new schema to keep them separate. This could be tough if you have a lot of stored procedures, views, and functions in the database. This may be a terrible answer, but it could also be true that the two databases should never have been separate in the first place.
In the INNER JOIN, you prefixed the table name with DatabaseName.Schema.:
... FROM TABLE_DB1 INNER JOIN [DB2].[dbo].TABLE_DB2 ...
But you didn't do it in the other places where TABLE_DB2 occurs.
So you either need to change this:
ON (TABLE_DB1.Comp_Item_No = TABLE_DB2.item_no) AND (TABLE_DB1.Loc = TABLE_DB2.loc)
...to this:
ON (TABLE_DB1.Comp_Item_No = [DB2].[dbo].TABLE_DB2.item_no) AND (TABLE_DB1.Loc = [DB2].[dbo].TABLE_DB2.loc)
Or (which I prefer) you can use aliases for the table names in the FROM clause:
... FROM TABLE_DB1 t1 INNER JOIN [DB2].[dbo].TABLE_DB2 t2...
...then you use the aliases everywhere else:
str = "SELECT t1.Parent_Item_No FROM TABLE_DB1 t1 INNER JOIN [DB2].[dbo].TABLE_DB2 t2 ON (t1.Comp_Item_No = " & _
"t2.item_no) AND (t1.Loc = t2.loc) " & _
"GROUP BY t1.Parent_Item_No " & _
"HAVING (((t1.Parent_Item_No)='" & str_Assembly & "'));"
Additional background information:
If you connect to an SQL Server via ADO, you're directly connecting to exactly one database - the one in the connection string:
.ConnectionString = "Server=[MY_SERVER];Database=[MY_DATABASE];User Id=sa; Password=password;"
So in your case, the database you're connecting to is named MY_DATABASE. Any SQL you're executing via ADO goes to that database.
If you need to get data from other databases on the same server, you need to prefix the names with DatabaseName.Schema. in all places where you use them.
So let's assume we have:
a table MY_TABLE in MY_DATABASE
a table OTHER_TABLE in OTHER_DATABASE on the same server
both tables have the schema dbo (the default in SQL Server)
With the connection string from above (connecting to MY_DATABASE), you can join them as follows:
select *
from MY_TABLE
inner join OTHER_DATABASE.dbo.OTHER_TABLE
on MY_TABLE.SomeColumn = OTHER_DATABASE.dbo.OTHER_TABLE.OtherColumn
where OTHER_DATABASE.dbo.OTHER_TABLE.EvenAnotherColumn = 'foo'
See? Everywhere I used OTHER_TABLE, I prefixed it with OTHER_DATABASE.dbo..
PS:
It's bad practice to use the sa user to connect to a database with an application. The sa user has the highest permissions possible.
You should either use Windows authentication or create a dedicated SQL user for your app.
Consider storing your SQL in a pass-through query instead of VBA code. You can apply your filter using a copy of the .sql property of the pass-through query's querydef object, modifying it with the criteria they enter in your form at runtime.
So I have a particular situation here with the CRM I'm tasked with developing, and I'd like to see if I can get some insight on more of a sanctioned, best-practice approach for this specific scenario. Basically, I have a form called Accounts that displays information specific to that account as well as a subform that needs to display quotes and orders associated with that account to give the user a snapshot of work done.
Due to the complexity of the query and number of possible records, scrolling through this subform can be slow if using a SQL VIEW or more obviously, a local access query. The best solution here is a pass through query so I can shoot that processing straight to SQL.
I've done quite a bit of research and the best approach I've seen is to either 1) use VBA to change the definition of the pass through query or 2) use an INSERT INTO statement to populate a local table which can be used as the recordsource for that subform. At this point I just want to confirm that either or both of these approaches would qualify as a best practice situation and possibly an example of code to make this happen. Right now I'm struggling with where this processing belongs. I don't want to call the code from the subform that launches the main Accounts form since this form can be opened from a few different places and I don't want to duplicate that code.
Here is the code I'm using to copy the dataset from the pass through query to a local table and then using the Master/Child relationship fields to only show the appropriate data for that account.
Dim db As DAO.Database
Dim strSQL As String
Set db = CurrentDb
'Purge local table where pass through results are copied
strSQL = "Delete * FROM tblPassThruResults"
db.Execute strSQL
'Insert the results of the pass through query into local table
strSQL = "INSERT INTO tblPassThruResults Select Q_ACCOUNT_BUSINESS_OVERVIEW_PT.* FROM Q_ACCOUNT_BUSINESS_OVERVIEW_PT"
db.Execute strSQL
db.Close
Set db = Nothing
I have the following requirement,
A SQL Server 2008 scheduled job that runs at 9 am in the morning. This job should send a spreadsheet with data in a workbook(workbook1) and the chart attached in the next workbook(workbook2). The data must have the values from a SQL table. The chart must reflect the values present in the workbook1. This sheet should be mailed across to n number of users. The n number of users are not even aware of the sql server and dont know anything about the username and password of the server. They must just have the spreadsheet with the two workbooks.
I have decided to do the below
1) Create a stored procedure that formats the data into a table 2) Invoke a SSIS , to copy the data into the workbook1 of excel 3) Create the Graph in the workbook2 of the same excel
The point 1 and 2, I have completed already.
After activity 1 and 2, I will have something like below in the excel
Workbook1:
Date | Column A | Column B | Column C |
10-7-14 | 0983883 | 09433344 | 4443333 |
11-7-14 | 0986444 | 06875544 | 4689073 |
I am not really able to do the activity 3. Activity 3 must have take in the values from the above table and create graphs in workbook2
I know it can be done with SSRS, but I would like to know other ways as we have only SSIS with us and SSRS is totally out of scope. For some reasons SSRS is restricted to use in our systems.
I have tried various ways to do this. (created a dynamic graph in workbook2 of the excel template for SSIS and let the values transferred to workbook1 renders the graph but it does not work sadly.). I am not able to create dynamic graphs with the empty excel templates that I feed to SSIS.
Excel is the problem here, it does not allows any pre-defined graphs in it. It always expects a value to create a graphs.
I can also go for a VBA macro option,(like creating a button and let the user click the button to generate graphs) but I am not really sure if it is a feasible one?
Excel experts please help !
I might like sound like an amateur but please pardon me as I am new to SSIS and none of my colleagues have an idea on how to do it and I could not get anything with the research I have done, which is a bit frustrating though.
As I don't know the specifics of your workbook structure you might need to adapt the following vba code (as the sub name suggests this should be placed in the Open event of the workbook):
Private Sub Workbook_Open()
Dim wb As Workbook
Dim ws As Worksheet
Dim ws2 As Worksheet
Dim LastRow As Long
Dim LastColumn As Long
Dim LastColumnLetter As String
Set wb = ThisWorkbook
Set ws = wb.Worksheets("Sheet1")
Set ws2 = wb.Worksheets("Sheet2")
Application.ScreenUpdating = False
LastRow = ws.Cells(Rows.Count, "A").End(xlUp).Row
LastColumn = ws.Cells(2, Columns.Count).End(xlToLeft).Column
LastColumnLetter = Chr(LastColumn + 64)
ws2.Activate
ws2.ChartObjects("Chart 1").Activate
ActiveChart.SetSourceData Source:=ws.Range("A1:" & LastColumnLetter & LastRow)
ws2.Range("A1").Select
Application.ScreenUpdating = True
End Sub
This code executes when the workbook is opened and first tries to find the range your data table is occupying and then updates the Data Source for the Chart. I have assumed that your data are in Sheet1 and start at cell A1 while your chart (Chart 1) is in Sheet2.
I do something similar for work but am using excel as a front end to SSAS but the concept is similar for sql server.
It sounds like you would find a pivot chart in excel useful. This is due to the fact that excels dynamically renders the chart based on the results of the pivot table.
Note that the pivot table allows for filtering so this means that the user can filter out parts of the data they do not need. Or if the user wants to alter how the chart looks, e.g. sales by month instead of sales by quarter it is extremely easy for them to do. So basically the presentaton of the results are being done dynamically in excel.
To create one based off external data you do the following:
Data --> from other sources
It is possible for excel to query sql server, though you probably want to limit the result set to a small read-only table for reporting.
Note that the data source information is saved into the file, with excel being able to use both user name and password hard coded in or the window's domain logins. Ideally you want to use the domain login over a general username and password
So the solution becomes more like the following:
SP or SSIS to populate a table for excel to query. This means excel is just writing simple selects and group bys
Using Data --> from other sources --> from SQL server
Create the stock pivot table, which will handle the presentation of the data
Give file to users. The file would have the defined datasource in it
They then just hit refresh which will requery the server for the results.
User is able to drag and drop fields, which updates the pivot table and redraws the excel chart
The only downside being that the user can only see a copy of the current dataset.
At work we've got a SQL Server database that several users connect to (read only) using Access 2003.
This was fine, except now one of the tables they look at is re-created fairly often and part of this process involves a cross-tab query.
Which means that, depending on the data, the number and and names of the columns potentially change each time the table is regenerated.
However when they look at the re-created table from Access, it still shows the column headings that were there when the table was first linked to.
Is there a way I can programmatically re-link the table each time they open the Access database?
What I ended up doing was creating a VBA function that looks something like below (needs error handling!)
Public Function ReConnectToLinkTable()
Dim db As Dao.Database
Dim tdf As Dao.TableDef
Set db = CurrentDb
Set tdf = db.CreateTableDef("local_table_name")
tdf.Connect = "ODBC;DRIVER=SQL Server;SERVER=server_name;UID=user_name;" & _
"PWD=password;APP=Microsoft Data Access Conponents;" & _
"DATABASE=database_name"
tdf.Attributes = TableDefAttributeEnum.dbAttachSavePWD
tdf.SourceTableName = "server_table_name"
db.TableDefs.Delete ("local_table_name")
db.TableDefs.Append tdf
End Function
Then I created a macro called AutoExec (the name guarantees it is called when the Access file is opened) which has an Action of RunCode, which calls the ReconnectToLinkTable() function.
ODBC linked tables break when the table or view on the server is altered. Some changes can result in them just becoming read-only, others will simply not include all the columns.
I have found that updating the connect string does not successfully fix this problem. It will usually fix missing fields, but it can still be read-only. The only reliable way to do this is to recreate the linked table on the fly.
Another alternative would be to not use a linked table at all, but use a saved QueryDef that has the appropriate connect string. This will never have to be updated, but could be a performance issue as the metadata stored in the table link helps Access figure out how to retrieve the data. Without that metadata stored in the table link, it has to retrieve that information from the server each time the query is run.
Something like this snippet is usually used. Search google for 'ms access refresh link table' and you'll find various solutions all similar to this one.