Basically, we have three databases to grab data from. One is a SQL Server database, one is an Access database (which is particularly annoying to connect to because we have to map a network drive and such), and the final one will be an Oracle database when IT finally gives us rights.
I'm thinking about creating a helper function that makes querying any one of these databases as easy as possible. Ideally, I want to create a two-dimensional array
Dim myEasyResultArray(10,10) as String
myEasyResultArray = DatabaseHelper("Access", "SELECT * FROM Employee")
Is this a good design decision? Also, how can I have the array be the right size? Can I just do this?
Dim myEasyResultArray = DatabaseHelper("Access", "SELECT * FROM Employee")
Should this be a module or a class? I don't really need to share variables,
I would try to put all my data access logic into a data access layer. Ideally this would be in a separate library and namespace, but it doesn't have to be. I would use classes, typically one per table/entity and design all the classes to be stateless (so you don't have to ever reuse the same instance of the data access object, you can just instantiate a new one any time you need to access the DB).
Instead of having it return arrays, I would have it return data objects (often called DTO's - data transfer objects). I would keep the DTO classes as clean as possible containing only public properties and no methods, if possible. The data access classes should all implement interfaces so that multiple versions of each one can be created. One for Access, one for Oracle, one for SQL, etc. Then, wherever I needed to access the database (hopefully only in my business layer, not ever in my UI layer), I would request the appropriate data access objects by their "generic" interface type (thereby requiring a factory class to inject the correct concrete data access object type into my business object).
Here's a real simple example of a DTO:
Public Class Employee
Public Id As Guid
Public Name As String
Public StartDate As Date
End Class
Here's an example Data Access interface
Public Interface IEmployeeDataAccess
Function GetEmployee(id As Guid) As Employee
Function GetEmployees() As List(Of Employee)
End Interface
Here's an example of an data access class:
Public Class SqlEmployeeDataAccess
Inherits IEmployeeDataAccess
Public Function GetEmployee(id As Guid) As Employee Implements IEmployeeDataAccess.GetEmployee
Dim employee As New Employee()
' Connect to SQL DB and populate employee DTO object
Return employee
End Function
Public Function GetEmployees() As List(Of Employee) Implements IEmployeeDataAccess.GetEmployees
Dim employees As New List(Of Employee)()
' Connect to SQL DB and populate list of employee DTO objects
Return employees
End Function
End Interface
You might then make similar classes called AccessEmployeeDataAccess and OracleEmployeeDataAccess which also implement the IEmployeeDataAccess interface. Then, also in the data access layer, I would create a factory class for each supported DB provider. I would make all the DataAccess factories implement the same interface, like this:
Public Interface IDataAccessFactory
Function NewEmployeeDataAccess() As IEmployeeDataAccess
End Interface
Public Class SqlDataAccessFactory
Implements IDataAccessFactory
Public Function NewEmployeeDataAccess() As IEmployeeDataAccess
Return New SqlEmployeeDataAccess()
End Function
End Class
Then, in my business class, I might do something like this:
Public Class EmployeeBusiness
Public Sub New(employeeDataAccess As IEmployeeDataAcess)
_employeeDataAccess = employeeDataAccess
End Sub
Private _employeeDataAccess As IEmployeeDataAcess
Public Function GetEmployee(id As Guid) As Employee
Return _employeeDataAccess.GetEmployee(id)
End Function
End Class
And then in my business factory, I'd do something like this:
Public Class BusinessFactory
Public Sub New()
Select Case dataAccessType
Case "SQL"
_dataAccessFactory = New SqlDataAccessFactory()
Case "Oracle"
_dataAccessFactory = New OracleDataAccessFactory()
Case "Access"
_dataAccessFactory = New AccessDataAccessFactory()
End Select
End Sub
_dataAccessFactory As IDataAccessFactory
Public Function NewEmployeeBusiness() As IEmployeeBusiness
Return New EmployeeBusiness(_dataAccessFactory.NewEmployeeDataAccess())
End Function
End Class
This could be simplified a great deal by having a single set of data access objects that work with any database provider. To do that, you'd need to use only base ADO types like IDbConnection instead of SqlConnection, and IDbCommand instead of SqlCommand. Then your data access objects could ask for a DB Provider factory which could create a new connection, etc., whenever the DataAccess class needs one. This, however, is easier to do when you are simply calling stored procedures, or something. Often, especially when you are dynamically building your SQL sentences in code, there are too many differences between providers so you can't just use the same DataAccess class for all DB providers.
But, that's just me...
I see one big issue with your proposal.
The first is that I often see programmers want to build a database help with methods that resemble this form:
DataTable GetData(string sql)
The important point here is that the method accepts an sql string, with no provision for query parameters. This is wrong. It is an anti-pattern, because it encourages you (pretty much forces you) to encode sql data as part of the query code. You must provide some mechanism for the correct passing of query parameters as separate data. Arrays are common, but that's not the only way. I also normally do not even provide an overload that accepts only the string, as it inevitably leads to abuse. If you want to send a query with no parameter, then pass an empty array. This way it is clear that is what you intend, and anyone who uses your database helper will be encouraged to learn to use parameters the right way.
There are other things that could be improved as well, but this to my mind is the main issue.
Related
I know this question has been asked and discussed a lot (e.g. Here and Here and this Article). Nevertheless, i still feel confused about it. I know that DbContexts should not live for the duration of application lifetime, i know they should be used per Form (window) or per Presenter. The problem is that i don't have Forms or Presenters. I have a single Form (Window) with many view models, some of them which live for the duration of the application and almost all of my view models depend on DbContext (LOB application, WPF, MVVM, Sql Server CE).
My solution is to hide DbContext behind a factory which is injected in all view models that need access to DbContext and those view models create/dispose of the DbContext when their corresponding view is loaded/unloaded.
I would like to know if this solution has any problems or if there is a better solution you could advise ?
I tend to lay my projects out as follows;
1) Presentation Layer:
Contains my Views and ViewModels
2) Business Layer:
Contains my business logic
3) Data Layer:
Contains my models
My Presentation layer calls into the business layer to populate local copies (held in the ViewModel) of the data I wish to use in my ViewModel / View.
This is achieved with a Using statement, something like;
Using DBContext As Entities = ConnectToDatabase()
Dim clsApprovalTypes As New Repositories.clsApprovalTypesRepository(DBContext)
dbResults = clsApprovalTypes.GetRecords()
End Using
Return dbResults
Here I simply pass in the Context into Repository, once the data has been returned, the "End Using" will dispose of my context.
To update the context with changes made in my ViewModel / View, I use an AddEdit routine, which accepts a record, and updates / Adds to the context as necessary, using a similar methodology to the above, something like;
Using DBContext As CriticalPathEntities = ConnectToDatabase()
Dim clsApprovalTypes As New Repositories.clsApprovalTypesRepository(DBContext)
clsApprovalTypes.AddEditRecord(ApprovalTypeToSave)
Try
clsApprovalTypes.SaveData()
Catch ex As Exception
Return ex
End Try
End Using
Where my AddEdit routine is something like;
SavedRecord = New ApprovalType 'Store the Saved Record for use later
Dim query = From c In DBContext.ApprovalTypes
Where c.ApprovalType_ID = RecordToSave.ApprovalType_ID
Select c
If query.Count > 0 Then
SavedRecord = query.FirstOrDefault
End If
'
' Use Reflection here to copy all matching Properties between the Source Entity
' and the Entity to be Saved...
'
SavedRecord = Classes.clsHelpers.CopyProperties(RecordToSave, SavedRecord)
If query.Count = 0 Then
Try
DBContext.ApprovalTypes.Add(SavedRecord)
Catch ex As EntityException
Return ex
End Try
End If
I wrote a little more about some of this here;
https://stackoverflow.com/a/15014599/1305169
I have recently started evaluating Dapper as a potential replacement for EF, since I was not too pleased with the SQL that was being generated and wanted more control over it. I have a question regarding mapping a complex object in my domain model. Let's say I have an object called Provider, Provider can contain several properties of type IEnumerable that should only be accessed by going through the parent provider object (i.e. aggregate root). I have seen similar posts that have explained using the QueryMultiple and a Map extension method but was wondering how if I wanted to write a method that would bring back the entire object graph eager loaded, if Dapper would be able to do this in one fell swoop or if it needed to be done piece-meal. As an example lets say that my object looked something like the following:
public AggregateRoot
{
public int Id {get;set;}
...//simple properties
public IEnumerable<Foo> Foos
public IEnumerable<Bar> Bars
public IEnumerable<FooBar> FooBars
public SomeOtherEntity Entity
...
}
Is there a straightforward way of populating the entire object graph using Dapper?
I have a similar situation. I made my sql return flat, so that all the sub objects come back. Then I use the Query<> to map the full set. I'm not sure how big your sets are.
So something like this:
var cnn = sqlconnection();
var results = cnn.Query<AggregateRoot,Foo,Bars,FooBar,someOtherEntity,AggregateRoot>("sqlsomething"
(ar,f,b,fb,soe)=>{
ar.Foo = f;
ar.Bars = b;
ar.FooBar = fb;
ar.someotherentity = soe;
return ar;
},.....,spliton:"").FirstOrDefault();
So the last object in the Query tag is the return object. For the SplitOn, you have to think of the return as a flat array that the mapping will run though. You would pick the first return value for each new object so that the new mapping would start there.
example:
select ID,fooid, foo1,foo2,BarName,barsomething,foobarid foobaritem1,foobaritem2 from blah
The spliton would be "ID,fooid,BarName,foobarid". As it ran over the return set, it will map the properties that it can find in each object.
I hope that this helps, and that your return set is not too big to return flat.
Time and again I find myself creating a database utility class which has multiple functions which all do almost the same thing but treat the result set slightly differently.
For example, consider a Java class which has many functions which all look like this:
public void doSomeDatabaseOperation() {
Connection con = DriverManager.getConnection("jdbc:mydriver", "user", "pass");
try {
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("SELECT whatever FROM table"); // query will be different each time
while (rs.next()) {
// handle result set - differently each time
}
} catch (Exception e) {
// handle
} finally {
con.close();
}
}
Now imagine a class with 20 of these functions.
As you can see, tons of boilerplate (opening a connection, try-finally block), and the only thing that changes would be the query and the way you handle the result set. This type of code occurs in many languages (considering you're not using an ORM).
How do you manage your DB utility classes so as to reduce code duplication? What does a typical DB utility class look like in your language/framework?
The way I have done in one of my project is that I followed what Spring does with JDBC template and came up with a Query framework. Basically create a common class which can take select statement or pl/sql calls and bind parameters. If the query returns resultset, also pass the Rowmapper. This rowmapper object will be called by the framework to convert each row into an object of any kind.
Example -
Query execute = new Query("{any select or pl/sql}",
// Inputs and Outputs are for bind variables.
new SQL.Inputs(Integer.class, ...),
// Outputs is only meaningful for PL/SQL since the
// ResultSetMetaData should be used to obtain queried columns.
new SQL.Outputs(String.class));
If you want the rowmapper -
Query execute = new Query("{any select or pl/sql}",
// Inputs and Outputs are for bind variables.
new SQL.Inputs(Integer.class, ...),
// Outputs is only meaningful for PL/SQL since the
// ResultSetMetaData should be used to obtain queried columns.
new SQL.Outputs(String.class), new RowMapper() {
public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
Actor actor = new Actor();
actor.setFirstName(rs.getString("first_name"));
actor.setSurname(rs.getString("surname"));
return actor;
});
Finally a Row class is the output which will have list of objects if you have passed the RowMapper -
for (Row r : execute.query(conn, id)) {
// Handle the rows
}
You can go fancy and use Templates so that type safety is guaranteed.
Sounds like you could make use of a Template Method pattern here. That would allow you to define the common steps (and default implementations of them, where applicable) that all subclasses will take to perform the action. Then subclasses need only override the steps which differ: SQL query, DB-field-to-object-field mapping, etc.
When using .net, the Data Access Application Block is in fairly widespread use to provide support for the following:
The [data access] application block
was designed to achieve the following
goals:
Encapsulate the logic used to perform
the most common data access tasks.
Eliminate common coding errors, such
as failing to close connections.
Relieve developers of the need to
write duplicated code for common data
access tasks.
Reduce the need for
custom code.
Incorporate best
practices for data access, as
described in the .NET Data Access
Architecture Guide.
Ensure that, as
far as possible, the application block
functions work with different types of
databases.
Ensure that applications
written for one type of database are,
in terms of data access, the same as
applications written for another type
of database.
There are plenty of examples and tutorials of usage too: a google search will find msdn.microsoft, 4guysfromrolla.com, codersource.com and others.
In the python app engine docs, I see something called dbReferenceProperty. I can't understand what it is, or how it's used. I'm using the java interface to app engine, so I'm not sure if there's an equivalent.
I'm interested in it because it sounds like some sort of pseudo-join, where we can point a property of a class to some other object's value - something like if we had:
class User {
private String mPhotoUrl;
private String mPhone;
private String mState;
private String mCountry;
.. etc ..
}
class UserLite {
#ReferenceProperty User.mPhotoUrl;
private String mPhotoUrl;
}
then if we had to update a User object's mPhotoUrl value, the change would somehow propagate out to all UserLite instances referencing it, rather than having to update every UserLite object instance manually,
Thanks
A db.ReferenceProperty simply holds the key of another datastore entity, which is automatically fetched from the datastore when the property is used.
There's some additional magic where the entity that is referenced has access to a query for entities of type Foo that reference it in the special attribute foo_set.
The Java datastore API instead has owned relationships, which serve the same purpose.
I want to pass a custom class as parameter to the query method which returns me a collection of entities. I need something like this
[Query]
public IEnumerable<MyEntity> Search(SearchParams params)
{
//do something here
}
public class SearchParams
{
public string FilterParam1 {get; set;}
public string FilterParam2 {get; set;}
public string FilterParam3 {get; set;}
public string FilterParam4 {get; set;}
public string FilterParam5 {get; set;}
...and so on...
}
I tried making SearchParams class available at client side using shared code. But the problem is that no operation(query or invoke) let me create a method where I can pass SearchParams class as it is not a native serializable type.
I have about 15 properties in SearchParams class like this.
I do not want to create a Query operation with 15 parameters.
Please suggest is there's a good workaround for that.
If you are prepared to sacrifice change tracking etc on the returned entities then you could use an 'Invoke' method instead. This will allow you to pass complex types as parameters (as long as they are exposed to the client).
You can expose the type by writing a dummy query method that returns your complex type.
See this other question for more details.
As far as I am aware, the abilitiy to pass complex types as parameters to 'Query' methods is on the roadmap but not currently available.
As Frederick said, one of the main points of RIASvcs is you can send a custom query (LINQ expression) over the wire from the client, to the server, and have it executed there.
It brings all the records to the web server and then filters them.
If I understand you correctly, this isn't true-- bring up Fiddler and see what gets sent back-and-forth, it is indeed filtering on the server and only returning (to the client) what was asked for.
If you read what Martin Fowler says about SOA:
When you're working with a remote interface, such as Remote Facade (388), each call to it is expensive. As a result you need to reduce the number of calls, and that means that you need to transfer more data with each call
So your question has only one answer, yes.
It doesn't make sense to expose complex type like Expression or Func through WCF just because you want to provide a simple syntax in the client.
You have the DataContract, use that to expose a Dto that represent your query.
I think you missed the whole point with the Query method :-)
The Load method which you use to "execute" the query method takes a Query as an argument. Use that one instead of using your own "query object".
For example something like this:
EntityQuery<YourEntity> query = from e in dx.GetYourEntityQuery()
where e.Foo == "something"
select e;
dx.Load<YourEntity>(query);