we are considering creating an API app that our internal apps and customers can use. This seems simple for CRUD API queries, but we are having problems with more complex queries.
For example, suppose an internal app wants to know "for each company, count number of users and number of dashboards, and return the results", how would that be exposed as part of the API?
I would imagine performance becomes an issue if we issue one query for list of all companies, then fire two queries per company to count users and dashboards.
Also, how do we deal with issues where we are currently hardcoding SQLs for optimization?
Any recommended readings are also appreciated.
I generate crud stored procedures for all database tables including selects by all foreign keys and then generate domain objects for all database tables. To me, object mapping feels more scalable to work with.
If there is a particular view of data that will cause problems or does not fit the model. I would extend an existing domain object with the extended data. For example:
public class Customer
{
public int CustomerID{get;set;}
}
public class CustomerListingView:Customer
{
public int NumberOfOrders{get;set;}
public int NumberOfSomethingElse{get;set;}
}
This will at least allow you to cast your custom view data to controller functions requiring the super type.
Related
I have been studying microservice architecture for a while. But I have a few questions on my mind.
If you need to give an example, they are
order-service
customer-service
Product-service
Suppose there are 3 microservices above. They are using relational databases.
I list orders in the order-service. But I also have to pull customer informations here.
If this were a monolotic structure, I could handle it with join. But how can I do that in microservis architecture.
Note: I’m not doing any projects. My goal is only to understand the microservice architecture.
Options:
Limit the dependency between orderservice and customerservice: normaly the order is a self containing object that has all the customers data (from the time of ordering) in it.
If still needed the order should have the id of the customer saved and then any UI or logic that want to access recent customer data need to use the "public api" of the customer service. The "public api" in general can be anything - it even can be a defined shared storage (like a database). However most teams decide to not allow direct access to the technical storage to avoid tight coupling. Thats why most of the times service talk Rest (or GRPC) for syncronous use cases or use some form of messaging for async interactions
However - decide why you want to split it up - are you expecting a growing developer base and high complexity? If not a monolith might be cheaper to build for your case..
But how can I do that in microservis architecture.
Just by calling another microservice and asking for required additional information.
Normally microservices do not share database, as you noticed.
So if you have a class Order like that
class Order
{
OrderId;
ItemName;
UserName;
}
And a method that returns an order GetOrder(id) like that
GetOrder(orderId)
{
item = ItemMicroserice.GetItem();
user = UserMicroservice.GetUser();
result = new Order()
{
OrderId = orderId,
ItemName = item.Name,
UserName = user.Name
}
return result;
}
You can notice that there are two calls to other microservices that will return data to construct Order object.
Thought you can see that it can be slightly not optimal in sense of performance. So sometimes microservices do store duplicate information to be able to construct objects faster (eliminate calls to other microservices). And, for example, if Users microservice updates data, it sends an event to Orders microservice so it can update cached data from other microservices.
Im at the beginning of my first "real" software project, and I'd like to start off right. The concept of DDD seems like a very clean approach which separates the various software parts, however im having trouble implementing this in reality.
My Software is measurement tracker and essentially stores list of measurement data, consisting of a timestamp and the data value.
My Domain Models
class MeasurementDM{
string Name{get;set;}
List<MeasurementPointDM> MeasurementPoints{get;set;}
}
class MeasurementPointDM{
DateTime Time{get;set;}
double Value{get;set;}
}
My Persistence Models:
class MeasurementPM{
string Id{get;set;} //Primary key
string Name{get;set;} //Data from DomainModel to store
}
class MeasurementPointPM{
string Id{get;set;} //Primary Key
string MeasurementId{get;set;} //Key of Parent measurement
}
I now have the following issues:
1) Because I want to keep my Domain Models pure, I don't want or need the Database Keys inside those classes. This is no problem when building my Domain models from the Database, but I don't understand how to store them, as the Domain Model no longer knows the Database Id. Should I be including this in the Domain model anyway? Should I create a Dictionary mapping Domain objects to Database ids when i retreive them from the Database?
2)The measurement points essentially have the same Id problem as the measurements themselves. Additionally I'm not sure what the right way is to store the MeasurementPoints themselves. Above, each MeasurementPointPM knows to which MeasurementPM it belongs. When I query, I simply select MeasurementPoints based on their Measurement key. Is this a valid way to store such data? It seems like this will explode as more and more measurements are added. Would I be better off serializing my list of MeasurementPoints to a string, and storing the whole list as an nvarchar? This would make adding and removing datapoints more difficult, as Id always need to deserialize, reserialize the whole list
I'm having difficulty finding a good example of DDD that handles these problems, and hopefully someone out there can help me out.
My Software is measurement tracker and essentially stores list of measurement data, consisting of a timestamp and the data value.
You may want to have a careful think about whether you are describing a service or a database. If your primary use case is storing information that comes from somewhere else, then introducing a domain model into the mix may not make your life any better.
Domain models test to be interesting when new information interacts with old information. So if all you have are data structures, it's going to be hard to discover a good model (because the critical element -- how the model entities change over time -- is missing).
That said....
I don't understand how to store them, as the Domain Model no longer knows the Database Id.
This isn't your fault. The literature sucks.
The most common answer is that _people are allowing their models to be polluted with O/RM concerns. For instance, if you look at the Cargo entity from the Citerus sample application, you'll find these lines hidden at the bottom:
Cargo() {
// Needed by Hibernate
}
// Auto-generated surrogate key
private Long id;
This is an indirect consequence of the fact that the "repository" pattern provides the illusion of an in-memory collection of objects that maintain their own state, when the reality under the covers is that you are copying values between memory and durable storage.
Which is to say, if you want a clean domain model, then you are going to need a separate in memory representation for your stored data, and functions to translate back and forth between the two.
Put another way, what you are running into is a violation of the Single Responsibility Principle -- if you are using the same types to model your domain that you use to manage your persistence, the result is going to be a mix of the two concerns.
So essentially you would say that some minimal pollution of the domain model, for example an Id, is standard practice.
Less strong; I would say that it is a common practice. Fundamentally, a lot of people, particularly in the early stages of a project, don't value having a boundary between their domain model and their persistence plumbing.
Could it make sense to have every Domain Model inherit from a base class or implement an interface that forces the creation of Unique Id?
It could. There are a lot of examples on the web where domain entities extend some generic Entity or Aggregate pattern.
The really interesting questions are
What are the immediate costs and benefits of doing that?
What are the deferred costs and benefits of doing that?
In particular, does that make things easier or harder to change?
I am using Google App Engine (Java) for my REST backend and google-datastore as the database and using objectify to access the database.
I want to create a unit entity to store Units where a unit can be a component or an assembled unit , basically an assembled unit is made up of multiple components and also has some properties of its own. There can be multiple types of assembled units and multiple types of components.
The Entity class would be something like
public class UnitEntity {
Long unitId;
String serialNumber;
String state;
String unitType;
Long parentId;// -> this would be null for all assembled units and in case of components, if they are part of any assembled unit, it will be the Id of the assembled unit (This is added so that I can list all components of a particular assembled unit)
UnitParameters unitParameters;
}
Here UnitParameters would be a polymorphic class to contain properties specific to a unit type, that is, based on the value of "unitType", there would be different classes which extend "UnitParameters".
Let's assume one the components (let's say component1, that is , unitType=component1) has a property called modelNumber. This property would be stored in the unitParameters of the all the entities where unitType=component1.
Now I want to able to list units where unitType=assembledUnit1 and which have a child component1 whose modelNumber is 2.0.
(I can easily get list units of type component1 where modelNumber is 2.0 , but I want to be able to get the parent entity also)
So basically here I am trying to get parent entities by filtering on the properties of children.
I want to know whether this is possible with datastore and objectify? Is there any way to achieve this functionality?
Update - Follow-up question based on the Answer by #stickfigure:
If I go with google cloud sql (which is based on mysql) for my use case, then how should I model my data ?
I initially thought of having a table for each unitType. Let's say there are 3 unitTypes - assembledUnit1, assembledUnit2 and component1. Now if I want to have an API which lists the details of each unit , how can I achieve this with cloud sql.
This is something I could have done with datastore since all the entities were of the same "kind".
I can obviously have separate APIs to list all units of type assembledUnit1, assembledUnit2 etc., but how can I have a single API which could list could list all the units ?
Also in this approach, if someone calls the REST API GET /units/{unitId} , I suppose I would have to check for the unitId in each of the tables which doesn't seem correct?
I suppose one way by which this could be solved is to just have one table called "Unit" whose columns would be a superset of the columns of all the unitTypes. However I don't think this is a good way of designing since there would be a lot of empty columns for each row and also the schema would have to be changed if a new unitType is added.
The datastore doesn’t do joins. So you’re left with two options, either 1) do the join yourself via fetching or 2) denormalize some of the child data into the parent and index it there. Which strategy works best will vary depending on the shape of your data and performance/cost considerations.
I should add there is a third option which is “store an external index of some of your data in another kind of database, such as the search api or an RDBMS”.
This is not always a very satisfying answer - the ability to do joins and aggregations in an RDBMS is incredibly useful. If you have highly relational data with modest size/traffic/reliability requirements, you may want to use something like Cloud SQL instead.
I'm working on a 2-tier application where WinForms client makes direct calls to the database. In one of the scenarios I need to display a list of Customer entities to a user. The problem is that Customer entity contains a lot of properties (some quite heavy) and I need only two of them - first and last names. So to improve performance and make presentation logic clearer I want to create some kind of a CustomerSummaryViewModel class with required properties only and use NHibernate's projections feature to load it. My concern here is that in this case my data access logic becomes coupled with presentation and to me it seems conceptually wrong.
Do you think this is ok or there are better solutions?
I think you can consider the CustomerSummaryViewModel as report (CustomerSummaryReport). It is fine to query your entities for scenario's like this and treat them as reports. Most reports are more complex, using multiple entities and aggregate queries. This report is very simple, but you can still use it like a report.
You also mention that the performance is significant. That is another reason to use a separate reporting query and DTO. The customer entity sounds like one of the "main" entities you use. That it takes a significant amount of time to retrieve them from the database with lazy-loaded properties not initialized, can be a warning to optimize the customer entity itself, instead using reporting queries to retrieve information about them. Just a warning because I have seen cases where this was needed.
By the way, you can consider linq instead of projections for the easier syntax like:
var reports = session.Linq<Customer>()
.Where(condition)
.Select(customer => new Report
{
FirstName = customer.FirstName,
LastName = customer.LastName
})
.ToList();
I've been following a mostly DDD methodology for this project, so, like any DDD'er, I created my domain model classes first. My intention is to use these POCO's as my LINQ-to-SQL entities (yes, they're not pure POCO's, but I'm ok with that). I've started creating the database schema and external mapping XML file, but I'm running into some issues with modeling the entities' relationships and associations.
An artifact represents a document. Artifacts can be associated with either a Task or a Case. The Case entity looks like this:
public class Case
{
private EntitySet<Artifact> _Artifacts;
public IList<Artifact> Artifacts
{
get
{
return _Artifacts;
}
set
{
_Artifacts.Assign(value);
}
}
.
.
.
}
Since an Artifact can be associated with either a Case, or a Task, I've the option to use inheritance on the Artifact class to create CaseArtifact and TaskArtifact derived classes. The only difference between the two classes, however, would be the presence of a Case field or a Task field. In the database of course, I would have a single table, Artifact, with a type discriminator field and the CaseId and TaskId fields.
My question: is this a valid approach to solving this problem, or would creating a join table for each association (2 new tables, total) be a better approach?
I would probably go with two tables - it makes the referential integrity-PK/FKs a little simpler to handle in the database, since you won't have to have a complex constraint based on the selector column.
(to reply to your comment - I ran out of space so post here as an edit) My overall philosophy is that the database should be modelled with database best practices (protect your perimeter and ensure database consistency, using as much RI and constraints as possible, provide all access through SPs, log activity as necessary, control all modes of access, use triggers where necessary) and the object model should be modelled with OOP best practices to provide a powerful and consistent API. It's the job of your SPs/data-access layer to handle the impedance mismatch.
If you just persist a well-designed object model to a database, your database won't have much intrinsic value (difficult to data mine, report, warehouse, metadata vague, etc) when viewed without going through the lens of the object model - this is fine for some application, typically not for mine.
If you just mimic a well-designed database structure in your application, without providing a rich OO API, your application will be difficult to maintain and the internal strucutres will be awkward to deal with - typically very procedural, rigid and with a lot of code duplication.
I would consider finding commonalities in between case and task, for the lack of better word let's call it "CaseTask" and then sub-typing (inheriting) from that one. After that you attach document to the super-type.
UPDATE (after comment):
I would then consider something like this. Each document can be attached to several cases or tasks.