Related
I have a situation where I'm currently building a multitenant application where each tenant has its own database. The problem that I'm struggling with is that each database could have a vastly different structure. There are approximately 150 different tenants with each database being different. Is there a method of being able to cleanly manage this?
I'm aware of how to handle a single application with multiple databases but I'm lost on how I can implement this over several different schemas without the code base becoming unmaintainable. An initial idea would be separating the data layer so each tenant has it's own repositories and entities as an individual micro service. This isn't really scalable long term but would provide the separation of keeping the core logic separated from the individual databases but feels a bit hacky.
Are there any patterns, advice or examples where I could be pointed to so I can read further on this? Or is the method going forward just completely unfeasible?
I'm not asking anybody to write code for me or to provide a solution. I would just like a bit of advice for someone who may have worked with this kind of situation and potentially recommend steps forward.
I think creating a common data layer is your best bet. You absolutely don't have a scalable solution because you don't have a scalable problem. Hopefully they have enough in common that you can have a common mapping layer with plugins/overrides for particular entities that differ from that common core.
I am in process of designing my CMS that I am about to create. I was thinking about the database and how I want to go by approaching it.
Do you think its best to create 1 master database for all my clients websites? or Should I have 1 database per site?
What is the benefits and negatives on both approaches? I am always thinking about the future so I was thinking about implementing memcache or APC cache to the project, to offer an option to my client.
Just trying to learn the best practices and what other developers apporach would be
I've run both. My business chooses to separate client-specific data into separate tables so that if one happens to go corrupt, not all are taken down. In an ideal world this might never happen, but murphy's law....It does seem very easy to find things with them separated. You will know with 100% certainty that one client's content will never show up on another's page.
If you do go down that route, be prepared to create scripts that build and configure databases for you. There's nothing fun about building a great system and having demand for it, only to spend your time manually setting up DB's and installs all day long. Also, setting db names is one additional step that's not part of using a single db table--it's a headache that will repeat itself seemingly over and over again.
Develop the single master DB. It will take a small amount of additional effort and add a little bit more complexity to the database design, but will give you a few nice features. The biggest is being able to share data between sites.
Designing for a master database means that you have the option to combine sites when it makes sense, but also lets you install a master per site. Best of both worlds.
It depends greatly upon the amount of customization each client will require. If you forsee clients asking for many one-off features specific to their deployment, separate databases based off of a single core structure might make sense. I would highly recommend trying to make any customizations usable by all clients though, and keep all structure defined in one place/database instead of duplicating it across multiple databases. By using one database, you make updating the structure straightforward and the implementation consistent across all sites so they can all use the same CMS code.
What is the best way or recommended best practice in the flow of database driven asp.net web application? I mean the database first or coding first or side by side?
Your data access code won't compile without an existing database - unless you stub (or Mock) it. So probably the database comes first.
But it is a bad idea to do whole chunks of the application in isolation. Ideally you should design and build slivers of the system - database and application - hand-in-hand. These slivers should be cohesive sub-sets of functionality, probably smaller than sub-systems. Inevitably, the act of coding screens and business rules will throw up problems in the data model. So it is good to have a data modeller or DBA who is happy to work incrementally alongside the developers.
edit
Stephanie makes an extremely pertinent point:
"the core tables which are persisting
your app's data really can't be
piecemealed. Most of the data is known
at project start. It has a form, you
need to find it."
I agree that the core entities are knowable at project start, and the physical data model can be derived from that logical data model. But I don't think it is ever possible to nail down completely the structure of any table, even a core table, at the start. This is because at the start of the design/build phase all we have to go on are the Requirements, and if there's one thing history tells us about the Requirements it is that they will change.
So, new tables will be needed and some existing tables will become obsolete. There will be columns which need to be added, columns which need to be modified, columns which need to be dropped. This is why Nature gave us the ALTER TABLE statement.
I am not suggesting that we don't design our tables, or assemble them piecemeal. I am merely suggesting that when we start designing the HR sub-system we need to worry about the EMPLOYEES table and the SALARIES table. We don't need to concern ourselves with INVENTORY or ORDERS until we commence work on Sales.
We personally start with the Domain and do things side-by-side. The important part is that we implement vertical slices of the application (fully working end-to-end features), not horizontal slices (e.g. first the whole database layer, then the data access, then the services, then the presentation): we build the application incrementally and demonstrate progress with working code after each iteration.
Applications are all about features.
You don't build apps to store data,
but to provide functionality. If we
can't agree on that, the discussion is
moot of course. Software should be
developed to satisfy the needs of its
users and not of its developers.
Well I have really no understanding of the second sentence. If you think my company pays me a good salary to write code that satisfies me and not my users you're crazy. So that argument is a strawman. Back to the first.
This is a common view point of application centric people (they), vs. database centric people (We). They see the entire point of the exercise to "provide features". Those are things the clients know they want and ask for them. To them, the database is just persistence required for these features. And when they are done, that's it, features delivered, database is sufficient for those features. Could be an entire Rube Goldberg inside the database with redundant data, severe violations of normal forms, constraints enforced by the application, what have you.
think overall usability alone outweighs database design
If the design of your database is affecting your usability than the design was bad. I have no doubt that one who strives for features will leave the database in such a state that it severely hampers usability.
Data Centric people, don't look at a system as a place to provide only what's been asked for, but a repository of Intellectual Capital that can be exploited by more than whatever the Application-du-jour is. I can't begin to describe the number of cases where one team has used the database of some other team's app to enhance their apps value. Just look at all the medical research that is nothing more that the meta-analysis of existing studies. None of that is possible if you believe that only the features of your app matter and subsequent uses of your apps data do not.
A good data model isn't inviolate. Sure you'll add to it, change it when requirements change. But if you don't completely understand your data, I don't know how anyone can begin to write code.
I guess you need first to define datamodel and only then going coding. You should plan everything carefully before actually writting the code.
First is a feature list.
Then, detailed spec.
Then test plan and design of all, including databases.
Then, it wouldn't matter which to implement first.
You'll probably end up doing it "side by side".
You need some data to be able to test the application, but you need the application to be able to verify that you're storing the correct data.
Do some modelling first and then build the minimum you can for one or two features. Then when these are working add the next feature and so on.
You'll need to write some database update procedures (both the code and the rules about what and when to update) as you will have to extend your tables, but you'll need those for the final system anyway as it will have to change as new requirements come along.
Having done it quite a few times, I find myself invariably doing it like so:
Define the problem I'm trying to solve.
Write out some use-cases.
Have my significant other or a friend tell me if this is even a problem.
Sketch out a few sample screens.
Write flow diagrams for the use cases.
Ask my Rubber-duck questions.
Use questions to refine 1-6.
Write out the 'nouns'. Those become my data Model.
Write out the actions. Those become application logic.
Code data Model.
Code Application Logic.
Realize I've gotten it a little wrong.
Repeat 10-12 as many times as needed.
Ask, "Have I solved the problem"?
If not, rinse, lather and repeat 1-15.
This is a trick question. IMO, they both come in parallel during your planning and design phase. They are so closely related that it make sense to do them together. Just keep in mind that your database design will be almost fully developed while your code is still in its infancy (though your application logic should be almost fully mapped out in you head or on paper)
The idea is that you're designing your solution in the context of the problem. When you're planning out your solution you will be (or should be) defining your application as a set of things and actions (nouns and verbs).
For example, a very basic helpdesk program has people and tickets. People need to create tickets, update tickets, and close tickets. The nouns that require persistent storage will comprise your database, and the nouns + actions will be contained in your application.
Sometimes your table mappings and the relationship between tables will be obvious (IE people create tickets, ticket.creatorID = people.personID) and other times the relationship doesn't really click in your head until you start working through use cases or until you start writing your code (IE different ppl have different access levels defining what they can do. At a glance this would seem like a simple field in a table, but in practice it is better as a separate table).
The form that currently loads during when our beta WinForm application starts up is one that shows a vast array of buttons... "Inventory", "Customers", "Reports", etc. Nothing too exciting.
I usually begin UI by looking at similar software products to see how they get done, but as this is a corporate application, I really can't go downloading other corporate applications.
I'd love to give this form a bit of polish but I'm not really sure where to start. Any suggestions?
EDIT: I am trying to come up with multiple options to present to users, however, I'm drawing blanks as well. I can find a ton of design ideas for the web, but there really doesn't seem to be much for Windows form design.
I have found that given no option, users will have a hard time to say what they want. Once given an option, it's usually easier for them to find things to change. I would suggest making some paper sketches of potential user interfaces for you application. Then sit down with a few users and discuss around them. I would imagine that you would get more concrete ideas from the users that way.
Update
Just a couple of thoughts that may (or may not) help you get forward:
Don't get too hung up on the application being "corporate". Many coprorate applications that I have seen look so boring that I feel sorry for the users that need to see them for a good share of their day.
Look at your own favourite UI's and ask yourself why you like them.
While not getting stuck in the "corporate template", also do not get too creative; the users collected experience comes from other applications and it may be good if they can guess how things work without training.
Don't forget to take in inspiration from web sites that you find appealing and easy to use.
Try to find a logical "flow"; visualize things having the same conceptual functionality in a consistent way; this also helps the user do successful "guesswork".
You might look to other applications that your users are familiar with. Outlook is ubiquitous in my company, and we were able to map our application to its interface relatively easily, so we used that application as a model when developing our UI.
Note that I'm not suggesting Outlook specifically to you, just that you look for UIs that would make your users' learning curve shallower.
The problem here is that you need some good user analysis and I'm guessing you've only done functional analysis.
Because your problem is so abstract, it's hard to give one good example of what you need to do. I'd go to usability.gov and check out the usability methods link, especially card sorting and contextual interviews.
Basically you want to do two things:
1- Discover where your users think how information is grouped on the page: This will help flesh out your functional requirements too. Once you've got information all grouped up, you've basically got your navigation metaphor set up. Also, you can continually do card sorting exercises right down to page and function levels - e.g. you do one card sorting session to understand user needs, then you take one group of cards and ask users to break that down into ranks of importance. Doing so will help you understand what needs to be in dominate areas of the screen and what can be hidden.
2- Understand what tools they already use: what they do and don't like about them. You need to get a list of tools/applications that they use externally and internally. Internally is probably the most important because there is a fair chance that most people in your business will share an experience of using it. External tools however might help give you context into how your users think.
Also, don't be afraid to get pencil and paper and sketch up ideas with users. People generally understand that sketches are a quick and useful way to help with early design work and you can get an immense amount of information out of them with just simple sketches. Yes, even do this if you suck at sketching - chances are it won't matter. In fact, crappy sketches could even work in your favour because then nobody is going to argue if buttons should be blue, red or whatever.
Frankly, a form with a “vast array of buttons” needs more than a little polish. A form dedicated solely to navigation generally means you’re giving your users unnecessary work. Provide a pulldown or sidebar menu on each form for navigating to any form.
The work area of your starting form should provide users with something to actually accomplish their tasks. Among the options are:
A “dashboard” main form, showing summarized information about the users’ work (e.g., list of accounts to review and status of each, number of orders at each stage of processing, To Do schedule). Ideally, users should be able to perform their most common tasks directly in the opening form (e.g., mark each account as “approved” or not). If further information is necessary to complete a task, links navigate to detailed forms filled with the proper query results. At the very least users should be able to assess the status of their work without going any further. Note that different groups of users may need different things on their respective dashboards.
Default form or forms. Users of a corporate application typically have specific assignments, often involving only one to three of all your forms. Users who work with Inventory, for example, may almost never need to look at Customer records, and vice versa. Users also often work on a specific subset of records. Each sales rep, for example may be assigned a small portion of the total number of customers in the database. Divide your users into groups based on the forms and records they usually use. For each user group, start the app by automatically opening the user group’s form(s) populated with the query results of their records. Users should be able to complete most of their work without any further navigation or querying.
If all else fails, open the app to whatever forms and content were last open when the user quit the app. Many corporate users will continue to work tomorrow on the same or similar stuff they’re working on today.
Analyze the tasks of your users to determine which of the above options to use. It is generally not productive to describe each option to the users and ask which they like better.
BTW, “Reports” is probably not a particularly good navigation option. It’s better if you consistently identify things primarily by what they show, rather than how they show it. Users may not know that the information they want to see is in a “report” rather than a form, but they’ll know what content they want to see. Reports on inventory are accessed under Inventory; reports about sales are accessed under Sales.
Have you tried asking your end users what they would like? After all they are the ones that are going to be using the system.
I use components from the company DevExpress. They have some really cool controls (such as the Office 2007 ribbon), form skinning utilities (with a vast amount of different skins), and a load more...
If you want to check it out they have 60 free components - if its corporate though you might have to check the licence but you can get it at... DevExpress 60 Free
I suggest starting with the design principles suggested by Microsoft: Windows User Experience Interaction Guidelines
Some places to get ideas for interaction designs:
Books
About Face 3 - The Essentials of Interaction Design
Don't Make Me Think (this is focused on web design, but many of the principles carry over to Windows design)
Web Sites
Windows User Experience Interaction Guidelines
In addition, many applications have free trial versions that you can download to determine how they handle user interaction. Also, don't discount items on your desktop right now.
Do you have any statistics or insights concerning what the most commonly-used or important functions might be? If so, you could use that to pare down your "vast array of buttons" and highlight only those that are most important.
That's sort of a trivial example, but the underlying point is that your understanding of your audience should inform your design, at least from a functional perspective. You might have past usage statistics, or user stories, or documented workflows, or whatever - even if you're drawing a blank right now, remember that you have to know something about your users, otherwise you wouldn't be able to write software for them.
Building on what they already know can make it easy on your users. Do they live in Outlook? Then you might want to mimic that (as Michael Petrotta suggested). Do they typically do the same thing (within a given role) every time they use the app? Then look for a simple, streamlined interface. Are they power users? Then they'll likely want to be able to tweak and customize the interface. Maybe you even have different menu forms for different user roles.
At this stage, I wouldn't worry about getting it right; just relax and put something out there. It almost doesn't matter what you design, because if you have engaged users and you give them the option, they're going to want to change something (everything?) anyway. ;-)
I am going to write a database application for the camp I work for. I am thinking about writing it in C# with a Windows GUI interface but using a browser as the application is seeming more and more appelaing for various reasons. What I am wondering is why someone would not choose to write an application as a web application. Ex. The back button can cause you some trouble. Are there other things that ayone can think of?
There are plenty of cons:
Speed and responsiveness tend to be significantly worse
Complicated UI widgets (such as tree controls) are harder to do
Rendering graphics of any kind is pretty tricky, 3D graphics is even harder
You have to mess around with logins
A centralised server means clients always need network access
Security restrictions may cause you trouble
Browser incompatibilities can cause a lot of extra work
UI conventions are less well-defined on the web - users may find it harder to use
Client-side storage is limited
The question is.. do enough of those apply to your project to make web the wrong choice?
One thing that was not mentioned here is the level of complexity and knowledge required to generate a good web application. The problem being unless you are doing something very simple, there is no "Single" knowledge or technology that goes into these applications.
For example if you were to write an application for some client server platform.. you may develop in Java or C++. For a complex web application you may have to have expertise in Java, Java Script, HTML, Flash, CSS, Ajax, SQL, J2EE.. etc. Also the components of a web based application are also more numerous, Web Application Server, HTTP Server, Database, Browser.. are tipical components but there could be more.. a client server app is tipical just what it says.. a client application and a Server application. My experience and personal preference is not web based .. web based is great for many things. But even though I am an IT Architect for a leading company that is completely emersed in Web Apps as the solution for everything... The cons are many still.. I do thing the technology will evolve and the cons will go away over time though.
Essentially the real limitations are only through of the platform, being the browser. If you have to account for all browsers in current use that can be a pain due to varying degrees of standards in each of them.
If have control of the which browser to use, that is everyone is on computers that you control on site, and say you install firefox on all of them, you could then leverage the latest Javascript and CSS standards to their fullest in your content delivery.
[edit] You could also look into options like the adobe integrated runtime or "AIR" as an option allowing you to code the front-end with traditional browser based options like xhtml/css/javascript, flash/flex and have the backend hooked up to your database online, only also providing functionality of a traditional desktop app at the same time.
The biggest difference and drawback I see with web applications is state management. Since the web is, by nature, stateless every thing you want to maintain has to be sent back and forth from the server with every request and response. How to efficiently store and retrieve it in a matter with respect to page size and performance is hard to do at times. Also the fact that there is no real standard (at least not that everyone adheres to) for browsers makes consistency really..........fun.
You need to have a network access to the server that you are going to have the web application on (if there are going to be multiple users for the application - which is typically the case).
Actually, there are more pros than cons - if you can give some details about your application, we could help a little more...
It completely depends on the requirements of your project. For the most part, there isn't much web applications cannot do these days. Admittedly, certain applications do belong on the desktop as browsers (while currently advancing, and rapidly), still are not quite there yet. From the advent of applications such as Google Docs, Gmail
There isn't much you -cannot- do on the web. If you're creating a World of Warcraft competitor however, the web is most certainly not the optimal solution. Again, unfortunately we'd need more insight on the application you're building for the camp. The best part about the web is that anyone with a browser can use your application.
Web applications delegate processing to a remote machine. Depending on the amount of processing, this can be a con. Consider a photo editor that's a web app.
Web applications also can't deal with a whole lot of data going back and forth to and from a client. You can watch video online.. when it's compressed. It will be awhile before we see any web-based video editing software.
Browser compatibility is also a hassle. You can't control the look-and-feel of the application 100%.
Vaibhav has a good point. What's your application?
A major one is down time for migrations... users will not expect the application to be down, ever, but realistically it will have to be down for major upgrades. When doing this with a desktop application, the user (or end-user systems admin) is in control of when upgrades happen; with an online app, they're not.
For applications which have large data, performance can be a major problem as you're storing a large number of users' data centrally, which means the IO performance will not be as good as it would be if you gave them all a laptop.
In general scalability gives problems for a server-based app. Desktop applications scale really well.
You can do an awful lot with a web-based app, but it is a lot easier to do certain things with a thick client:
Performance: You get simple access to the full power of the client's CPU.
Responsiveness: Interactivity is fast and easy.
Graphics: You can easily use graphics libraries such as DirectX and OpenGL to create fast impressive graphics.
Work with local files
Peer-to-peer
Deciding whether a web application is a good approach depends on what you are trying to achieve. However here are some more general cons of web applications:
Real integration with desktop apps (e.g. Outlook) is impossible
Drag and drop between your app and the desktop / other running apps
With a web application, there are more privacy concerns, when you are storing user data on your servers. You have to make sure that you don't loose/disclose it and your users have to be comfortable with the idea of storing that data on your servers.
Apart from that, there are many security problems, like Man-in-the-middle attacks, XSS or SQL injections.
You also need to make sure that you have enough computing power and bandwidth at hand.
"Ex. The back button can cause you some trouble."
You'll have to be specific on this. A lot of people make fundamental mistakes in their web applications and introduce bugs in how they handle transactions. If you do not use "Redirect after Post" (also known as Post-Redirect-Get, PRG design), then you've created a bug which appears as a problem with the back button.
A blanket statement that the back button in trouble is unlikely to be true. A specific example would clarify your specific question on this.
The back button really is not that much of an issue if you design your application correctly. You can use AJAX to manipulate parts of the current page, without adding items into the browser history (since the page itself wont change).
The biggest issue with designing web applications has to do with state, and the challenges that need to be programmed around. With a desktop application, state is easy to handle, you can leave a database connection opened, lock the record and wait for the user to make the changes and commit. With a web application, you could lock the record...but then what if the user closes the browser? These things must be overcome in the design of your application.
When designing a web application, make sure that each trip to the server "stands alone" and provides a complete answer. Always re-initialize your variables before performing any work and never assume anything. One of the challenges I ran into once was pulling "pages" of grid data back to the user. In a real busy system, with record additions/modifications happening in real time, the user navigation from page to page would vary greatly, sometimes even resulting in viewing the same set of a few records as new additions were added in-front of the query.