I want to store some data in the database. Then using those data I will answer the queries for the user using Dialog flow.
Any idea on implementing these
You will need to use a webhook to do fulfillment. In your webhook, you can make the database queries you want.
You may want to use an NLIDB (natural language interface to database). An NLIDB maps natural language questions over the database schema into SQL, solves such SQL queries and returns answers. Additional misconception and ambiguity resolution steps may be included.
NLIDBs are in contrast to dialog management systems (such as DialogFlow) which use interactive dialog to fill in slots for specific question types, and then execute these questions in specialized code. This specialized code may very well interact with a database, but it is relative to a specific question type so it is fairly straight forward to implement.
The advantage of NLIDBs however is that if the mapping tool is robust, a practically infinite number of questions may be answered over a complex database schema. The disadvantage is that the mapping tools are often sometimes less than robust. But this is an area under active R&D.
There are several companies currently offering NLIDB systems.
See for example: https://friendlydata.io/, http://c-phrase.com and http://kueri.me/.
AWS might be of help. I have some answers where I detail how to use API gateway for example, as a pseudo back-end so you can run this all from a front end ( or static ) page. DOing this, my hack would be to just write a JSON file or create a variable thats imported (key/vales) which would include your database info. I created a react page once where I used a long list of database data (SQL) which i just put in a json file and imported. worked great.
Of course if you have experience building a back end, you can figure all this out. if not, i would recommend looking into wix. They have a great platform, which you can use javascript in and it also has a node back end with access to node modules. they also have fully functional built in databases. good luck!
I am looking for a quick way of enabling data entry to a well structured SQL database.
We have defined all our data entry rules (by that I meean validation, mandatory fields etc.) and we're now looking for a tool to make it as easy as possible to have this database populated.
The database is largely product orientated, so the ability to add/edit products is important. Whenever a change to a field is made, it should be validated against the data rules to ensure it 'makes sense' before being written to the database. The ability for us to run reports on historical information is important, so the database must hang onto all of the products historical attirbutes too.
Eventually this data will be used for some standard reporting, and will be made available via a dashboard style reporting service that we will one day be able to afford! (hopefully)
Would Infopath be a good way of doing this? Are there any articles around the place that discuss a similar approach and how they were able to do it?
If not, what would users here suggest?
Background: I am from a business/product background and am trying to do some leg work myself to get a good outcome.
Regarding Infopath - I think it will depend on the complexity of your rules.
Regardless, you will need your database design to be able to record history - have you already determined a database design which is able to meet your requirements for storing the historical attributes?
You should consider your business requirements for reporting and any point-in-time consistency and the architecture should follow. Like you said, you've got a proper IT guy, so if you've got your business rules all sorted out, they should be able to critically evaluate it and expose any things you haven't thought about. I've become a big fan of having a transactional operational database (perhaps retaining full history, but not providing rich access to it) with limited reporting capabilities and a reporting database with a model (like Kimball's dimensional data warehouse) conducive to point-in-time reporting etc.
I have a database which manages login information for an application, and find myself (for rather tragic reasons involving combo boxes - don't ask) needing to manage the registration of new users through a web frontend. Quite literally all it needs to do is have a single form composed of a few edit boxes and a handful of combo boxes. Keeping in mind that I'm completely inept and possessed of functionally no previous experience as a web programmer, how would one suggest going about making this form? (If there's a solution that keeps web simplicity but is functionally heavy on C++, or less ideally Java, I'm all ears.)
As a bonus question, it would also be useful (but, I imagine, rather difficult) to have one of the combo boxes update with information pulled from a SELECT query to the database. Is this possible for a web design beginner like me?
If I understand you right, you just want a web interface to login/create a user account?
Since I have a heavy C++ background, I would have done that with PHP, CSS and SQL. With PHP you can access your database in any way you want with SQL. Use CSS to display faulty fields in the user interface.
I run an Apache server with PHP5 and MySQL.
I'm I on the right track? or did you mean something else?
If this is what you are looking for let me know and I can try to help you a bit further.
so as title says, I would like to hear your advices what are the most important questions to consider and ask end-users before designing database for their application. We are to make database-oriented app, with special attenion to pay on db security (access control, encryption, integrity, backups)... Database will also keep some personal information about people, which is considered sensitive by law regulations, so security must be good.
I worked on school projects with databases, but this is first time working "in real world", where this db security has real implications.
So I found some advices and questions to ask on internet, but here I always get best ones. All help appreciated!
Thank you!
Some other specifics besides what has already been said:
Do you have any Regulatory
requirements for data access and
storage (Sarbanes-Oxley and HIPAA
come to mind)
Do you need to be able to audit
record changes
What internal controls do you need
reflected in the database
What business rules must be followed
under what circumstances
How large to you expect the data to
get - the larger the data store
expected the more critical to design
with performance in mind from the
start
How flexible do you want the system
to be (do you want to be able to add
columns on the fly? OR add business
rules) Be careful with this one, make
sure the client understands that
flexibilty often comes at the cost of
performance.
Do you need a separate data warehouse
for reporting?
How do you need the data populated?
Will it come from an application,
multiple applications, data imports
or a combination?
What databases do you currently have
license for? Do you want to have
this application use it?
Will different groups of users need
different accesses?
How is the process currently being
handled, can we have access to that
database or see the current process
in action. Observe, for a minimum of
one day, the client using the current
system. Take extensive notes, you will learn many things no one will think to tell you.
Do you need to migrate data from the
old system
i would start with:
Please explain your business to me.
Which processes are you looking to
automate or improve?
Do you have any reports you need to
generate?
Do you need inputs to any other
systems?
use cases (google for that, it does not need to be drawings, text is fine)
inputs
outputs
static data
historical data
From there you derive the info you need to store, you apply 4th NF, and go !
Good luck ! 8-))
People seem to avoid building user interfaces that pull their information (names, field types, etc. as well as relationships) from a database; they instead hard-code forms (and tables, etc.) that have pretty much the same data names and types and things.
Am I making sense?
For instance, imagine an emumerated field in MySQL: why not just have the UI construct a drop-down list whenever it encounters an ENUM? Why put the same values in both the database and the code?
Perhaps I'm just missing something; perhaps there are projects out there that do this — sort of super-crud interfaces that can be pointed at any database and from it build a fully-functional relationally-aware user interface. Are there?
I'm possibly not quite conforming to the stackoverflow norms with this question; I shall summarise:
Can you please tell me of a project that constructs its user interface (solely) from analysis of the database schema?
Why is this not a common way to do it — surely it is good to only define data structure in one place (i.e. the database)?
Thank you, and may joyous code-love rain upon your IDE.
I'd like to point out that, last time I checked, .NET and Qt (and probably other environments) make it possible to use "database-aware widgets" (sometimes shortened to just data-aware widgets), which is probably the best pragmatic solution available. What I mean by data-aware widgets is that the widgets themselves know that they're linked directly to database fields, so you would have a combobox that knows that it's backed by an enum and fetches the possible values directly from the database at runtime, just like you suggested.
This is a really neat utility, and used well, it probably won't hurt anything. It still requires that you spend some time laying out widgets manually on a form, but then if you update the database to add a new value to that enum, you don't have to rebuild your app to see it show up in the UI.
But the reason most usability experts will cringe when they hear your question is because programmers tend to think that, well, why not just generate the entire UI, form layout and everything, from the database? And this is where it starts to get really nasty, really fast.
Let's say you have a simple Person table, with first_name, last_name, email_address, street_address, city, state, zip, and phone_number. You want to automatically generate a UI based on these fields. How do you sort the fields? I mean, ideally, first name and last name should be right next to one another. And it would look very silly if you had city and state before street address. So you have to add a new column to the table to specify sort order, if you go with the quickest method, or a new table to specify each field's order index to their field ID.
What if you want to group parts of the information separately? Then you have to add more UI-specific cruft into your database layout (to do this generically, you'll need a new table specifying which UI fields belong to which UI groupboxes). So you've only solved two problems and already your database layout has gotten twice as ugly, plus now instead of a simple O(1) layout operation when you load the UI, you've gotta do several database queries to find out what fields exist and dynamically lay them out while applying the correct widget order... and we haven't even dealt with sizing (should every field be the maximum size to fit its possible contents, or should all text fields be the same width? Wouldn't it be nice if you could say that some text fields should be one width and height, and some should be another combination? etc), or text justification, or formatting, or any other really common elementary usability requirements that will require further sacrifices from the clarity and simplicity of your database schema.
Most visual database editors. phpMyAdmin for instance.
Because the database structure isn't always a very good logical structure for a user to be using, especially in the case of databases that have been denormalised on purpose for efficiency reasons.
Yup, this route has already been traveled.
Simply pointing at a database will create an oversimplified UI, not giving much more than the CRUD of an Access UI. That's why Naked Objects (I'm one of its committers) builds its metamodel from a pojo domain model. This allows the UI to expose any public methods as menus ... we call this "behaviourally complete".
Per the comment about the UI not being suitable for end-users, I have two points:
distinguish between power users vs casual users. Most internal apps are for the former (we use Alan Coopers' term of a "sovereign application" for this), who understand the domain and don't want fancy UI stuff getting in the way. Most external apps, eg public web sites, are for the latter.
for the latter, there's nothing to prevent the autogenerated UI of a tool like Naked Objects being replaced with a custom or semi-customized viewer. One such viewer is Scimpi, I'm also working on an Eclipse RCP viewer that'll expose extension points. But even here, the auto gen UI is still very valuable for the development team and business analysts for exploration and prototyping.
Hope some of the above has piqued your interest. If you want more, google around, or you might want to check out my book on domain-driven design and NO, at pragprag.com.
HTH
Dan
List of projects that implement this idea.
.NET
dotObjects
Naked Objects
TrueView
Java
Domain Object Explorer
JMatter
Naked Objects
Sanssouci
Trails
Lablz
C++
Typical Objects
Specifically to your second question: Alot of it really depends on your data model. Some are very complicated and would lead to un-intuitive user interfaces. Perhaps for simply CRUD based systems, having your UI be a front end to the database would be preferable. In that case, I think that some of these tools would be great. However, for some more complicated systems where some db data needs to be hidden from the users, it would be better if you UI didn't mirror the db schema.
Microsoft Access has used this model for years - the database and UI development are very closely tied. You can auto-generate a form directly from a table definition with smart defaults and search built in. The model works well for developing applications with few concurrent users such as custom applications for small businesses where the amount of data stored is small.
If you are scaling to larger relational DBs with a number of concurrent users, or large databases then reliability and performance become more important, and separately constructed UI and databases make more sense. When more users are involved they often have different requirements so decoupling the UI from the DB schema makes it more efficient to develop.
Just a note on Java "projects that implement this idea" - tynamo is the new version of Trails framework
There are many systems that build an interface for you to edit stuff directly from table information. End-user interfaces, however, must be tweaked a little bit. You may not want to reveal to the user every field in your table.
Frameworks that make good use of the MVC design pattern can let you do all kinds of things with your models, which are the preferred way to build new systems (rather than creating database tables directly).
To answer your questions specifically:
django allows you to construct forms (and a complete admin CMS) out of models.
It is a common thing to do.
Naked Objects is about one step removed from this. They base the UI on an object model, and then persist the object model.
I think you are forgetting to consider the user in your design process if you are thinking like that. Bad mistake. Users don't like it when the interface changes, they would especially not like it if it changed frequently as they then wouldn't know what to do. Further, if you generate your UI on the fly based on the database structure, then what order would the objects be in? UIs need to have objects in an order that makes sense to the users not the database designers.
Further in a well-designed database there are fields that are not meant for the users to see. Things like numeric keys, insert date, last updated etc. You don't want to automatically expose these to the users and you certainly don't want them to have the ability to mess with the data in such fields.
Finally, if you don't think about the functionality of the page, then you aren't doing your job. A UI needs to be more than just a list of fileds that can be edited. You need to have constraints on who can see what, checks of the data before inserting to the database, business rules that need to be applied. You can't just autogenerate a lot of this (and you shouldn't even if you could!). Design needs thought and care.
Now as to drop down lists, of course you can generate them from the database and not the code, in fact it is the better choice. Just make the query the source for your particular object, not a list generated in code.
You can do it with the help of this cool tool from a developer in Philippines, it is called COBALT. You can download it here.