I am working on an application which will require one or more additional fields to be added to a table in order to track user defined information. This additional info is only used for reporting purposes(Crystal Reports), and will have no effect on the behavior of the application. The data for this field is populated from an outside application.
What would be the best way to handle this additional information? Here are some options based off of other SO answers:
Entity-Attribute-Value (would this be overkill? Seems like there are many critics of EAV)
Add additional column to table (not sure how Entity Framework would like this)
Create a new Table for each UDF and use primary of parent table to link
If I understand the requirement correctly, you will need a datapoint to save information that would come from an external application and its structure is undefined at design time. if that is correct, then I would suggest using an xml datatype. by choosing this, you will not need to redesign your database in future when new key value pairs are inserted. Crystal reports should be easily able to include an xsl for this column.
Hope this help and good luck
Related
In an application we inherited (Winforms) based on DevExpress, an object of type UnitOfWork is used to keep track and save multiples records in the database.
Usually around 100 objects can be saved in the database on a button click using the method UnitOfWork.CommitChanges();
The table where the records are inserted has an unique constraint on a column.
It might happen that different users try to treat the same entities and to try to enter in that table the same value in the unique column.
So definitively before using UnitOfWork.CommitChanges() we should test if one or more values do not exist already in the database.
What would be the best approach to test if one or more objects are not already in the database before calling UnitOfWork.CommitChanges(), so that we can surely warn the user on his validation?
Thank you
It's not really the DevExpress way of working, see the help topic on auto-generated keys.
However, since you have inherited the code, you could use UnitOfWork.GetObjectsToSave() to obtain an ICollection of the IXPObjects you are about to commit. From this collection you could generate a list of potential key conflicts. Then you could use UnitOfWork.ExecuteScalar() to run a direct SQL command to determine if there are conflicts and resolve them.
The UnitOfWork.CommitChanges() calls CommitTransaction, so you could perform this check in one of the UnitOfWork events such as UnitOfWork.BeforeCommitTransaction.
As with everything DevExpress, your best support option is the DevExpress Support Center.
The functionality you ask for is not possible to implement in a multiuser system. As every user (necessarily) works in his own database transaction, there is always a chance for a user to insert a datarow with a key that another user inserted right before. You could only avoid this by employing database mechanisms like autogenerated keys. But I think this is not applicable for you.
So I suggest to catch the exception thrown by commitChanges and act upon it.
Description of Goal
Trying to make a CMS (for internal use, but on many sites) with CakePHP.
I'd like to be able to have a generic Content model, then have many different user-generated content types.
This would be simple to do, but my single contents table would eventually become massive if every piece of content on the site was in it. I'd like to split it into tables to help query times/site speed...etc.
My thought (not sure if possible) would be to somehow tell CakePHP that if the type field of the Content is "article", that it should use the content_articles table...etc These tables would be generated afterSave (I suppose) when creating a new content_type.
Would be nice to give them options of which fields the specific content-type would use - even manage this by adding/removing fields...etc, then only generate those fields in the table, and somehow do validation on them based on the content_fields table data.
//my thoughts on tables:
content_types //id, name, description, use_table
content_fields //id, name, content_type_id, required, field_type, max_chars
content_articles //generated by code
content_people //generated by code
Questions:
Is it even possible? Are there better ways to go about this?
Perhapse use a key value table for content rather than a standard table? The utils plugin from CakeDC can do just that with a supported RDBMS.
Or, you could set this model to use a key value data source like MongoDB, which is a great use case for using NoSQL. I'd probably take that approach if you are talking about massive key value stores and a changing schema. There's a plugin for MongoDb on github.
I need to query tables neither known or existing at compile time, publish the table via odata and then make it available to a silverlight client for CRUD.
Would be wonderful to use a PCO of type dynamic or ExpandoObject to acheive this but that doesn't seem to work (as suspected).
I'm wondering if there are Interfaces that would allow me to perform the type mapping and serializing at the row level so I would dynamically take the data row and round trip it's values on the server side. Perhaps an interface for the PCO to "help", or dynmically created property getter/setter. I'm also toying with dynamically creating the context class at run time but that's kind of ugly.
Then - on the client side, something to do the same thing with the odata feed, I have a solution here but it aint pretty enough to share with the world.
EF doesn't offer any "dynamic" approach as well as any simple way to let you create a new table and add it to mapping. Another question is how well can WCF Data Services work with changing data - I believe it is not supported as well.
If you want dynamically changed structure (added tables, columns etc.) use some metadata model instead of creating new table for each entity. Metadata model usually have something like table with common properties and related table with key value pair of attribute name and value. It can be futher extended to more complex scenarios but it is the only way how to achieve this. Instead of mapping in EF take your entity types as data.
I have an MS Access database with plenty of data. It's used by an application me and my team are developing. However, we've never added any foreign keys to this database because we could control relations from the code itself. Never had any problems with this, probably never will either.
However, as development has developed further, I fear there's a risk of losing sight over all the relationships between the 30+ tables, even though we use well-normalized data. So it would be a good idea go get at least the relations between the tables documented.
Altova has created DatabaseSpy which can show the structure of a database but without the relations, there isn't much to display. I could still use to add relations to it all but I don't want to modify the database itself.
Is there any software that can analyse a database by it's structures and data and then do a best-guess about its relations? (Just as documentation, not to modify the database.)
This application was created more than 10 years ago and has over 3000 paying customers who all use it. It's actually document-based, using an XML document for it's internal storage. The database is just used as storage and a single import/export routine converts it back and to XML. Unfortunately, the XML structure isn't very practical to use for documentation and there's a second layer around this XML document to expose it as an object model. This object model is far from perfect too, but that's what 10 years of development can do to an application. We do want to improve it but this takes time and we can't disappoint the current users by delaying new updates.Basically, we're stuck with its current design and to improve it, we need to make sure things are well-documented. That's what I'm working on now.
Only 30+ tables? Shouldn't take but a half hour or an hour to create all the relationships required. Which I'd urge you to do. Yes, I know that you state your code checks for those. But what if you've missed some? What if there are indeed orphaned records? How are you going to know? Or do you have bullet proof routines which go through all your tables looking for all these problems?
Use a largish 23" LCD monitor and have at it.
If your database does not have relationships defined somewhere other than code, there is no real way to guess how tables relate to each other.
Worse, you can't know the type of relationship and whether cascading of update and deletion should occur or not.
Having said that, if you followed some strict rules for naming your foreign key fields, then it could be possible to reconstruct the structure of the relationships.
For instance, I use a scheme like this one:
Table Product
- Field ID /* The Unique ID for a Product */
- Field Designation
- Field Cost
Table Order
- Field ID /* the unique ID for an Order */
- Field ProductID
- Field Quantity
The relationship is easy to detect when looking at the Order: Order.ProductID is related to Product.ID and this can easily be ascertain from code, going through each field.
If you have a similar scheme, then how much you can get out of it depends on how well you follow your own convention, but it could go to 100% accuracy although you're probably have some exceptions (that you can build-in your code or, better, look-up somewhere).
The other solution is if each of your table's unique ID is following a different numbering scheme.
Say your Order.ID is in fact following a scheme like OR001, OR002, etc and Product.ID follows PD001, PD002, etc.
In that case, going through all fields in all tables, you can search for FK records that match each PK.
If you're following a sane convention for naming your fields and tables, then you can probably automate the discovery of the relations between them, store that in a table and manually go through to make corrections.
Once you're done, use that result table to actually build the relationships from code using the Database.CreateRelation() method (look up the Access documentation, there is sample code for it).
You can build a small piece of VBA code, divided in 2 parts:
Step 1 implements the database relations with the database.createrelation method
Step 2 deleted all created relations with the database.delete command
As Tony said, 30 tables are not that much, and the script should be easy to set. Once this set, stop the process after step 1, run the access documenter (tools\analyse\documenter) to get your documentation ready, launch step 2. Your database will then be unchanged and your documentation ready.
I advise you to keep this code and run it regularly against your database to check that your relational model sticks to the data.
There might be a tool out there that might be able to "guess" the relations but I doubt it. Frankly I am scared of databases without proper foreign keys in particular and multi user apps that uses Access as a DBMS as well.
I guess that the app must be some sort of internal tool, otherwise I would suggest that you move to a proper DBMS ( SQL Express is for free) and adds the foreign keys.
I work for a billing service that uses some complicated mainframe-based billing software for it's core services. We have all kinds of codes we set up that are used for tracking things: payment codes, provider codes, write-off codes, etc... Each type of code has a completely different set of data items that control what the code does and how it behaves.
I am tasked with building a new system for tracking changes made to these codes. We want to know who requested what code, who/when it was reviewed, approved, and implemented, and what the exact setup looked like for that code. The current process only tracks two of the different types of code. This project will add immediate support for a third, with the goal of also making it easy to add additional code types into the same process at a later date. My design conundrum is that each code type has a different set of data that needs to be configured with it, of varying complexity. So I have a few choices available:
I could give each code type it's own table(s) and build them independently. Considering we only have three codes I'm concerned about at the moment, this would be simplest. However, this concept has already failed or I wouldn't be building a new system in the first place. It's also weak in that the code involved in writing generic source code at the presentation level to display request data for any code type (even those not yet implemented) is not trivial.
Build a db schema capable of storing the data points associated with each code type: not only values, but what type they are and how they should be displayed (dropdown list from an enum of some kind). I have a decent db schema for this started, but it just feels wrong: overly complicated to query and maintain, and it ultimately requires a custom query to view full data in nice tabular for for each code type anyway.
Storing the data points for each code request as xml. This greatly simplifies the database design and will hopefully make it easier to build the interface: just set up a schema for each code type. Then have code that validates requests to their schema, transforms a schema into display widgets and maps an actual request item onto the display. What this item lacks is how to handle changes to the schema.
My questions are: how would you do it? Am I missing any big design options? Any other pros/cons to those choices?
My current inclination is to go with the xml option. Given the schema updates are expected but extremely infrequent (probably less than one per code type per 18 months), should I just build it to assume the schema never changes, but so that I can easily add support for a changing schema later? What would that look like in SQL Server 2000 (we're moving to SQL Server 2005, but that won't be ready until after this project is supposed to be completed)?
[Update]:
One reason I'm thinking xml is that some of the data will be complex: nested/conditional data, enumerated drop down lists, etc. But I really don't need to query any of it. So I was thinking it would be easier to define this data in xml schemas.
However, le dorfier's point about introducing a whole new technology hit very close to home. We currently use very little xml anywhere. That's slowly changing, but at the moment this would look a little out of place.
I'm also not entirely sure how to build an input form from a schema, and then merge a record that matches that schema into the form in an elegant way. It will be very common to only store a partially-completed record and so I don't want to build the form from the record itself. That's a topic for a different question, though.
Based on all the comments so far Xml is still the leading candidate. Separate tables may be as good or better, but I have the feeling that my manager would see that as not different or generic enough compared to what we're currently doing.
There is no simple, generic solution to a complex, meticulous problem. You can't have both simple storage and simple app logic at the same time. Either the database structure must be complex, or else your app must be complex as it interprets the data.
I outline five solution to this general problem in "product table, many kind of product, each product have many parameters."
For your situation, I would lean toward Concrete Table Inheritance or Serialized LOB (the XML solution).
The reason that XML might be a good solution is that:
You don't need to use SQL to pick out individual fields; you're always going to display the whole form.
Your XML can annotate fields for data type, user interface control, etc.
But of course you need to add code to parse and validate the XML. You should use an XML schema to help with this. In which case you're just replacing one technology for enforcing data organization (RDBMS) with another (XML schema).
You could also use an RDF solution instead of an RDBMS. In RDF, metadata is queriable and extensible, and you can model entities with "facts" about them. For example:
Payment code XYZ contains attribute TradeCredit (Net-30, Net-60, etc.)
Attribute TradeCredit is of type CalendarInterval
Type CalendarInterval is displayed as a drop-down
.. and so on
Re your comments: Yeah, I am wary of any solution that uses XML. To paraphrase Jamie Zawinski:
Some people, when confronted with a problem, think "I know, I'll use XML." Now they have two problems.
Another solution would be to invent a little Domain-Specific Language to describe your forms. Use that to generate the user-interface. Then use the database only to store the values for form data instances.
Why do you say "this concept has already failed or I wouldn't be building a new system in the first place"? Is it because you suspect there must be a scheme for handling them in common?
Else I'd say to continue the existing philosophy, and establish additional tables. At least it would be sharing an existing pattern and maintaining some consistency in that respect.
Do a web search on "generalized specialized relational modeling". You'll find articles on how to set up tables that store the attributes of each kind of code, and the attributes common to all codes.
If you’re interested in object modeling, just search on “generalized specialized object modeling”.