I've been looking around for a while but didn't happen to come across any official name (not even an unofficial one) of a document that summarizes all the tables in the database structure including the field names and a brief description of the purpose of this field within the table.
Does such a document actually have an official or commonly used name? I made one to help myself understand the current database design of a company while designing the new one, and also made one of the new database to help others understand the new database design for the company. If this kind of document has an official name then I could have a look in a certain document to ensure mine meets the requirements for it and making maybe more informative/professional.
The document is formally called Data Dictionary.
A platform-specific example is Data Description Specifications (DDS) that allow the developer to describe data attributes in file descriptions that are external to the application program that processes the data, in the context of an IBM System
Related
Is there a way in snowflake to describe/catalog schema so it is easier to search? For example, column name is "s_id", which really represents "system identifier". Is there a way to specify that s_id is "system identifier" and user can search "system identifier", which would return s_id as column name.
Just about every object in Snowflake allows for a comment to be added. You could leverage the comment field to be more descriptive of the columns intent, which can then be searched in information_schema.columns. I've seen many Snowflake customers use the comments as a full-blown data dictionary. My only suggestion is to not use any apostrophe's in your comments, as they screw up your get_ddl() results.
Adding comments along with table and field can or may serve the purpose but if you are in a complex enterprise environment and a lot of groups are part of it, it will be hard to follow this approach for long. Either you must have single ownership for data models tool which promotes such changes to the snowflake. Managing such information via comment is error-prone and that's why snowflake has partners which help on the different development requirement.
If you go to their partner website (link), you will find a lot of companies that have good integration with SF and one of them is Alation (link). You can try it for free for 30 days and see if that serves your purpose of data cataloging or not.
I am currently studying database as my core subject in my institute. I was going through some basic concepts of database and I found this written here.
The description of a database is also called Schema. Schema are frequently changed when any entry in database is added or deleted .
According to my knowledge schema is the definition of a database and changes infrequently. It is sort of a structure description of a database just like a blueprint, whereas entry is process of entering data into the database, so how can it affect the schema of the database?
Your understanding is correct: a schema is a structural definition which changes infrequently due to the difficulty of applying changes. It certainly doesn't change every time you write data. However, "entry" is not really a technical term. You're taking it to mean a record in a table, which is what I'd generally expect too. It looks as if the author of those notes speaks English as a second or later language and is using it to mean a structural component like a table or view (in technical terms, a relation). This is the first time I've seen this usage, and I wouldn't follow it because most people will, like you, think entry = record.
I am trying to get my head around the common patterns for database abstraction.
So far I've found:
Database Layer
just a separate class which holds the SQL
does not conform to any other rules
Data Access Object (DAO)
like above but there is a transfer object which represents the columns of the database table
create, delete, update methods take the filled transfer object as input
the find methods may take an input like string (findByName) or an integer (findByAge) but always return lists of transfer objects
Repository
abstraction of a collection of objects
closer to the domain model
I need to read more here
Object Relational Mapper
tool which gives me an object which is mapped to the database table in the background
the object represents a row in the table
a property change of the object leads to an update
Please don't worry too much about my quick explanations of the patterns. I am still in an understanding phase.
But is this list complete or are there other concepts which are missing here?
Martin Fowler's "Patterns of Enterprise Application Architecture" is an excellent book, well respected in the community, which documents about fifty design patterns, around half of which are concerned with interacting with databases. It includes Repository, several kinds of DAOs (more or less covering your Database Layer and DAO) and several entire categories of patterns found in object-relational mappers. So there's a good place to start.
It's hard to summarize any more of the content of POEAA in this answer without simply repeating the list of patterns. Fortunately the list can be found at Fowler's web site. Unfortunately the copyright symbol there suggests that I shouldn't just include it here.
This question has already been asked in this forum, but I really cannot understand those answers since I am still not an expert and has not dealt with this kind of scenario earlier. I am still a student.
I am going to build a loan management system and in my system there is a form in which user can attach scanned documents. These are not large documents and they are all image types.
I need to store those electronic documents in my database (in a specific table) which will be used by the officers for future references. How do I store those files in the DB? Is that the document path (directory) that we are recording in a table?
I am really grateful for anyone who could show me how to accomplish this task?
If you can give me an example that would be really helpful!
Before you store the document in a file and a filename in the database, ask: what will protect the file?
If the document is in the database, the DBMS ensures that as long as record of the document exists, so too does the document itself. If the document is updated, the DBMS ensures that either both versions are kept, or one replaces the other, and keeps the associated attributes in sync.
The moment you move the document outside the database, all those affordances are gone. Do you want to surrender that, in exchange for some dubious convenience or efficiency?
Keep your data in your database. Just because some data are "a document" stored in "a file" doesn't change much, if you care about them.
Data lineage is defined as a kind of data life cycle that includes the data's origins and where it moves over time. This term can also describe what happens to data as it goes through diverse processes. Data lineage can help with efforts to analyze how information is used and to track key bits of information that serve a particular purpose.
I want to know if there is a specific way to perform data lineage using Erwin. I have searched but could not find a place where it clearly says how to perform data lineage. Please help.
Other than the Table editor and then "Where Used" I don't believe there is... it's not going to build the traditional flow chart that other lineage tools create.
Erwin have the ability to natively see this data lineage using 2 different features.
first Design layer. Allow a conceptual/logical model to be drive to another logical or physical model.
Or
Go to your ERwin model properties and make sure that you are supporting the data movement properties capabilities of ERwin. This can be combined with the dimensional features of the ERwin modeling tool.
Once have done this. You will have the ability to go and properly document data movement rules attached to certain tables and in the column properties of the table, the is now the ability to add source to target mapping. The source to target mapping can to use to add content to the answer provided earlier. Looking at the "Where used" properties of a table.
The data lineage however in the are report that provide source to target mapping. Comment of the mapping can be added.
To see this graphically, you will need the new CA web portal. Once the model is publish to portal. Model connections are created and additional semantic, data and other lineage information are now visible graphically. This however is another product in the ERwin solution Suite.
Enabling the Data Movement option under General Model Properties
Will allow you to specify the source table/field and transformation rule for every attribute in your model.
This the only known Erwin option to establish source-to-target lineage for documentation purpose.
Ref :
https://support.ca.com/cadocs/0/CA%20ERwin%20%20Data%20Modeler%20r7%203%2011-ENU/Bookshelf_Files/HTML/ERwin%20Online%20Help/Set_General_Model_Properties.html
Hope this helps.