I am working with database tables that contains PII and sensitive information. Some of the data is PII and company sensitive information. The design document may not always be available to the developer especially when the data is being exposed by a view outside of the database (Oracle database link) to another program.
Is there good naming conventions for letting the developers know the column contains PII or sensitive information?
Column names? Weak.
Use a table of PII data that contains all of the PII attributes. Keep the PII tables separate from other tables with non-PII.
A one-to-one join between the non-PII and PII isn't all that costly. And it provides clear, obvious use of the PII table and PII columns.
I agree with S.Lott but I'll be more explicit. If the data are sensitive then it's even MORE important that column names be chosen to mean what they actually contain, that datatypes be actually appropriate for the contents; and the constraints actually match the problem domain.
Then, on top of that, move this stuff to another table, where they can be appropriately encrypted, stored in the correct location, etc.
I would have separate views (eg table PERSON with PERSON_BASIC having no PII columns and PERSON_PII with the PII columns). That way, if it is later decided that a column is sensitive (eg Date of Birth), then you can easily recreate the views to remove the column from the basic view rather than some massive data restructuring exercise which you would get with separate tables.
Also, the optimizer is getting better at correlation between columns on the same table (and you'd expect that to improve over time). Once you start joining non-PII tables to PII tables, you've just made it more complicated.
If you do go for separate tables, and think they'll need to be joined often, look into clusters so that the records for the same person are on the same block.
Consider using a role secured by a password or through a package to control access to the PII view/columns.
CONTEXT can do this too. They are forced to call the package to set the context before they see the columns.
The view could be
SELECT name, date_of_birth,
case when SYS_CONTEXT('SEC','xxx') ='ALLOW' THEN ssn END
from ...
Related
Recently I encountered an application, Where a Master Table is maintained which contain the data of more than 20 categories. For e.g. it has some categories named as Country,State and City.
So my question is, it is better to move out this category as a separate table and fetching out the data through joins or Everything should be inside a single table.
P.S. In future categories count might increase to 50+ or more than it.
P.S. application based on EF6 + Sql Server.
Edited Version
I just want to know that in above scenario what should be the best approach, one should go with single table with proper indexing or go by the DB normalization approach, putting each category into a separate Table and maintaning relationship through fk's.
Normally, categories are put into separate tables. This conforms more closely with normalized database structures and the definition of entities. In particular, it allows for proper foreign key relationships to be defined. That is a big win for data integrity.
Sometimes categories are put into a single table. This can, of course, be confusing; consider, for instance, "Florida, Massachusetts" or "Washington, Iowa" (these are real places).
Putting categories in one table has one major advantage: all the text is in a single location. That can be very handy for internationalization efforts. To be honest, that is the situation where I have seen this used.
We have a situation in a database design in our company. We are trying to figure out the best way to design the database to store transactional data. I need expert’s advice on the best relational design to achieve it. Problem: We have different kind of “Entities” in our system, for example; Customers, Services, Dealers etc. These Entities are doing transfer of funds between each other. We need to store the history of the transfers in database.
Solutions:
One table of transfers and another table to keep “Accounts” information. There are three tables “Customers”, “Services”, “Dealers”. There is another table “Accounts”. An account can be related to any of the “Entities” mentioned above; it means (and that’s the requirement) that logically there should be a one-to-one relationship to/from Entities and Accounts. However, we can only store the Account_ID in the Entities table, but we cannot store the foreign key of Entities in Accounts table. Here the problem happens in terms of database design. Because if there is a customer’s account, it is not restricted by the database design to not be stored in Services table etc. Now we can keep all transfers in one table only since Accounts are unified among all the entities.
Keep the balance information in the table primary Entities table and separate tables for all transfers. Here for all kind of transfers between the entities, we are keeping separate tables. For example, a transfer between a Customer and Service provider will be stored in a table called “Spending”. Another table will have transfer data for transfer between Service and Dealers called “Commission” etc. In this case, we are not storing all the transfers of the funds in a single table, but the foreign keys are properly defined since the tables “Spending” and “Commission” are only between two specific entities.
According to the best practices, which one of the above given solutions is correct, and why?
If you are simply looking for schemas that claim to deal with cases like yours, there is a website with hundreds of published schemas. Some of these pertain to storing transaction data concerning customers and suppliers. You can take one of these and adapt it.
http://www.databaseanswers.org/data_models/
If your question is about how to relate accounts to business contacts, read on.
Customers, Services, and Dealers are all sub classes of some super class that I'll call Contacts. There are two well known design patterns for modeling sub classes in database tables. And there is a technique called Shared primary Key that can be used with one of them to good advantage.
Take a look at the info and the questions grouped under these three tags:
single-table-inheritance class-table-inheritance shared-primary-key
If you use class table inheritance and shared primary key, you will end up with four tables pertaining to contacts: Contacts, Customers, Dealers, and Services. Every entry in Contacts will have a corresponding entry in one of the three subclass tables.
An FK in the accounts table, let's call it Accounts.ContactID will not only reference a row in Contacts, but also a row in whichever of Customers, Dealers, Services pertains to the case at hand.
This may work outwell for you. Alternatively, single table table inheritance works out well in some of the simpler cases. It depends on details about your data and your intended use of it.
You can make table Accounts with three fields with FK to Customers,Dealers and Services and it's will close problem. But also you can make three table for each type of entity with accounting data. You have the deal with multi-system case in system design. Each system solve the task. But for deсision you need make pros and con analyses about algorithm complexity, performance and other system requirements. For example one table will be more simple to code, but three table give more performance of sql database.
I'm creating an Account table in my project's database. Each account has A LOT of properties:
login
email
password
birthday
country
avatarUrl
city
etc.
Most of them are nullable. My question is, how should I design this in database?
Should it be one table with all those properties? Or maybe should I create two tables, like AccountSet, and AccountInfoSet, where I would store all those 'advanced' user's settings? And last, but not least: if this should be two tables, what kind of relation should be between those tables?
If this is a relational database, then I definitely would not store those properties as fields in the Account table. Some reasons why:
Once your application goes to production (or maybe it's already there), the schema maintenance will become a nightmare. You will absolutely add more properties and having to constantly touch that table in production will be painful.
You will most likely end up with orphaned fields. I've seen this many times where you'll introduce a property and then stop using it, but it's baked into your schema and you might be too scared to remove it.
Ideally you want to avoid having such sparse data in a table (lots of fields with lots of nulls).
My suggestion would be to do what you're already thinking about and that's to introduce a property table for Accounts. You called it AccountInfoSet.
The table should look like this:
AccountId int,
Property nvarchar(50),
Value nvarchar(50)
(Of course you'll set the data types and sizes as you see fit.)
Then you'll join to the AccountInfoSet table and maybe pivot on the "advanced" properties - turn the rows into columns with a query.
In .NET you can also write a stored procedure that returns two queries with one call and look at the tables in the DataSet object.
Or you could just make two separate calls. One for Account and one for the properties.
Lots of ways to get the information out, but make sure you don't just add fields to Account if you're using a relational database.
I want to write an enterprise software and now I'm in the DB design phase. The software will have some master data such as Suppliers, Customers, Inventories, Bankers...
I considering 2 options:
Put each of these on one separate table. The advantage: the table will have all necessary information for that kind of master file (Customer: name, address,.../Inventory: Type, Manufacturer, Condition...). Disadvantage: Not flexible. When I want to have a new type of master data, such as Insurer, I have to design another table.
Put all in one table and this table have foreign key to another table which have type of each kind of master data (table 1: id, data_type, code, name, address....; table 2: data_type, data_type_name). Advantage: flexible - if I want more master data such as Insurer, I just put in table 2: code: 002, name: Insurer, and then put detail each insurer into table 1). Disadvantage: table 1 must have sufficient field to store all kind of information including: customer name, address, account, inventory's manufacturer, inventory's quality...).
So which method do you usually do (or you think work better).
Thank you very much
I would advise creating separate tables for each entity type - it will be a lot easier to maintain in the future when you discover things you want to add for one entity type that don't apply to the others. If all of the entities (Suppliers, Customers, etc) are going to have the same fields and the only difference is their type then you could theoretically use one table. However, I would expect that there would be enough differences between the entity types that it would be worth creating separate tables for each. If there are several fields in common (e.g. address information) you could create a table for the common elements and have a foreign key in the individual tables to the table with the common data (e.g. AddressID).
logically, each "master" entity should be in its own table
if you don't, you'll find joins will become very painful, and your generic lookup table will accumulate all kinds of useless fields
I've done many web apps where the first thing you do is make a user table with usernames, passwords, names, e-mails and all of the other usual flotsam. My current project presents a situation where non-users records need to function similarly to users, but do not need to the ability to be a first order user.
Is it reasonable to create a second table, people_tb, that is the main relational table and data store, and only use the users_tb for authentication? Does separating user_tb from people_tb present any problems? If this is commonly done, what are some strategies and solutions as well as drawbacks?
This is certainly a good idea, as you are normalizing the database. I have done a similar design in an app that I am writing, where I have an employee table and a user table. Users may a from an external company or an employee, so I have separate tables because an employee is always a user, but a user may not be an employee.
The issues that you'll run into is that whenever you use the user table, you'll nearly always want the person table to get the name or other common attributes you would want to show up.
From a coding standpoint, if you're using straight SQL, it will take a little more effort to mentally parse the select statement. It may be a little more complicated if you're using an ORM library. I don't have enough experience with those.
In my application, I'm writing it in Ruby on Rails, so I'm constantly doing things like employee.user.name, where if I kept them together, it would be just employee.name or user.name.
From a performance standpoint, you are hitting two tables instead of one, but given proper indexes, it should be negligible. If you had an index that contained the primary key and the person name, for instance, the database would hit the user table, then the index for the person table (with a nearly direct hit), so the performance would be nearly the same as having one table.
You could also create a view in the database to keep both tables joined together to give you additional performance enhancements. I know in the later versions of Oracle you can even put an index on a view if needed to increase performance.
I routinely do that because for me the concept of "user" (username, password, create date, last login date) is different from "person" (name, address, phone, email). One of the drawbacks that you may find is that your queries will often require more joins to get the info you're looking for. If all you have is a login name, you'll need to join the "people" table to get the first and last name for example. If you base everything around the user id primary key, this is mitigated a bit, but still pops up.
If user_tb has auth info, I would very much keep it separate from people_tb. I would however keep a relationship between the two, and most of users' info would be stored in people_tb except all of the info needed for auth (which i guess will not be used for much else) Its a nice tradeoff between design and efficiency i think.
That is definitely what we do as we have millions of people records and only thousands of users. We also separate address, phones and emails into relational tables as many people have more than one of each of these things. Critial is to not rely on name as the identifier as name is not unique. Make sure the tables are joined through some type of surrogate key (an integer or a GUID is preferable) not name.
I always try to avoid as much data repetition as possible. If not all people need to login, you can have a generic people table with the information that applies to both people and users (eg. firstname, lastname, etc).
Then for people that login, you can have a users table that has a 1~1 relationship with people. This table can store the username and password.
I'd say go for the normalized design (two tables) and only denormalize (go down to one user/person table) if it will really make your life easier down the line. If however practically all people are also users it may be simpler to denormalize up front. Its up to you; I have used the normalized approach without problems.
Very reasonable.
As an example, take a look at the aspnet_* services tables here.
Their built in schema has a aspnet_Users and aspnet_Membership with the later table having more extended information about a given user (hashed passwords, etc) but the aspnet_User.UserID is used in the other portions of the schema for referential integrity etc.
Bottom line, it's very common, and good design, to have attributes in a separate table if they are different entities, as in your case.