I've heard that exposing database IDs (in URLs, for example) is a security risk, but I'm having trouble understanding why.
Any opinions or links on why it's a risk, or why it isn't?
EDIT: of course the access is scoped, e.g. if you can't see resource foo?id=123 you'll get an error page. Otherwise the URL itself should be secret.
EDIT: if the URL is secret, it will probably contain a generated token that has a limited lifetime, e.g. valid for 1 hour and can only be used once.
EDIT (months later): my current preferred practice for this is to use UUIDS for IDs and expose them. If I'm using sequential numbers (usually for performance on some DBs) as IDs I like generating a UUID token for each entry as an alternate key, and expose that.
There are risks associated with exposing database identifiers. On the other hand, it would be extremely burdensome to design a web application without exposing them at all. Thus, it's important to understand the risks and take care to address them.
The first danger is what OWASP called "insecure direct object references." If someone discovers the id of an entity, and your application lacks sufficient authorization controls to prevent it, they can do things that you didn't intend.
Here are some good rules to follow:
Use role-based security to control access to an operation. How this is done depends on the platform and framework you've chosen, but many support a declarative security model that will automatically redirect browsers to an authentication step when an action requires some authority.
Use programmatic security to control access to an object. This is harder to do at a framework level. More often, it is something you have to write into your code and is therefore more error prone. This check goes beyond role-based checking by ensuring not only that the user has authority for the operation, but also has necessary rights on the specific object being modified. In a role-based system, it's easy to check that only managers can give raises, but beyond that, you need to make sure that the employee belongs to the particular manager's department.
There are schemes to hide the real identifier from an end user (e.g., map between the real identifier and a temporary, user-specific identifier on the server), but I would argue that this is a form of security by obscurity. I want to focus on keeping real cryptographic secrets, not trying to conceal application data. In a web context, it also runs counter to widely used REST design, where identifiers commonly show up in URLs to address a resource, which is subject to access control.
Another challenge is prediction or discovery of the identifiers. The easiest way for an attacker to discover an unauthorized object is to guess it from a numbering sequence. The following guidelines can help mitigate that:
Expose only unpredictable identifiers. For the sake of performance, you might use sequence numbers in foreign key relationships inside the database, but any entity you want to reference from the web application should also have an unpredictable surrogate identifier. This is the only one that should ever be exposed to the client. Using random UUIDs for these is a practical solution for assigning these surrogate keys, even though they aren't cryptographically secure.
One place where cryptographically unpredictable identifiers is a necessity, however, is in session IDs or other authentication tokens, where the ID itself authenticates a request. These should be generated by a cryptographic RNG.
While not a data security risk this is absolutely a business intelligence security risk as it exposes both data size and velocity. I've seen businesses get harmed by this and have written about this anti-pattern in depth. Unless you're just building an experiment and not a business I'd highly suggest keeping your private ids out of public eye. https://medium.com/lightrail/prevent-business-intelligence-leaks-by-using-uuids-instead-of-database-ids-on-urls-and-in-apis-17f15669fd2e
It depends on what the IDs stand for.
Consider a site that for competitive reason don't want to make public how many members they have but by using sequential IDs reveals it anyway in the URL: http://some.domain.name/user?id=3933
On the other hand, if they used the login name of the user instead: http://some.domain.name/user?id=some they haven't disclosed anything the user didn't already know.
The general thought goes along these lines: "Disclose as little information about the inner workings of your app to anyone."
Exposing the database ID counts as disclosing some information.
Reasons for this is that hackers can use any information about your apps inner workings to attack you, or a user can change the URL to get into a database he/she isn't suppose to see?
We use GUIDs for database ids. Leaking them is a lot less dangerous.
If you are using integer IDs in your db, you may make it easy for users to see data they shouldn't by changing qs variables.
E.g. a user could easily change the id parameter in this qs and see/modify data they shouldn't http://someurl?id=1
When you send database id's to your client you are forced to check security in both cases. If you keep the id's in your web session you can choose if you want/need to do it, meaning potentially less processing.
You are constantly trying to delegate things to your access control ;) This may be the case in your application but I have never seen such a consistent back-end system in my entire career. Most of them have security models that were designed for non-web usage and some have had additional roles added posthumously, and some of these have been bolted on outside of the core security model (because the role was added in a different operational context, say before the web).
So we use synthetic session local id's because it hides as much as we can get away with.
There is also the issue of non-integer key fields, which may be the case for enumerated values and similar. You can try to sanitize that data, but chances are you'll end up like little bobby drop tables.
My suggestion is to implement two stages of security.
"Security through obscurity": You can have integer Id as primary key and Gid as GUID as surrogate key in tables. Whereas integer Id column is used for relations and other database back-end and internal purposes (and even for select list keys in web apps to avoid unnecessary mapping between Gid and Id while loading and saving) and Gid is used for REST Urls i.e for GET,POST, PUT, DELETE etc. So that one cannot guess the other record id. This gives first level of protection against guess-based attacks. (i.e. number series guessing)
Access based control at Server side : This is most important, and you have various way to validate the request based on roles and rights defined in application. Its up to you to decide.
From the perspective of code design, a database ID should be considered a private implementation detail of the persistence technology to keep track of a row. If possible, you should be designing your application with absolutely no reference to this ID in any way. Instead, you should be thinking about how entities are identified in general. Is a person identified with their social security number? Is a person identified with their email? If so, your account model should only ever have a reference to those attributes. If there is no real way to identify a user with such a field, then you should be generating a UUID before hitting the DB.
Doing so has a lot of advantages as it would allow you to divorce your domain models from persistence technologies. That would mean that you can substitute database technologies without worrying about primary key compatibility. Leaking your primary key to your data model is not necessarily a security issue if you write the appropriate authorization code but its indicative of less than optimal code design.
Related
I have a web app that stores objects in a database and then sends emails based on changes to those objects. For debugging and tracking, I am thinking of including the Document Id in the email metadata. Is there a security risk here? I could encrypt it (AES-256).
In general, I realize that security through obscurity isn't good practice, but I am wondering if I should still be careful with Document Ids.
For clarity, I am using CouchDB, but I think this can apply to databases in general.
By default, CouchDB uses UUIDs with a UTC time prefix. The worst you can leak there is the time the document was created, and you will be able to correlate about 1k worth of IDs likely having been produced on the same machine.
You can change this in the CouchDB configuration to use purely 128bit random UUIDs by setting the algorithm setting within the uuids section to random. For more information see the CouchDB Docs. Nothing should be possible to be gained from them.
Edit: If you choose your own document IDs, of course, you leak whatever you put in there :)
Compare Convenience and Security:
Convenience:
how useful is it for you having the document id in the mail?
can you quickly get useful information / the document having the ID ?
does encrypting/hashing it mean it's harder to get the actual database document? (answer here is yes unless you have a nice lookup form/something which takes the hash directly, avoid manual steps )
Security:
having a document ID what could I possibly do that's bad?
let's say you have a web application to look at documents..you have the same ID in a URL, it can't be considered 'secret'
if I have the ID can I access the 'document' or some other information I shouldn't be able to access. Hint: you should always properly check rights, if that's done then you have no problem.
as long as an ID isn't considered 'secret', meaning there aren't any security checks purely on ID, you should have no problems.
do you care if someone finds out the time a document was created? ( from Jan Lehnardt's answer )
I have an application with a database that has a lot of foreign keys.
A lot of the data being inserted comes from users.
I was just wondering is: Is it better to run validation on user input before inserts or insert and just write error catching code?
Are there performance/stylistic/security benefits to either?
The knee jerk response to me seems to be do both, but doing validation before seems a safer option if only one of the two was done.
This is a brilliant question - and in my view, one of the benefits of the use of an object relational mapper.
In general, the relational tools your database provides are all about protecting data validity - "customer" must have an "account manager" relationship, "user_name" may not be null, "user_id" must be unique etc.
Those tools are necessary, but not sufficient, to validate the data users input into the database.
Your front-end/middle tier code has its own rules - which are usually not expressed in relational terms; in most modern development languages, they're about objects and the relationships between the objects, or their attributes - for instance, a phone number must contain numbers, a name must start with a capital letter.
I'm assuming your users don't interact with the database through SQL - that you've built some kind of user interface which allows them to look up associations (and thus populate the foreign key).
In this case, my preferred architecture is:
Validate as early as you can - in JavaScript for web apps, or in the GUI code for desktop apps. This reduces the number of round trips, and thus creates a more responsive user experience.
Have each layer implement validation for its key domain logic - your classes should validate their expectations, your database should validate the foreign keys and nullability.
Don't depend on "higher" levels to validate - you can't predict how your code is going to be used in the future; a class you've written for one application may get re-used by another.
Work out a way to keep the validation rules in sync across the layers - either through technology or process.
I don't think it should be a question of which. I always do both, as you suggested. As far as pros and cons, I will say that I think it is easier to find user input problems and handle them in a user friendly manner earlier in the life cycle, as opposed to waiting until the database throws back an error. However, if your validation does not catch everything or if there is a database error unrelated to user input, it is still important to catch the error at some point.
No matter what, you'll need to trap errors. Who knows what will happen? SQL timeout? No amount of data validation will help on those kind of problems.
Like most problems, they are cheapest to catch early; don't let users enter invalid data/validate data before sending it to a server. Not business logic per se, but that dates don't contain alpha characters, etc.
In a mid-tier, I would still validate inputs. Another developer may call your business logic without vetting the inputs. I prefer my business logic in this area because it is easier to write and debug than creating logic in stored procedures.
On the database, I prefer constraints when rules are absolutes such as you can't have an internal user without a user id; the reason is to prevent other developers from writing scripts that would do things that should not be allowed.
I'd like to keep the real names, emails, and any other personal identifiable information out of my primary application database, and in another database/encrypted file. And I'm curious on if there's a best practices solution for this or if I'm totally over looking something.
Some thoughts I had were the following:
User logs in with a username and password that are both hashed in the primary database
This server then makes some sort of secure call to the member database with the user's id
And in return the member database returns the name, email, address etc.
I'm wondering if this is the right approach, and if so where the keys are stored and authenticated etc..
It's an interesting question, I think, but it needs some more context. That is, you need to be clear about who you are wishing to anonymise them against. That is, who is the threat, here? Do you want the information hidden from only the public? Clearly, that's trivial, just don't show it (don't link it). Do you want the information hidden from someone who gains access to your database? How hidden? How will they get access to your db? Can they, if they gain access to the not-anonymous one, get access to the other? OpenID may also be of interest to you (externalise the authentication, you just do role management).
I'd suggest sit down and plan that out a bit.
You don't want to introduce complexity (multiple db's, linking, etc) if they're all just on the same server anyway, and hence accessible to any successful attacker.
I'd think the number 1 solution to keeping things anonymous is to never actually collect any information yourself. It's more of a model thing (i.e. the details of your app matter).
I am building an application with health information inside. This application will be consumer-facing with is new for me. I would like a method to put privacy concerns completely at ease. As I review methods for securing sensitive data in publicly accessible databases I have frequently come across the notion of database translucency. There is the original book on the subject and an excellent tutorial on the subject from Oriellynet.
My concern is that I have seen very little information regarding this idea on what I would consider very-modern programming sites (like this one). There does not seem to be an article about the idea on wikipedia. No questions on the subject here, and no very recent tutorials or articles on the subject. To be uber-brief, the idea is that certain data is clear to some users of the system, while other users a cryptographically prevented from accessing that data, even if they have administrator access.
I have done substantial work on a prototype database that provides translucent data access. I have run across a considerable problem: To be truly translucent, there can be no mechanism for password recovery. If an administrator can reset a users password, then they can briefly gain access to a users data. To be truly translucent, the user must never loose the password.
Those of us who use strong encryption to protect private data in our daily lives (technorati to be sure) are used to this problem when using these kinds of strong encryption systems. If the word "blowfish" is part of your daily lexicon that is one thing, but a website that is consumer focused? I am concerned that users will not be willing to wrap their mind around the "truly encrypted just for you" notion implicit with true database translucency. I am afraid of the support call that begins with "I lost my password" and ends with me saying "There is nothing that I can do for you".
My question: Should I implement this method in my application? Are there other open source applications that have gone down this route that I can compare database designs with (esp using php/MySQL)? I anyone else pursuing these kind of truly secure, but really inconvenient feature sets? Is there another database security model that is more popular and modern that I have missed? Was database translucency a fad or a legitimate database design method that I should embrace? While I always appreciate discussion I would prefer objective answers that I can leverage in my design.
So, I've been looking at something similar to this recently, and hit upon the same issue. The solution I'm considering implementing is as follows:
Upon registration, create a unique, secure (long) key for the user and use this to encrypt their data.
Encrypt this key with the user's password using e.g. AES and store it in the database.
At this point, you're still in the situation where if the user forgets their password, they've had it.
Create a public/private key pair representing your organisation, and store the public key on the server.
Split the private portion of the key into several components and give each to people (e.g. directors of your company) who have a significant stake (preferably financial) in the continued success of your company. Do this such that any two, or any three people can get together and restore the full private key when required. Encrypt each person's key with their own password.
When a user registers, as well as encrypting their key with their password, encrypt it with the organisational public key and store it somewhere.
Create a password reset form which records a request to reset the password of a user, along with some proof that the user is who they say they are (e.g. challenge/response).
Record these reset requests (optionally encrypted using the public key again) in the database.
Once per hour/day/week/month, get the requisite key-holders together, and use their combined keys to process the accrued reset requests, decrypting the keys of users who successfully proved they are who they say they are.
There are lots of challenges and considerations in this. I've got a few thoughts on most of these, but would be interested in others opinions too:
How to split the key safely between multiple people so that no one person can decrypt the stored keys.
How to minimise the number of keys that would be exposed if the 'master keys' genuinely fell into the wrong hands.
How to make sure that if (heaven forbid) your key-holders lost their keys, then (a) there's no risk of exposure of the data, and (b) there's no risk that suddenly the ability to reset passwords is lost forever.
How to successfully validate that someone really is who they say they are without making this a glaring hole in your whole security approach.
Anything you implement in this area WILL reduce the security of the translucent database approach, without a doubt, but this may be a worthwhile compromise depending on the nature of your data.
Should I implement this method in my application?
Well like other things in life, there is a trade off :) It's probably more secure but harder to built.
Are there other open source applications that have gone down this route that I can compare database designs with (esp using php/MySQL)?
Don't know, I guess the tools are there to do it yourself :)
Is anyone else pursuing these kind of truly secure, but really inconvenient feature sets?
Yes, but it seems like it's still in an immature state, like your problem you describe concerning lost passwords.
Is there another database security model that is more popular and modern that I have missed?
Basically there are two kinds of database connections. One option gives users a real database account, the other is to use single sign-on to the database. Prior to the web coming along, there were proponents of both models in the client/server world, but amongst web developers the single sign-on method is leading.
Was database translucency a fad or a legitimate database design method that I should embrace?
Don't think so, the UNIX password database, for instance, is a great example of a basic translucent database ;)
here something to read link text
Re: translucent databases. You could, I suppose, use fingerprints. What about burn victims, or people who end up losing their fingerprints? Oops. Is it worth that small percentage of users?
Familiarize yourself with HIPAA, especially when it comes to technology.
Remember that no system is truly secure, except Skynet*, and look what happened with that! Humans are in charge. When you work in a medical company, you sign an NDA indicating that you won't release any of the information you learn as part of your duties because it is confidential.
There will be someone to reset people's passwords. That's the way it is, because not everyone is technologically competent, and that's the way it stays for now.
You only have to implement security as well as HIPAA says.
in truth, there is another truly secure system: it is unplugged from both the network and the electricity, and it is turned off.
Slightly different solution, you might want to check out cryptdb:
http://css.csail.mit.edu/cryptdb/
My previous job involved maintenance and programming for a very large database with massive amounts of data. Users viewed this data primarily through an intranet web interface. Instead of having a table of user accounts, each user account was a real first-class account in the RDBMS, which permitted them to connect with their own query tools, etc., as well as permitting us to control access through the RDBMS itself instead of using our own application logic.
Is this a good setup, assuming you're not on the public intranet and dealing with potentially millions of (potentially malicious) users or something? Or is it always better to define your own means of handling user accounts, your own permissions, your own application security logic, and only hand out RDBMS accounts to power users with special needs?
I don't agree that using the database for user access control is as dangerous others are making it out to be. I come from the Oracle Forms Development realm, where this type of user access control is the norm. Just like any design decision, it has it's advantages and disadvantages.
One of the advantages is that I could control select/insert/update/delete privileges for EACH table from a single setting in the database. On one system we had 4 different applications (managed by different teams and in different languages) hitting the same database tables. We were able to declare that only users with the Manager role were able to insert/update/delete data in a specific table. If we didn't manage it through the database, then each application team would have to correctly implement (duplicate) that logic throughout their application. If one application got it wrong, then the other apps would suffer. Plus you would have duplicate code to manage if you ever wanted to change the permissions on a single resource.
Another advantage is that we did not need to worry about storing user passwords in a database table (and all the restrictions that come with it).
I don't agree that "Database user accounts are inherently more dangerous than anything in an account defined by your application". The privileges required to change database-specific privileges are normally MUCH tougher than the privileges required to update/delete a single row in a "PERSONS" table.
And "scaling" was not a problem because we assigned privileges to Oracle roles and then assigned roles to users. With a single Oracle statement we could change the privilege for millions of users (not that we had that many users).
Application authorization is not a trivial problem. Many custom solutions have holes that hackers can easily exploit. The big names like Oracle have put a lot of thought and code into providing a robust application authorization system. I agree that using Oracle security doesn't work for every application. But I wouldn't be so quick to dismiss it in favor of a custom solution.
Edit: I should clarify that despite anything in the OP, what you're doing is logically defining an application even if no code exists. Otherwise it's just a public database with all the dangers that entails by itself.
Maybe I'll get flamed to death for this post, but I think this is an extraordinarily dangerous anti-pattern in security and design terms.
A user object should be defined by the system it's running in. If you're actually defining these in another application (the database) you have a loss of control.
It makes no sense from a design point of view because if you wanted to extend those accounts with any kind of data at all (email address, employee number, MyTheme...) you're not going to be able to extend the DB user and you're going to need to build that users table anyway.
Database user accounts are inherently more dangerous than anything in an account defined by your application because they could be promoted, deleted, accessed or otherwise manipulated by not only the database and any passing DBA, but anything else connected to the database. You've exposed a critical system element as public.
Scaling is out of the question. Imagine an abstraction where you're going to have tens or hundreds of thousands of users. That's just not going to manageable as DB accounts, but as records in a table it's just data. The age old argument of "well there's onyl ever going to be X users" doesn't hold any water with me because I've seen very limited internal apps become publicly exposed when the business feels it's could add value to the customer or the company just got bought by a giant partner who now needs access. You must plan for reasonable extensibility.
You're not going to be able to share conn pooling, you're not going to be any more secure than if you just created a handful of e.g. role accounts, and you're not necessarily going to be able to affect mass changes when you need to, or backup effectively.
All in there seems to be numerous serious problems to me, and I imagine other more experienced SOers could list more.
I think generally. In your traditional database application they shouldnt be. For all the reason already given. In a traditional database application there is a business layer that handles all the security and this is because there is such a strong line between people who interact with the application, and people who interact with the database.
In this situation is is generally better to manage these users and roles yourself. You can decide what information you need to store about them, and what you log and audit. And most importantly you define access based on pure business rules rather than database rules. Its got nothing to do with which tables they access and everything to do with whether they can insert business action here. However these are not technical issues. These are design issues. If that is what you are required to control then it makes sense to manage your users yourself.
You have described a system where you allow users to query the database directly. In this case why not use DB accounts. They will do the job far better than you will if you attempt to analyse the querys that users write and vet them against some rules that you have designed. That to me sounds like a nightmare system to write and maintain.
Don't lock things down because you can. Explain to those in charge what the security implications are but dont attempt to prevent people from doing things because you can. Especially not when they are used to accessing the data directly.
Our job as developers is to enable people to do what they need to do. And in the situation you have described. Specifically connect to the database and query it with their own tools. Then I think that anything other than database accounts is either going to be insecure, or unneccasarily restrictive.
"each user account was a real first-class account in the RDBMS, which permitted them to connect with their own query tools, etc.,"
not a good idea if the RDBMS contains:
any information covered by HIPAA or Sarbanes-Oxley or The Official Secrets Act (UK)
credit card information or other customer credit info (POs, lines of credit etc)
personal information (ssn, dob, etc)
competitive, proprietary, or IP information
because when users can use their own non-managed query tools the company has no way of knowing or auditing what information was queried or where the query results were delivered.
oh and what #annakata said.
I would avoid giving any user database access. Later, when this starts causing problems, taking away their access becomes very dificult.
At the very least, give them access to a read-only replica of the database so they can't kill your whole company with a bad query.
A lot of database query tools are very advanced these days, and it can feel a real shame to reimplement the world just to add restrictions. And as long as the database user permissions are properly locked down it might be okay. However in many cases you can't do this, you should be exposing a high-level API to the database to insert objects over many tables properly, without the user needing specific training that they should "just add an address into that table there, why isn't it working?".
If they only want to use the data to generate reports in Excel, etc, then maybe you could use a reporting front end like BIRT instead.
So basically: if the users are knowledgeable about databases, and resources to implement a proper front-end are low, keep on doing this. However is the resource does come up, it is probably time to get people's requirements in for creating a simpler, task-oriented front-end for them.
This is, in a way, similar to: is sql server/AD good for anything
I don't think it's a bad idea to throw your security model, at least a basic one, in the database itself. You can add restrictions in the application layer for cosmetics, but whichever account the user is accessing the database with, be it based on the application or the user, it's best if that account is restricted to only the operations the user is allowed.
I don't speak for all apps, but there are a large number I have seen where capturing the password is as simple as opening the code in notepad, using an included dll to decrypt the configuration file, or finding a backup file (e.g. web.config.bak in asp.net) that can be accessed from the browser.
*not a good idea if the RDBMS contains:
* any information covered by HIPAA or Sarbanes-Oxley or The Official Secrets Act (UK)
* credit card information or other customer credit info (POs, lines of credit etc)
* personal information (ssn, dob, etc)
* competitive, proprietary, or IP information*
Not true, one can perfectly manage which data a database user can see and which data it can modify. A database (at least Oracle) can also audit all activities, including selects. To have thousands of database users is also perfectly normal.
It is more difficult to build good secure applications because you have to program this security, a database offers this security and you can configure it in a declarative way, no code required.
I know, I am replying to a very old post, but recently came across same situation in my current project. I was also thinking on similar lines, whether "Application users be Database users?".
This is what I analysed:
Definitely it doesn't make sense to create that big number of application users on database(if your application is going to be used by many users).
Let's say you created X(huge number) of users on database. You are opening a clear gateway to your database.
Let's take a scenario for the solution:
There are two types of application users (Managers and Assistant). Both needs access to database for some transactions.
It's obvious you would create two roles, one for each type(Manager and Assistant) in database. But how about database user to connect from application. If you create one account per user then you would end up linearly creating the accounts on the database.
What I suggest:
Create one database account per Role. (Let's say Manager_Role_Account)
Let your application have business logic to map an application user with corresponding role.(User Tom with Manager role to Manager_Role_Account)
Use the database user(Manager_Role_Account) corresponding to identified role in #2 to connect to database and execute your query.
Hope this makes sense!
Updated: As I said, I came across similar situation in my project (with respect to Postgresql database at back end and a Java Web app at front end), I found something very useful called as Proxy Authentication.
This means that you can login to the database as one user but limit or extend your privileges based on the Proxy user.
I found very good links explaining the same.
For Postgresql below Choice of authentication approach for
financial app on PostgreSQL
For Oracle Proxy Authentication
Hope this helps!
It depends (like most things).
Having multiple database users negates connection pooling, since most libraries handle pooling based on connection strings and user accounts.
On the other hand, it's probably a more secure solution than anything you or I will do from scratch. It leaves security up to the OS and Database server, which I trust much more than myself. However, this is only the case if you go to the effort to configure the database permissions well. If you're using a bunch of OS/db users with the same permissions,it won't help much. You'll still get an audit trail, but that's about it.
All that said, I don't know that I'd feel comfortable letting normal users connect directly to the database with their own tools.
I think it's worth highlighting what other answers have touched upon:
A database can only define restrictions based on the data. Ie restrict select/insert/update/delete on particular tables or columns. I'm sure some databases can do somewhat cleverer things, but they'll never be able to implement business-rule based restrictions like an application can. What if a certain user is allowed to update a column only to certain values (say <1000) or only increase prices, or change either of two columns but not both?
I'd say unless you are absolutely sure you'll never need anything but table/column granularity, this is reason enough by itself.
This is not a good idea for any application where you store data for multiple users in the same table and you don't want one user to be able to read or modify another user's data. How would you restrict access in this case?