Most of you would have encountered the problem of creating db users for developers across multiple database (using common user is not allowed). We have around 90 DB's on AWS and 200-250 dev's. Everyday someone needs access to a database and this is manual and repetitive task.
I am looking for a solution to automate end-to-end lifecycle of user management, scripting or creating a terraform module are solutions which I already have in my mind, but how does other organization manage DB users at scale ?
I did look at AWS IAM authentication but I am not sure how can we grant fine grain access using IAM roles.
Cheers,
Fun Learn
The way I've done this is (high level);
Create your RDS Terraform Config / Module(s)
Create a sql file with the user & grant creations needed
Create a wrapper script that deploys terraform then connects to it to deploy your SQL file with user creation
Your wrapper script will need to use Terraform Outputs to get your newly created RDS Endpoint to connect to | Say you created an output called rds_endpoint in your terraform plan / config... This is how you grab it in bash terraform output rds_endpoint
Assuming your new RDS DB is not publicly accessible, your wrapper script will need to tunnel in through a bastion or some other instance that is publicly accessible with access to the DB. Example: ssh -oStrictHostKeyChecking=no -p 22 -i ~/.ssh/bastion-host-key.pem -C -N ec2-user#$bastion_ip -L 3306:$rds_endpoint:3306 &
Your wrapper script will need to use the RDS user & password you created with terraform as well to run the SQL File
In fact IAM authentication could be the key to do that.
What you can do is in fact create all you databases with terraform.
do not forget to enable iam authentication via your terraform module.
Once all you databases are created via teraform, you have to create local role(s) in all of theses databases (either via terraform using SQL script or still via terraform using modules that allow you to create user/roles, for postgresql you can use this module ) and you have to grant them the pre-created, existing, database role for iam (for example with postgresql its named "rds_iam")
The thing that is interresting with iam authentication is that all of your developper can connect using their account to aws and request a token that will be used as a password (the username will be the role you created before) and by doing this you create only one role, but each authentication is made by each developpers account.
If your company really needs you to create roles for each devs (even if the roles are exactly the same, It makes no sense since by definition, we ASSUME a role, so anyone can assume ONE role, this is not awful) you can then create a local database users (instead of a role) for all of your developpers in all of your database by using an SQL script that your terraform will execute.
Of course do not forget to grant the rds_iam existing role to either the unique role that will be used by all the developpers (in case you choose this solution) or to all the db users you created before.
You will have to manage IAM policy for all of theses users to be accurate regarding to the security (or use * in the policy to let all your developpers connect to all you db users lol)
and then your developpers will be able to use aws rds command to generate an auth token and connect to their local db user that will have to correct rights.
There is a hole bunch of informations and precisions here:
https://aws.amazon.com/premiumsupport/knowledge-center/users-connect-rds-iam
have a nice journey on aws
Related
I have a runbook that is currently executed via a webhook call in order to execute some powershell code on a site. This is been working great.
I have been instructed to convert this code to an Azure function. No big deal, right? Well....
I was able to get it set up without too much trouble - I created it in VS Studio and then deployed to Azure.
The problem is that in the code I need to log in and my username and password are hard-coded in the script. When I had run this in the runbook I called
a Credential stored in the Automation account but that does not seem to be an option here.
I have seen other people ask the same question and the answer is often "application settings", which of course is not protected.
This is to run unattended so I (obviously) can't ask for credentials. I am running PowerShell 7.
Any help would be greatly appreciated.
You can use Key vault references with Azure Function App to access secrets. You can protect your secrets stored in Key vault using RBAC. Azure function app also provides integration with AAD where you can use Identity instead of secrets. Additionally you can go through this document regarding more information on securing Azure Functions.
I have a specific problem to which I couldn't find any answer online.
The situation is the following:
We created a Canvas app that connects to the Azure SQL database. We set the connection type to be AAD integrated.
Users that are members of the AD can see the data in the app, but guest users, even though we gave them all the rights and PowerApps plan, cannot see the data. they recieve the same authorization window as members, but when they click on allow, the app starts but no data is being pulled from the SQL database.
When we try to connect directly to the Azure SQL database with the guest user email and credentials (via SQL server management studio), everything works as expected and the guest can see all the tables.
When we use implicitly shared connection (with SQL server authentification), guests can see the data, but we need to use AAD integrated due to its security.
So far we tried:
Changing PA environment from sandbox to production
Adding special permissions in SQL database like database owner etc
Trying out different AAD guest user settings, eq- setting that guest users have the same privileges as members (picture below).
Nothing seems to work. I would be more than happy if you could tell me how to make this work or even push me in the right direction.
I've reproduced your problem in my side. Here's my experience.
After assigning powerapp license(I use O365 E5 here) to guest account and share the app to this account, I can visit the app but can't see the data in the table. I assume that it resulted from the connector hasn't been shared, but it's true that this connector can't be shared because of no 'share button'.
Then I tried to add access policy to my guest account with these two sql:
CREATE USER [tiny-wa_outlook.com#EXT##xx.onmicrosoft.com] FROM EXTERNAL PROVIDER;
GRANT SELECT ON dbo.tinyTest TO [tiny-wa_outlook.com#EXT##xx.onmicrosoft.com];
Pls note here, I used the User Principal Name here(can find the principal name in azure ad->users), I also tested to use 'tiny-wa#outlook.com' in the sql but after executing successfully, it didn't work.
So I think you can try to use the sql above to add your guests accounts and make them can access the powerapp.
Here's some related docs:
create contained users mapped to azure ad identities
Share app resources
add table permission to user
==========================UPDATE==========================
Share my experience of this troubleshot.
First I need to create a power app but I found that after creating the connector with sql server azure ad authentication, it can't connect to the sql server, the error message is like 'Credentials are missing or not valid. inner exception: The credentials provided for the SQL source are invalid', solution is add my admin account as sql server instance Active Directory Admin.
Then I choose a table and successfully create a sample app. With the creating account, I can visit the app but other accounts can't. Here needs to share the app and it's related connectors to other users. But other accounts still can't reach the app because of no license. Because sql server connector is premium connector, so I assign O365 E3 license here. I met an error when assign license, the user's 'Usage location' can't be null or it can't assign license in M365 admin center.
Then I met similar error with Op, the difference is that both member account and guest account can't see the data in app. I try to find the way to share the connector to these uses but failed, I haven't made sure if those connectors without share button can be shared to others. So I have no options to study if this kind of connectors are authenticated in other ways so they don't need to be shared at all.
Next actions is using the account which used to create the sql server and database to sign in database and execute the sqls above.
Then the member account can see data in the power app while the guest account can't see. The reason is I used xx#outlook.com as the parameter in the sql, when I used xx_outlook.com#EXT##xx.onmicrosoft.com, it worked finally.
Hope this can also help you.
===================================================
For creating my demo app: First, I'm sure my environment isn't a sandbox(the environment in the screenshot below). And I think it's easy to create a demo app, and my app is simple, just choose to create an app from data and then select sql server as the connector, next I choose auzre ad auth and click the connector, enter server name and database name then choose a table, after that my app has created. That table has one row of data so when I signed in the app with the creating account, I see it in the screen while other accounts(member or guests) can't.
My sql server instance and database are created long time ago, but I'm sure I followed this tutorial to create them.
This appears to still be a limitation to access to Azure SQL via PA connector for guest users:
https://powerusers.microsoft.com/t5/Power-Apps-Ideas/Azure-SQL-to-PowerApps-Connector-AAD-doesn-t-work-for-guest/idi-p/1637817
If the "guest" does NOT have a PowerApps Per-App/Per-User plan, they cannot use your PowerApps with SQL data source (Note: SQL is a premium connector).
Determine exactly which type of license the guest has. Then, either your organization or the guest (or guest's org) must purchase one of these licenses. $5/$10/$20 per month depending on your use case.
REF
I have been searching for articles and SQL script examples that would demonstrate how to securely and conveniently solve one of the most common scenarios - connecting from a .Net Core Entity Framework based web application to an SQL database.
But somehow I could not find any coherent step-by-step guide from a reputable source.
Let's assume the following:
I cannot use integrated Windows auth in the connection string and must use username and password based auth (because hosting on a Linux server and the DB is on a different Windows server)
the web service will need your usual minimum set of permissions - connect to the database, read data, write data, delete data, execute stored procedures
While reading many tutorials, I find there are multiple ways to manage the connection permissions. To avoid this question being too broad, I'll list my current choices as I understand them (please correct me if I'm missing something).
Users and logins:
create a login and a user for the database
create a database-only user without a login (not sure if this is applicable to a web app and connection string, but still it's a feature that I've seen being used)
Assigning permissions:
assign the user to some fixed SQL role (db_datareader, db_datawriter AND also will have to grant EXECUTE permission)
grant all fixed permissions
create a custom role (let's say, db_web_apps) with required permissions
Which choices are better (more secure and easier to manage in general) and recommended by SQL DBAs?
I think every database administrator should have a template script handy for quickly adding a new user with minimum required permissions every time when developers ask for a new connection for their shiny new web app.
If you know a good, reliable tutorial or GitHub / Gist example that explains what and why is being done that way or a script that you yourself have used for years without any issues in production environments, I'll really appreciate if you could share it.
Create a role in the database and assign the required privileges to the role. Don't use the fixed database roles. Instead grant permissions directly to objects, schemas, or the entire database if necessary. Like this:
create role trusted_app_role
grant select, insert, update, delete, execute
on schema::dbo to trusted_app_role
That will grant the role full DML permissions on all the objects in the default dbo schema. So if you have any tables or procedures you don't want the app to have access to, just create them in a different schema, say, admin. This way you never have to fiddle with permissions as you add objects. The fixed database roles predate schema-based permissions, and aren't really needed any more.
For your application's identity, add Active Directory or Azure Active Directory (Azure SQL) identities to this role, or, if you can't, add SQL Users to the role.
If you are on Azure SQL, you should normally use a database user without a login. On SQL Server you can only add "contained database users" if you enable Partial Database Containment. Which you can do, but is incompatible with Change Tracking and Change Data Capture, so it's a tradeoff.
So normally for SQL Server you still create a login and map the user to the login. EG:
create login web_service_user with password = '5X+jeuAB6kmhw85R/AxAg'
create user web_service_user for login web_service_user
And then add that user to your role
alter role trusted_app_role add member web_service_user
Running SQL Server 2016. Currently have a solution that is hosted in one domain and of course our access point is in another. we need to pull data in an automated fashion.
We have added a windows credential with the credential manager which collects endpoint information and a set of credentials.
e.g.
Internet or Networkaddress: mydatabase.remotedomain.com
Username : remotedomain\username
Password : password
This solution works with many tools, Excel, SSMS direct query, Visual Studio. The user enters the endpoint (server url or IP/port) and uses windows integrated security. the connection is made and credential store does the trick and user is authenticated.
SSMS example
Server name: mydatabase.remotedomain.com
Authentication: Windows Authentication
My challenge is SSIS and SQL Agent. The SSIS package runs in VS2015. deploy the package to Integration Services Catalog - highlight package and execute and it runs.
Create a SQL Server Agent Job and execute the job and I receive this gift.....
Login Failed: The login is from an untrusted domain and cannot be used with Windows authentication
I have created a SQL credential, created a Proxy (SSIS Package Execution), created a job that uses the Run As with the Credential but this ends with the same result. The credential has to be in my local domain or the job wont run....and of course localdomain\username does not authenticate against the remote data connection. So Proxy does not help the situation.
What I was expecting is that the windows credential manager would swap the credentials as it does when the job is run manually, or through excel or a number of any other ways...
I opened a ticket with Microsoft and worked with one of their senior resources on this.
this appears to be a bug in SQL Agent. There is no known reason or issue that prevents SQL Agent from picking up the remote credential from the Windows Credential Store, but it is not.
A working alternative was to use the command line utility DTEXEC. some slight modification to the SSIS project to make sure all connection managers are at the package level instead of project (created a reference issue).
this solution is not ideal, but it worked and DTEXEC allowed SSIS to pickup the required credential in the store and execute without issue.
I will follow up once Microsoft completes their research and gets back to me, the ticket is still open.
Sorry but changing the group to Global or Universal for the local AD account is not having any effect. I am bit lost on how making a change on the local account in use for SQL Agent will make any difference. The solution works in all the tools by local account substitution with the remote account setup in Credential Manager. If I missed it, and making this change should work, please provide an example of the setup if possible.
Again it appears this process is not being executed/followed by SQL Server Agent since it works everywhere else but not in a job executed by the Agent.
so again my hope is somebody has seen something like this before and has a solution.
my end goal is to just automate pulling data from a remote SQL server where there is no trust. I was hopeful that the proxy solution would work, but when you set the credentials to the remote domain\username, the job wont even execute.
Is there a way to setup my connection in the SSIS package to expressly set the credentials to the remote domain\username\pwd. I took a stab at that and couldnt get that to fly either. if so, an example is priceless to me.
I dont care how i get to the goal line, just need to...thanks all for the help
Your window credential account should be an AD user account which is in a Group with a scope of either Global or Universal. Universal groups are useful where you have multiple domains.
The process will execute in whatever context it's called in (i.e. by you, SQL Agent, or the proxy account). It doesn't change executable context as it calls different processes, unless you pragmatically make it, and that's bad idea anyway...
Had similar issue and it was a nightmare to resolve! Learned a lot of fun AD configuration tips along the way.
Understanding User and Group Accounts states the following:
Groups can have different scopes—domain local, built-in local, global, and universal. That is, the groups have different areas in which they are valid.
Global groups:
Groups that are used to grant permissions to objects in any domain in the domain tree or forest. Members of global groups can include only accounts and groups from the domain in which they are defined.
Universal groups:
Groups that are used to grant permissions on a wide scale throughout a domain tree or forest. Members of global groups include accounts and groups from any domain in the domain tree or forest.
EDIT
If it's just a data pull from one domain to another, can the data be first exported to a csv in the untrusted domain and then SFTP'd into your environment where the TL(Transform and Load) of the ETL(Extract-Transform-Load) process could take place?
SSIS would be a good tool for this, but C# and Powershell could also be used.
I spent a long time yesterday to configure for my CouchDB instance in order to create a little app and letting CouchDB manage authentication and authorizations for me.
So I ended up with something like that :
On top of everything I've got a server admin, who basically is god on my CouchBD instance.
Then I created a database named "mydatabase" (for example) and added the role "mydatabase_dba" as admin and also the role "mydatabase_user" as reader.
I also created a database named "_users" which contains all the database admins and users with their roles and also a design document named "_auth" which manages authorizations.
Only the server admin is admin of this database, and I added users with role "mydatabase_dba" as readers. Then, for those of you who knows about it, I modified the "validate_doc_update" field o the "_auth" document so that users with role "mydatabase_dba" can only deals with users with role "mydatabase_user".
So, to summarize at this point :
server admin is still god
users with role "mydatabase_user" can connect to "mydatabase" but they are just readers
users with role "mydatabase_dba" are admins of "mydatabase"
users with role "mydatabase_dba" can connect to database "_users" where they are readers
users with role "mydatabase_dba" can only manage users of role "mydatabase_user" in "_users"
Hope this is clear :D
What I can do now is create an application that will not manage users itself, but let users connect to CouchDB directly (transparently).
The problem come when it deals with users creation/update/deletion.
Because only users with role "mydatabase_dba" can access to the "_users" database and work on users with roles "mydatabase_user", I need at some point to connect to CouchDB as this db admin.
I have two solutions :
Create a user interface into my app that will let the admin connect and do what he has to do
or
Make some more code and let the app do it automatically, this is the solution I prefer, but the problem is : I have to store the admin credentials...
Sorry for the long introduction but I had to describe the landscape first :)
I created a post yesterday about how I could secure the connection between my app and the CouchDB instance : here
The solution I was given is to use HTTP over SSL (/TLS) to secure the communication. I'm okay with that, but now I have another concern, maybe I'm paranoid, but because my app will need to connect as "mydatabase_dba", I have to store its credential somewhere.
But how to store them securely ? As said in my previous post, even if I store the hashed password instead of the plain text password, if an attacker access my app source code, he'll have my admin credentials...
An application should never have an administrative rights. It should only be given the bare minim rights it needs to function. If the application needs some administrative rights, make sure it has as few as possible. Other than that, most of the time these credentials are stored in plain text in some file that only your application can access.
Never commit this text file into your source code manager (Subversion, Git, etc.)! Placing the file into a running system must be a step in the installation procedure.