Running SQL Server 2016. Currently have a solution that is hosted in one domain and of course our access point is in another. we need to pull data in an automated fashion.
We have added a windows credential with the credential manager which collects endpoint information and a set of credentials.
e.g.
Internet or Networkaddress: mydatabase.remotedomain.com
Username : remotedomain\username
Password : password
This solution works with many tools, Excel, SSMS direct query, Visual Studio. The user enters the endpoint (server url or IP/port) and uses windows integrated security. the connection is made and credential store does the trick and user is authenticated.
SSMS example
Server name: mydatabase.remotedomain.com
Authentication: Windows Authentication
My challenge is SSIS and SQL Agent. The SSIS package runs in VS2015. deploy the package to Integration Services Catalog - highlight package and execute and it runs.
Create a SQL Server Agent Job and execute the job and I receive this gift.....
Login Failed: The login is from an untrusted domain and cannot be used with Windows authentication
I have created a SQL credential, created a Proxy (SSIS Package Execution), created a job that uses the Run As with the Credential but this ends with the same result. The credential has to be in my local domain or the job wont run....and of course localdomain\username does not authenticate against the remote data connection. So Proxy does not help the situation.
What I was expecting is that the windows credential manager would swap the credentials as it does when the job is run manually, or through excel or a number of any other ways...
I opened a ticket with Microsoft and worked with one of their senior resources on this.
this appears to be a bug in SQL Agent. There is no known reason or issue that prevents SQL Agent from picking up the remote credential from the Windows Credential Store, but it is not.
A working alternative was to use the command line utility DTEXEC. some slight modification to the SSIS project to make sure all connection managers are at the package level instead of project (created a reference issue).
this solution is not ideal, but it worked and DTEXEC allowed SSIS to pickup the required credential in the store and execute without issue.
I will follow up once Microsoft completes their research and gets back to me, the ticket is still open.
Sorry but changing the group to Global or Universal for the local AD account is not having any effect. I am bit lost on how making a change on the local account in use for SQL Agent will make any difference. The solution works in all the tools by local account substitution with the remote account setup in Credential Manager. If I missed it, and making this change should work, please provide an example of the setup if possible.
Again it appears this process is not being executed/followed by SQL Server Agent since it works everywhere else but not in a job executed by the Agent.
so again my hope is somebody has seen something like this before and has a solution.
my end goal is to just automate pulling data from a remote SQL server where there is no trust. I was hopeful that the proxy solution would work, but when you set the credentials to the remote domain\username, the job wont even execute.
Is there a way to setup my connection in the SSIS package to expressly set the credentials to the remote domain\username\pwd. I took a stab at that and couldnt get that to fly either. if so, an example is priceless to me.
I dont care how i get to the goal line, just need to...thanks all for the help
Your window credential account should be an AD user account which is in a Group with a scope of either Global or Universal. Universal groups are useful where you have multiple domains.
The process will execute in whatever context it's called in (i.e. by you, SQL Agent, or the proxy account). It doesn't change executable context as it calls different processes, unless you pragmatically make it, and that's bad idea anyway...
Had similar issue and it was a nightmare to resolve! Learned a lot of fun AD configuration tips along the way.
Understanding User and Group Accounts states the following:
Groups can have different scopes—domain local, built-in local, global, and universal. That is, the groups have different areas in which they are valid.
Global groups:
Groups that are used to grant permissions to objects in any domain in the domain tree or forest. Members of global groups can include only accounts and groups from the domain in which they are defined.
Universal groups:
Groups that are used to grant permissions on a wide scale throughout a domain tree or forest. Members of global groups include accounts and groups from any domain in the domain tree or forest.
EDIT
If it's just a data pull from one domain to another, can the data be first exported to a csv in the untrusted domain and then SFTP'd into your environment where the TL(Transform and Load) of the ETL(Extract-Transform-Load) process could take place?
SSIS would be a good tool for this, but C# and Powershell could also be used.
Related
I have a specific problem to which I couldn't find any answer online.
The situation is the following:
We created a Canvas app that connects to the Azure SQL database. We set the connection type to be AAD integrated.
Users that are members of the AD can see the data in the app, but guest users, even though we gave them all the rights and PowerApps plan, cannot see the data. they recieve the same authorization window as members, but when they click on allow, the app starts but no data is being pulled from the SQL database.
When we try to connect directly to the Azure SQL database with the guest user email and credentials (via SQL server management studio), everything works as expected and the guest can see all the tables.
When we use implicitly shared connection (with SQL server authentification), guests can see the data, but we need to use AAD integrated due to its security.
So far we tried:
Changing PA environment from sandbox to production
Adding special permissions in SQL database like database owner etc
Trying out different AAD guest user settings, eq- setting that guest users have the same privileges as members (picture below).
Nothing seems to work. I would be more than happy if you could tell me how to make this work or even push me in the right direction.
I've reproduced your problem in my side. Here's my experience.
After assigning powerapp license(I use O365 E5 here) to guest account and share the app to this account, I can visit the app but can't see the data in the table. I assume that it resulted from the connector hasn't been shared, but it's true that this connector can't be shared because of no 'share button'.
Then I tried to add access policy to my guest account with these two sql:
CREATE USER [tiny-wa_outlook.com#EXT##xx.onmicrosoft.com] FROM EXTERNAL PROVIDER;
GRANT SELECT ON dbo.tinyTest TO [tiny-wa_outlook.com#EXT##xx.onmicrosoft.com];
Pls note here, I used the User Principal Name here(can find the principal name in azure ad->users), I also tested to use 'tiny-wa#outlook.com' in the sql but after executing successfully, it didn't work.
So I think you can try to use the sql above to add your guests accounts and make them can access the powerapp.
Here's some related docs:
create contained users mapped to azure ad identities
Share app resources
add table permission to user
==========================UPDATE==========================
Share my experience of this troubleshot.
First I need to create a power app but I found that after creating the connector with sql server azure ad authentication, it can't connect to the sql server, the error message is like 'Credentials are missing or not valid. inner exception: The credentials provided for the SQL source are invalid', solution is add my admin account as sql server instance Active Directory Admin.
Then I choose a table and successfully create a sample app. With the creating account, I can visit the app but other accounts can't. Here needs to share the app and it's related connectors to other users. But other accounts still can't reach the app because of no license. Because sql server connector is premium connector, so I assign O365 E3 license here. I met an error when assign license, the user's 'Usage location' can't be null or it can't assign license in M365 admin center.
Then I met similar error with Op, the difference is that both member account and guest account can't see the data in app. I try to find the way to share the connector to these uses but failed, I haven't made sure if those connectors without share button can be shared to others. So I have no options to study if this kind of connectors are authenticated in other ways so they don't need to be shared at all.
Next actions is using the account which used to create the sql server and database to sign in database and execute the sqls above.
Then the member account can see data in the power app while the guest account can't see. The reason is I used xx#outlook.com as the parameter in the sql, when I used xx_outlook.com#EXT##xx.onmicrosoft.com, it worked finally.
Hope this can also help you.
===================================================
For creating my demo app: First, I'm sure my environment isn't a sandbox(the environment in the screenshot below). And I think it's easy to create a demo app, and my app is simple, just choose to create an app from data and then select sql server as the connector, next I choose auzre ad auth and click the connector, enter server name and database name then choose a table, after that my app has created. That table has one row of data so when I signed in the app with the creating account, I see it in the screen while other accounts(member or guests) can't.
My sql server instance and database are created long time ago, but I'm sure I followed this tutorial to create them.
This appears to still be a limitation to access to Azure SQL via PA connector for guest users:
https://powerusers.microsoft.com/t5/Power-Apps-Ideas/Azure-SQL-to-PowerApps-Connector-AAD-doesn-t-work-for-guest/idi-p/1637817
If the "guest" does NOT have a PowerApps Per-App/Per-User plan, they cannot use your PowerApps with SQL data source (Note: SQL is a premium connector).
Determine exactly which type of license the guest has. Then, either your organization or the guest (or guest's org) must purchase one of these licenses. $5/$10/$20 per month depending on your use case.
REF
I have just created an SSIS package and was able to hook it up to a SQL Server Agent Job. In the tutorial, it showed me to create a Proxies account. What exactly is a proxies account?
In many job step types you can specify "Run as..." accounts (or proxies, or credentials) to execute this specific job step.
You need this, for example, if your SQL Server Agent service is running under Local System account, but you wish to run your SSIS Package under particular domain user (to access file shares, remote servers and so on - actually there are a lot of cases there you need different logins to run your tasks).
So, you create (provide) Credentials (login and password) of this user (SQL Server encrypts and store them for future uses - so you should provide them just once), and then create so called "proxy", assigning this credentials to different types of tasks. Then you just use them in the "Run as..." fields.
Other practice (I think not good, but could be used too) is to simply run SQL Agent Service under elevated account. Even domain admin, if you domain just local, or very secured. If you do so - you'll need proxy only in the very specific cases...
So the proxy - is the feature to enhance (or maintain) security of your environment.
One more case there you may need different logins for your ssis packages is the Resource Governor feature. For instance: 4 databases, 4 ETL cycles (many jobs'n'steps) for each database, and a 4 proxy accounts (logins) for each ETL cycle assigned to them tasks. Then you can assign Workload group for each login and divide your server resources between them as you wish. And you'll see in the SSISsyslog the execution account for each of your packages, so you should be able to determine easily that do you need to do in case of any problems.
Hope this helps :)
I want to create a publication task in Jenkins to automatically publish my database changes along with my application.
If I understood correctly, a common practice is to create a publish profile that includes the database name as well as the account (login and password) of the account used for the deployment.
This means that the deployment account username and password will be stored in clear text on each developer computer as well as on the version control server and the continuous integration server.
Even though I created a specific login and password for the deployment, it seems pretty unsecured to me.
Is there a workaround? I can only think of replacing the password within the msbuild command line on the continuous integration server.
tl;dr version
Windows Authentication is the preferred, secure method of connecting to your SQL Server instance and if it's possible to use that then it's recommended to use that for connections.
If SQL Authentication is used then the default in publish profiles is that the password isn't saved. For build servers and other shared profile scenarios you may need to accept lower levels of security (by editing the publish profile to add the password, or setting it as a parameter in the build configuration) or work around it in some other way (custom script that reads it from some kind of a secret store, such as an encrypted value).
Long version
Windows Authentication: If at all possible use Windows Authentication, giving permissions as required to users who need it. For Continuous Integration scenarios you would need to give appropriate permissions to the account the build server executes under - full details are in the recent whitepaper on the SSDT blog.
SQL Authentication: If you look at the publish profile (Open With... Xml Editor) you'll see that the password information isn't actually stored there.
If you choose "Save Password" you'll have "Persist Security Info=True;" stored in the connection string rather than the password itself.
When a connection is made to a server/database in SSDT with "Save Password" enabled, the connection info is encrypted and stored in the registry under "HKEY_CURRENT_USER\Software\Microsoft\SSDT\ConnectionStrings". This has to be present on the machine in order to successfully publish using the publish profile.
Hence in a team environment every user would need to connect at least once before that publish profile would work for them. However, the password would be safely encrypted on user machines.
For the build server, your options are more limited. One possibility is to manually log in as the build server user and then connect to the database, but this isn't very scalable. To avoid the less secure options you mentioned you'd need to implement your own logic for persisting the password securely. You can look at the Protected Data API which can be used to do something similar to what SSDT does but on a per-machine level, or use an encrypted configuration file.
If you have to use SQL Authentication I think passing the password into the publish action as part of the build configuration may be the "best" way to go in terms of a tradeoff between ease of development and security. At least that way you can restrict who can view and edit the build configuration in TFS and regular developers won't see it.
I've been searching hard to get my head around security related issues in a SQL Server.
We're developing a .NET application that targets SQL Server 2008 and we want to use FileStream.
Now I've found out that SQL Server only allows FileStream through the Win32 API if you use Integrated Security. The problem is that we have around 80% of our application finished, but it is entirely based on SQL Authentication. So we are doing INSERT's straight form our application and are not using Stored Procedures for every CRUD operation.
This is relatively safe because I can store the the SQL username and password in an encrypted form. I know the password is transported in Clear Text, but I'm willing to accept that.
We want end-users to be able to connect to the databse through tools such as Crystal Reports and for that we have an extra SQL login that has only SELECT-rights granted.
Now, if we change to Integrated Security we would have to give individual users (via AD groups, etc.) rights to do the things the application can do. Otherwise the application would not be able to do it's work. But then the end-user would also have these rights when he connects straight to the DB.
I see people saying that you should use Stored Procedures for every CRUD operation and grant the EXEC-rights only to the AD-group, but how would I do this? I do not see how a user would have different authorizations when he connects directly or through the application... Can anybody enlighten me on this.
An extra question for bonus-points: Intergrated Security will not work on a Workgroup as far as I understand. How do people get FileStream to work in a Workgroup then? Or is this considered an impossibility?
Integrated security WILL work in a workgroup, using the legacy mechanism, where you have a matching username and password on the two machines. Also, a domain user can use the legacy mechanism to log into a non-domain server if the server has a matching user account.
Integrated security can even work with non-matching usernames and passwords. This may help you in your scenario.
Try this:
NET USE \\DBSERVER /USER:DOMAIN\USERNAME
You will be prompted for your password. This establishes a NetBIOS session with the database server. You should be able to see the shared folders and shared printers on the database server once you have done that.
Once a netbios session has been established between the client computer and the database server, you will THEN be able to use integrated security without being prompted for a password.
You may have to specify "named pipes" as the network protocol to usem, if it doesn't work with TCP (but I think it will). Named Pipes inherits your existing NetBIOS session, so provided you can list the shares you are probably good to go.
You can also establish the logon session using the windows API function NetUseAdd with USE_INFO_2 (level 2) information which incorporates the password.
I guess the short answer then is that you can have a special Windows logon for your application and have the users log in using that. However note that they cannot also be connected to the same server using their own username and password.
We have an application that uses Windows authentication to authenticate users with the database, and the SQL Server user accounts need to have certain read/write access to database tables.
The trouble is that the users can then install SQL Server Management Studio and potentially use the database in ways it's not supposed to be used, which isn't what I want.
Everything that I have read says that using integrated authentication is more secure but at the moment, any user can use Management Studio or Access/Excel to just connect to the database.
I have read question SQL Server Authentication or Integrated Security?, which suggests some workarounds, but I don't really have the option of changing the app as drastically as re-factoring all of the stored procedures etc. so I was hoping there might be another option?
Thank you,
NIco
Everything that I have read says that
using integrated authentication is
more secure
--> It's more secure in a way because it's more difficult to get the password.
If you use SQL Server authentication, the connection string contains user and password. If you know where the connection string is (often in a config file), you can open it and see user and password.
On the other hand, if you use Windows authentication, the connection string just says "Integrated Security=True" and you connect to the server with your Windows account, with the actual password buried somewhere deep in Windows' guts and more difficult to retrieve.
Of course, the big downside of Windows authentication is that if your users need write permissions on a certain table for your application, this means that they can write to the same table with ANY other application as well.
There are some workarounds, but none of them is THE silver bullet:
If your app only needs certain tables of the DB, you can just give permissions on these. So at least, the users can't do stuff in all the other tables
If the users are not allowed to access any tables at all from outside your application, there are unfortunately only two things you can do:
Change your app to SQL authentication and remove all permissions for Windows users
(you can also use a proxy service like Will Hughes suggested, but the effect is the same when the app accesses the DB directly...the point is that your users' Windows accounts don't have any permissions anymore!)
Create views and stored procedures (if they don't already exist anyway) for the stuff your app can do with the database. Give the users permissions to use these, and remove the permissions to the real tables.
--> the users can access the views and SPs directly with other tools (even if they don't have any permissions on the underlying tables...permissions on the views and SPs are enough), but they can't do anything that they can't do in your app as well.
If you don't want users to have access to your database, don't grant them access.
If you need to control what they can do - then you should do your access control in a webservice (or some other form of proxy service), which will then execute approved queries, return data, etc.