Variations to this have been asked. I have no problem searching a local directory with the below piece of code.
EXEC MASTER.sys.xp_dirtree 'C:\', 1, 1
When I switch the path to a network location the results are empty.
EXEC MASTER.sys.xp_dirtree '\\Server\Folder', 1, 1
I first thought maybe it was something to do with permissions. I added the SQL Server Service to the ACL list on the shared volume as well as the security group.
Any help or direction to point me in is greatly appreciated or even another way to get a list of files in a directory and sub directories.
[Edited]
The two things to look out for are:
Make certain that the Log On account for the SQL Server service (the service typically listed as "SQL Server (MSSQLSERVER)" in the Services list) has rights to that network share.
UPDATE
The problem ended up being that the O.P. was running the SQL Server service as a local system account. So, the O.P. created a domain account for SQL Server, assigned that new domain account as the Log On As account for the SQL Server service, and granted that domain account the proper NTFS permissions.
Please note that this might have also been fixable while keeping the SQL Service running as a local system account by adding the server itself that SQL Server is running on to the NTFS permissions. This should usually be possible by specifying the server name followed by a dollar sign ($). For example: MySqlServer01$. Of course, this then gives that NTFS permission to all services on that server that are running as a local system account, and this might not be desirable. Hence, it is still preferable to create a domain account for the SQL Server service to run as (which is a good practice in any case!).
It sounds like this has been done, so it should be tested by logging onto windows directly as that account and attempting to go to that specific network path.
Make sure that the Login in SQL Server that is executing xp_dirtree has "sysadmin" rights:
This can be done directly by adding the account to the sysadmin server role, or
Sign a stored procedure that runs xp_dirtree:
Create a certificate in [master]
Create a login based on that certificate
Add the certificate-based login to the sysadmin server role
Backup the certificate
Restore the certificate into whatever database has, or will have, the stored procedure that runs xp_dirtree
Sign the stored procedure that runs xp_dirtree, using ADD SIGNATURE and the certificate that was just restored
GRANT EXECUTE on that stored procedure to the user(s) and/or role(s) that should be executing this.
Just to have it stated, another option is to do away with xp_dirtree altogether and instead use SQLCLR. There is probably sample C# code on various blogs. There are also a few CodePlex projects that have file system functions and might also provide a pre-compiled assembly for those that don't want to deal with compiling. And, there is also the SQL# library that has several filesystem functions including File_GetDirectoryListing which is a TVF (meaning: you can use it in a SELECT statement with a WHERE condition rather than needing to dump all columns and all rows into a temp table first). It is also fully-streamed which means it is very fast, even for 100k or more files. Please note that the FILE_* functions are only in the Full version (i.e. not free) and I am the creator of SQL#, but it does handle this situation quite nicely.
Related
Running SQL Server 2016. Currently have a solution that is hosted in one domain and of course our access point is in another. we need to pull data in an automated fashion.
We have added a windows credential with the credential manager which collects endpoint information and a set of credentials.
e.g.
Internet or Networkaddress: mydatabase.remotedomain.com
Username : remotedomain\username
Password : password
This solution works with many tools, Excel, SSMS direct query, Visual Studio. The user enters the endpoint (server url or IP/port) and uses windows integrated security. the connection is made and credential store does the trick and user is authenticated.
SSMS example
Server name: mydatabase.remotedomain.com
Authentication: Windows Authentication
My challenge is SSIS and SQL Agent. The SSIS package runs in VS2015. deploy the package to Integration Services Catalog - highlight package and execute and it runs.
Create a SQL Server Agent Job and execute the job and I receive this gift.....
Login Failed: The login is from an untrusted domain and cannot be used with Windows authentication
I have created a SQL credential, created a Proxy (SSIS Package Execution), created a job that uses the Run As with the Credential but this ends with the same result. The credential has to be in my local domain or the job wont run....and of course localdomain\username does not authenticate against the remote data connection. So Proxy does not help the situation.
What I was expecting is that the windows credential manager would swap the credentials as it does when the job is run manually, or through excel or a number of any other ways...
I opened a ticket with Microsoft and worked with one of their senior resources on this.
this appears to be a bug in SQL Agent. There is no known reason or issue that prevents SQL Agent from picking up the remote credential from the Windows Credential Store, but it is not.
A working alternative was to use the command line utility DTEXEC. some slight modification to the SSIS project to make sure all connection managers are at the package level instead of project (created a reference issue).
this solution is not ideal, but it worked and DTEXEC allowed SSIS to pickup the required credential in the store and execute without issue.
I will follow up once Microsoft completes their research and gets back to me, the ticket is still open.
Sorry but changing the group to Global or Universal for the local AD account is not having any effect. I am bit lost on how making a change on the local account in use for SQL Agent will make any difference. The solution works in all the tools by local account substitution with the remote account setup in Credential Manager. If I missed it, and making this change should work, please provide an example of the setup if possible.
Again it appears this process is not being executed/followed by SQL Server Agent since it works everywhere else but not in a job executed by the Agent.
so again my hope is somebody has seen something like this before and has a solution.
my end goal is to just automate pulling data from a remote SQL server where there is no trust. I was hopeful that the proxy solution would work, but when you set the credentials to the remote domain\username, the job wont even execute.
Is there a way to setup my connection in the SSIS package to expressly set the credentials to the remote domain\username\pwd. I took a stab at that and couldnt get that to fly either. if so, an example is priceless to me.
I dont care how i get to the goal line, just need to...thanks all for the help
Your window credential account should be an AD user account which is in a Group with a scope of either Global or Universal. Universal groups are useful where you have multiple domains.
The process will execute in whatever context it's called in (i.e. by you, SQL Agent, or the proxy account). It doesn't change executable context as it calls different processes, unless you pragmatically make it, and that's bad idea anyway...
Had similar issue and it was a nightmare to resolve! Learned a lot of fun AD configuration tips along the way.
Understanding User and Group Accounts states the following:
Groups can have different scopes—domain local, built-in local, global, and universal. That is, the groups have different areas in which they are valid.
Global groups:
Groups that are used to grant permissions to objects in any domain in the domain tree or forest. Members of global groups can include only accounts and groups from the domain in which they are defined.
Universal groups:
Groups that are used to grant permissions on a wide scale throughout a domain tree or forest. Members of global groups include accounts and groups from any domain in the domain tree or forest.
EDIT
If it's just a data pull from one domain to another, can the data be first exported to a csv in the untrusted domain and then SFTP'd into your environment where the TL(Transform and Load) of the ETL(Extract-Transform-Load) process could take place?
SSIS would be a good tool for this, but C# and Powershell could also be used.
I'm working on an application right now that requires a link to a couple of SQL Server tables. My windows network account has permission to connect to this server, but I am not going to be the only one using this application. I'm going to send it out for people to save to their PC or just put it on the company shared drive to use (I know, that's asking for problems sometimes). It's inconvenient to make a windows account for users to share because they would need to log out and in to use the app, so I was wondering if the application or ODBC connection file itself can store the credentials to access the table.
Should I configure the connection object to use something other than the windows login information (maybe a SQL server username/password), and just store the connection object in a shared location? I don't have much experience with this and haven't tried out many different solutions and I am open to suggestions.
Thank you for the suggestions
As suggested in a comment to the question, one solution would be to
create a User Group in Windows on the SQL Server,
create a SQL Server login for that group,
assign permissions within SQL Server to that login,
and then just add or remove particular Windows Users from that group as required.
That way you don't need to mess with the various SQL Server permissions for each database user, and your application can connect to the SQL Server using Windows Authentication so you don't have to mess with saved SQL Server credentials (in connection strings, or elsewhere).
You certainly can specify the username & password in the connection string -- ConnectionStrings.Com is highly recommended if you are having trouble with connection strings -- their first example for Sql Server is
Server=myServerAddress;Database=myDataBase;User Id=myUsername;Password=myPassword;
The issue is security, if users can see this in a configuration file, they can do anything that account can do. You have to handle security within you application if you do this. Most apps that handle their own security have to create users and passwords in a database table (best not to store password at all, much less plaintext -- a one way hash is recommended).
One good strategy is the create a "login user" account with well known name and password, grant no read / write, etc. for that account at all, and grant execute access to single stored proc
IsLoginPermitted #ID, #PASS
When successful, IsLoginPermitted returns the ID & PASS for subsequent use (of course these are hidden from the user) and you create your new connection string based on these.
I don't know if it should be like this. When I'm trying to do anything with xp_cmdshell procedure it almost every time gives me Access Denied.
For example I can't create new .txt file, can't create new user, nothing. I'm logged in with windows administrator user.
Is there any way to run this procedure with administrator privileges?
XP_CmdShell will execute under the context of the Service Account running the SQL Server Service. The service account needs the permissions to the external resources.
Could I point out however, that enabling xp_cmdshell is not a good idea. It opens lots of security holes. For example, if your app has an unknown volnerability to SQL injection, a hacker could do all sorts on your network that you rather avoid.
If you must use external resources then better approaches would include a CLR procedure or calling a Job that executes a CMDEXEC step.
xp_cmdshell executed by a windows login is executing under an impersonation context. as such any access of a remote resource (eg. access a file on a share, an operation on AD like adding an user) will fall under the constrained delegation restrictions, likely resulting in a access denied because constrained delegation is probably not to be configured on all those resources.
I've created a stored procedure in SQL Server that in turn is using a linked server to query two databases. The results are returned as expected in Query Analyzer, but when I try running the same stored procedure using a pass-through query in Access I get an authentication error.
Access has no problem quering either database when I use the passthrough and either DSN, it's only when I try to execute the stored procedure.
It almost seems like the DSN connection can't pass on the credentials to the linked server via the current DSN server.
The user is a member of domain admins, has full rights to both sql servers and I'm using Integrated Security on both DSNs.
Sounds like a classic double hop issue. The article Configuring Linked Servers for Delegation discusses how to work around this. But you'll note that one of the first requirements listed is:
The user Active Directory property, Account is sensitive and cannot be delegated, must not be selected.
which is almost certainly selected for the domain admin account (or at least, it should be in a well run domain)
i have this script:
BACKUP DATABASE timeclockplus TO DISK = 'C:\Program Files\Microsoft SQL Server\MSSQL.2\MSSQL\Backup\backup.bak'
how do i know which username is executing it? the reason i ask is because i want to grant extra privelages to the current username so that instead of writing that backup folder i can write to a different folder
There is no context as such except for sa inside the DB engine.
Based on your recent questions, you mean external SQL Server context which will be the service account used (you can see it in Services.msc). So if you are using domain\Specialuser then you could permission domain\Specialuser on a folder.
However, SQL Server 2005 introduced some groups that should contain the service account specified at install time and these should be used for external (eg outside SQL Server) permissions
Some details in KB 283811 , but best seeing "Reviewing Windows NT Rights and Privileges Granted for SQL Server Service Accounts". The groups mentioned here are what you should permission on your desired folder.
Take a look at SUSER_SNAME().
(That page is a bit misleading, passing no param returns the current user - either an SQL Logon or a domain\user depending on the auth mode)