I have an SSIS Package I have developed with an 'Execute Process Task' and 'Data Flow Task'. I am having an issue with execution of the package.
RESULTS: (Manual vs. Scheduled)
All processes are successful when run executed manually in 32-bit mode from VS and SSISDB when I login with a specific user profile 'GEORGES/BL0040EP'.
'Execute Process Task' -- Run a powershell script to refresh excel connections and save the excel file. (SUCCESS)
'Data Flow Task' -- Reads the excel data and inserts it to SQL Server table. (SUCCESS)
I am trying to run the package with SQL Agent (using a proxy account), and the process has some issues.
'Execute Process Task' -- Run a powershell script to refresh excel connections (DOES NOT WORK, NO ERROR MESSAGES). Save the excel file (SUCCESS).
'Data Flow Task' -- Reads the excel data and inserts it to SQL Server table. (SUCCESS)
PROBLEM:
The powershell script to refresh the Excel file seemingly does not get issued to the SSAS Server when run from SQL Agent. Not "query issues successfully". Not "query issues with permissions error". Simply "query is not at all executed". No permissions issues are logged or detected.
I can tell because I ran SQL Profiler on the server. When Agent calls the package there is no query activity. When VS/SSISDIB calls the package I can see the query being issued. Both successfully with user profile (GEORGES\bl0040ep); and unsuccessfully with permissions error with an unauthorized user profile (GEORGES\bl0040).
QUESTION:
Why would SQL Agent not run the query?
I even added to the posh command $env:UserName | Out-File -filepath to output a text file containing the user name. And the Proxy Account setup appears to be running under the expected user profile context. Content of the text file is bl0040ep.
Create Proxy Account
Creating a Proxy User to run an SSIS package in SQL Server Agent
USE master
GO
-- Create a proxy credential for xp_cmdshell.
EXEC sp_xp_cmdshell_proxy_account 'GEORGES\bl0040ep', '!myPW!';--SELECT * FROM [master].[sys].[credentials]
-- Grant execute permission on xp_cmdshell to the SQL Server login account.
GRANT exec ON sys.xp_cmdshell TO [GEORGES\bl0040ep]
GO
-- Create a credential containing the GEORGES account PowerGEORGES\PowerUser and its password
CREATE CREDENTIAL Credential_BL0040EP WITH IDENTITY = N'GEORGES\bl0040ep', SECRET = N'!myPW!'
GO
USE [msdb]
GO
-- Create a new proxy called SSISProxy and assign the PowerUser credentail to it
EXEC msdb.dbo.sp_add_proxy #proxy_name=N'Proxy_BL0040EP',#credential_name=N'Credential_BL0040EP',#enabled=1
-- Grant SSISProxy access to the "SSIS package execution" subsystem
EXEC msdb.dbo.sp_grant_proxy_to_subsystem #proxy_name=N'Proxy_BL0040EP', #subsystem_id=11
-- Grant the login testUser the permissions to use SSISProxy
EXEC msdb.dbo.sp_grant_login_to_proxy #login_name = N'GEORGES\bl0040ep', #proxy_name=N'Proxy_BL0040EP'
GO
DatabaseSSAS_UsageStats_xlsx_ExcelRefresh.ps1
# Refresh the excel workbook connections and save the updated file
$file = 'C:\SVN\BusinessAnalysts\ExcelTools\DatabaseSSAS_UsageStats.xlsx'
$x1 = New-Object -ComObject Excel.Application
$x1.Visible = $false
$x1.DisplayAlerts = $False
$enddate = (Get-Date).tostring("dd-MM-yy")
# $filename = 'C:\SVN\BusinessAnalysts\ExcelTools\DatabaseSSAS_Usage Stats ' + $enddate + '.xlsx'
$filename = 'C:\SVN\BusinessAnalysts\ExcelTools\DatabaseSSAS_UsageStats.xlsx'
$env:UserName | Out-File -filepath C:\SVN\BusinessAnalysts\ExcelTools\RefreshAll_process.txt
$wb = $x1.workbooks.Open($file)
$wb.refreshall()
# REM: Use SLEEP to eliminate the message: "This will cancel a pending data refresh. Continue?"
Start-Sleep -Second 20
$wb.SaveAs($filename)
$wb.Close()
$x1.Quit()
Remove-Variable wb,x1
SQL Profiler
No activity is captured when running the package from SQL Agent. When run from VS and SSISDB, a login error is captured.
SQLProfiler_(VS/SSIDB-connectionerror_user-bl0040).png
SQLProfiler_(SQLAGENT-connection_no-activity).png
There is no image to attache. Simply there is no activity for the user (bl0040ep) when the package is run under the SQL Agent context.
Wed 02/06/2019 14:31:46.96
UPDATE 1: System Desktop Folder
I added ‘Desktop’ folder on System32, ran the job from SQL Agent, and the issues persists. This was recommended on a similar issue reported on TechNet Issues with simple script executed via SQL Server Agent... C:\Windows\System32\config\systemprofile\Desktop and C:\Windows\SysWOW64\config\systemprofile\Desktop. The folder already existed on SysWOW64.
UPDATE 2: PowerShell executable in 32-bit
I have also tried to directly invoke the 32-bit version of PowerShell: %SystemRoot%\syswow64\WindowsPowerShell\v1.0\powershell.exe. Still the excel refresh does not complete when run via SQL Agent with proxy account. Reference Run a 32 bit Powershell script on Sql Server Agent
UPDATE 3: Windows Task Scheduler instead of SQL Agent Job
I have tried a different scheduling method for the package execution (Windows Task Scheduler, instead of SQL Agent). The schedule completes but it behaves the exact same way that SQL Agent does... the query is not being sent to the datasource. Reference Nirav's Diary, Schedule SSIS Package Without Deploying, 2-Windows Schedule Task
I have solve this by removing Excel from the picture. Instead of Excel source I am now using the connector .Net Providers for OleDb\Microsoft OLE DB Provider for Analysis Services 2.0. There are a couple extra steps needed in the database to get the same dataset (using SQL Views instead of the Excel Powerquery M).
I lost this fight to Excel... it did not want to be automated. I did end up still using the same CREDENTIALS and PROXY to schedule the job.
I have an SSIS Package that I created through the Import Export (32-bit) Tool. When I executed the package manually through the Execute Package Utility the package run successfully with no issues. However when I try to run the package through a Job Agent in SSMS I keep getting errors. The primary error I get seems to be:
Failed to decrypt protected XML node "DTS:Password" with error
0x8009000B "Key not valid for use in specified state" You may not be
authorized to access this information. This error occurs when there is
a cryptographic error. Verify that the correct key is available.
I'm using SQL Server 2008 R2.
I have researched this error to some degree and I think it has something to do with the package protection level. I feel like I've tried the configurations that make the most sense but none seem to be working for me. The Options are:
Encrypt sensitive data with user key
Do not save sensitive data
Encrypt sensitive data with password
Encrypt all data with user key
Encrypt all data with password
Rely on server storage and roles for access control
I feel like Ishould be using the last option here (Rely on server storage...) because I prefer to use SQL Server Authentication. I use SQL Server Authentication on the 'Choose Destination' window of the SQL Server Import and Export Wizard, and similarly I use this with the same username and password when I create the Job Agent in SSMS on the General Tab of the Job Step Properties. Is it possible that there is something that I need to add to the User I'm using in SSMS - even though it works outside of SSMS?
Something else I wondered that might have an impact is having the option "Drop and Re-create destination table" checked in the Column Mappings window of the Import and Export Tool. I was using a stored procedure to remove the tables before executing the Job Agent and I feel like ti was working at one point - could that have something to do with it?
Again the thing that baffles me most is that it runs no problem when I execute it manually through the 'SQL Server Execute Utility Package' tool.
I've included images of some of the windows I mentioned above if that helps.
I'm having a very strange problem with a fresh install I have of SQL Server 2008 Express edition (yeah it's a bit old now, but whatever). When I connect via SQL Server Management Studio, I can both read and edit data (update or insert), but when I connect via my web application's data access layer, which uses SqlConnection and SqlCommand to try and update and insert data in tables, no changes occur in the database. The strange thing is that the code runs as if no error had occurred though; no exceptions are thrown, and my update statement causes SqlCommand.ExecuteNonQuery to return 1, indicating that supposedly 1 row has been updated. However, it hasn't. The application can, however, read data from the database via select statements.
Does anyone have any idea what's going on here? I even tried tracing SQL Server using ExpressProfiler, and its output seemed to indicate that the update should have occurred:
exec sp_executesql N'UPDATE Match SET TicketsSold=#ticketsSold WHERE MatchId=#matchId',N'#matchId int,#ticketsSold int',#matchId=1,#ticketsSold=1234
go
Yet TicketsSold stays at the same value (123) it was at before, and does not update to 1234. Is there some kind of "silent" read-only mode SQL Server 2008 Express could be running in? I'm baffled as to why the database isn't being updated.
By the way, this is a proper SQL Server database I created in SSMS, not some attached MDF file that resides in the same directory as my web application. The database is not set to "read-only" in database options, and I'm pretty sure that the user that the web application is logging in as has read/write permission on the MDF file; it is logging in as the same user I am logging in as using SSMS - with integrated Windows security - and I am able to update/insert as that user via SSMS.
Thanks to shf301 in the comments - I was creating a transaction but forgetting to call .Commit before the end of the using block. :-D I put that in and now it works.
I'm just trying to right click then select "Generate Model from database." and I get the error:
Unable to generate the model because of the following exception:
'An error occurred while executing the command definition. See the inner
exception for details.
There is insufficient system memory in
resource pool 'internal' to run this query. '.
Loading metadata from
the database took 00:00:04.0661492. Generating the model took
00:00:01.7104861.
I'm running SQL Server 2008 express, and only trying to generate 10 tables.
It looks like there is a bug in SQL Server and they have fixed it. Try downloading the fix.
Temporary fix :
Go to services.msc and stop and start SQL Server (MSSQLSERVER) and you are good to go!
I'm trying to determine what instances of sql server/sql express I have installed (either manually or programmatically) but all of the examples are telling me to run a SQL query to determine this which assumes I'm already connected to a particular instance.
At a command line:
SQLCMD -L
or
OSQL -L
(Note: must be a capital L)
This will list all the sql servers installed on your network. There are configuration options you can set to prevent a SQL Server from showing in the list. To do this...
At command line:
svrnetcn
In the enabled protocols list, select 'TCP/IP', then click properties. There is a check box for 'Hide server'.
You could query this registry value to get the SQL version directly:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90\Tools\ClientSetup\CurrentVersion
Alternatively you can query your instance name and then use sqlcmd with your instance name that you would like:
To see your instance name:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\Instance Names
Then execute this:
SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')
If you are using C++ you can use this code to get the registry information.
All of the instances installed should show up in the Services Snap-In in the Microsoft Management Console. To get the instance names, go to Start | Run | type Services.msc and look for all entries with "Sql Server (Instance Name)".
-- T-SQL Query to find list of Instances Installed on a machine
DECLARE #GetInstances TABLE
( Value nvarchar(100),
InstanceNames nvarchar(100),
Data nvarchar(100))
Insert into #GetInstances
EXECUTE xp_regread
#rootkey = 'HKEY_LOCAL_MACHINE',
#key = 'SOFTWARE\Microsoft\Microsoft SQL Server',
#value_name = 'InstalledInstances'
Select InstanceNames from #GetInstances
I know this thread is a bit old, but I came across this thread before I found the answer I was looking for and thought I'd share. If you are using SQLExpress (or localdb) there is a simpler way to find your instance names.
At a command line type:
> sqllocaldb i
This will list the instance names you have installed locally. So your full server name should include (localdb)\ in front of the instance name to connect. Also, sqllocaldb allows you to create new instances or delete them as well as configure them. See: SqlLocalDB Utility.
If you just want to see what's installed on the machine you're currently logged in to, I think the most straightforward manual process is to just open the SQL Server Configuration Manager (from the Start menu), which displays all the SQL Services (and only SQL services) on that hardware (running or not). This assumes SQL Server 2005, or greater; dotnetengineer's recommendation to use the Services Management Console will show you all services, and should always be available (if you're running earlier versions of SQL Server, for example).
If you're looking for a broader discovery process, however, you might consider third party tools such as SQLRecon and SQLPing, which will scan your network and build a report of all SQL Service instances found on any server to which they have access. It's been a while since I've used tools like this, but I was surprised at what they found (namely, a handful of instances that I didn't know existed). YMMV. You might Google for details, but I believe this page has the relevant downloads: http://www.sqlsecurity.com/Tools/FreeTools/tabid/65/Default.aspx
SQL Server permits applications to find SQL Server instances within the current network. The SqlDataSourceEnumerator class exposes this information to the application developer, providing a DataTable containing information about all the visible servers. This returned table contains a list of server instances available on the network that matches the list provided when a user attempts to create a new connection, and expands the drop-down list containing all the available servers on the Connection Properties dialog box. The results displayed are not always complete.
In order to retrieve the table containing information about the available SQL Server instances, you must first retrieve an enumerator, using the shared/static Instance property:
using System.Data.Sql;
class Program
{
static void Main()
{
// Retrieve the enumerator instance and then the data.
SqlDataSourceEnumerator instance =
SqlDataSourceEnumerator.Instance;
System.Data.DataTable table = instance.GetDataSources();
// Display the contents of the table.
DisplayData(table);
Console.WriteLine("Press any key to continue.");
Console.ReadKey();
}
private static void DisplayData(System.Data.DataTable table)
{
foreach (System.Data.DataRow row in table.Rows)
{
foreach (System.Data.DataColumn col in table.Columns)
{
Console.WriteLine("{0} = {1}", col.ColumnName, row[col]);
}
Console.WriteLine("============================");
}
}
}
from msdn http://msdn.microsoft.com/en-us/library/a6t1z9x2(v=vs.80).aspx
One more option would be to run SQLSERVER discovery report..go to installation media of sqlserver and double click setup.exe
and in the next screen,go to tools and click discovery report as shown below
This will show you all the instances present along with entire features..below is a snapshot on my pc
SQL Server Browser Service http://msdn.microsoft.com/en-us/library/ms181087.aspx
This query should get you the server name and instance name :
SELECT ##SERVERNAME, ##SERVICENAME
If you are interested in determining this in a script, you can try the following:
sc \\server_name query | grep MSSQL
Note: grep is part of gnuwin32 tools
From Windows command-line, type:
SC \\server_name query | find /I "SQL Server ("
Where "server_name" is the name of any remote server on which you wish to display the SQL instances.
This requires enough permissions of course.
I had the same problem. The "osql -L" command displayed only a list of servers but without instance names (only the instance of my local SQL Sever was displayed).
With Wireshark, sqlbrowser.exe (which can by found in the shared folder of your SQL installation) I found a solution for my problem.
The local instance is resolved by registry entry. The remote instances are resolved by UDP broadcast (port 1434) and SMB.
Use "sqlbrowser.exe -c" to list the requests.
My configuration uses 1 physical and 3 virtual network adapters.
If I used the "osql -L" command the sqlbrowser displayed a request from one of the virtual adaptors (which is in another network segment), instead of the physical one.
osql selects the adpater by its metric. You can see the metric with command "route print".
For my configuration the routing table showed a lower metric for teh virtual adapter then for the physical. So I changed the interface metric in the network properties by deselecting automatic metric in the advanced network settings.
osql now uses the physical adapter.
The commands OSQL -L and SQLCMD -L will show you all instances on the network.
If you want to have a list of all instances on the server and doesn't feel like doing scripting or programming, do this:
Start Windows Task Manager
Tick the checkbox "Show processes from all users" or equivalent
Sort the processes by "Image Name"
Locate all sqlsrvr.exe images
The instances should be listed in the "User Name" column as MSSQL$INSTANCE_NAME.
And I went from thinking the poor server was running 63 instances to realizing it was running three (out of which one was behaving like a total bully with the CPU load...)
I just installed Sql server 2008, but i was unable to connect to any database instances.
The commands #G Mastros posted listed no active instances.
So i looked in services and found that the SQL server agent was disabled. I fixed it by setting it to automatic and then starting it.
I had this same issue when I was assessing 100+ servers, I had a script written in C# to browse the service names consist of SQL. When instances installed on the server, SQL Server adds a service for each instance with service name. It may vary for different versions like 2000 to 2008 but for sure there is a service with instance name.
I take the service name and obtain instance name from the service name. Here is the sample code used with WMI Query Result:
if (ServiceData.DisplayName == "MSSQLSERVER" || ServiceData.DisplayName == "SQL Server (MSSQLSERVER)")
{
InstanceData.Name = "DEFAULT";
InstanceData.ConnectionName = CurrentMachine.Name;
CurrentMachine.ListOfInstances.Add(InstanceData);
}
else
if (ServiceData.DisplayName.Contains("SQL Server (") == true)
{
InstanceData.Name = ServiceData.DisplayName.Substring(
ServiceData.DisplayName.IndexOf("(") + 1,
ServiceData.DisplayName.IndexOf(")") - ServiceData.DisplayName.IndexOf("(") - 1
);
InstanceData.ConnectionName = CurrentMachine.Name + "\\" + InstanceData.Name;
CurrentMachine.ListOfInstances.Add(InstanceData);
}
else
if (ServiceData.DisplayName.Contains("MSSQL$") == true)
{
InstanceData.Name = ServiceData.DisplayName.Substring(
ServiceData.DisplayName.IndexOf("$") + 1,
ServiceData.DisplayName.Length - ServiceData.DisplayName.IndexOf("$") - 1
);
InstanceData.ConnectionName = CurrentMachine.Name + "\\" + InstanceData.Name;
CurrentMachine.ListOfInstances.Add(InstanceData);
}
Will get the instances of SQL server
reg query "HKLM\Software\Microsoft\Microsoft SQL Server\Instance Names\SQL"
or Use
SQLCMD -L
Here is a simple method:
go to
Start then
Programs then
Microsoft SQL Server 2005 then
Configuration Tools then
SQL Server Configuration Manager then
SQL Server 2005 Network Configuration then
Here you can locate all the instance installed onto your machine.
I know its an old post but I found a nice solution with PoweShell where you can find SQL instances installed on local or a remote machine including the version and also be extend get other properties.
$MachineName = ‘.’ # Default local computer Replace . with server name for a remote computer
$reg = [Microsoft.Win32.RegistryKey]::OpenRemoteBaseKey(‘LocalMachine’, $MachineName)
$regKey= $reg.OpenSubKey("SOFTWARE\\Microsoft\\Microsoft SQL Server\\Instance Names\\SQL" )
$values = $regkey.GetValueNames()
$values | ForEach-Object {$value = $_ ; $inst = $regKey.GetValue($value);
$path = "SOFTWARE\\Microsoft\\Microsoft SQL Server\\"+$inst+"\\MSSQLServer\\"+"CurrentVersion";
#write-host $path;
$version = $reg.OpenSubKey($path).GetValue("CurrentVersion");
write-host "Instance" $value;
write-host "Version" $version}
If your within SSMS you might find it easier to use:
SELECT ##Version