I have SQL script which I want to execute using azure DevOps pipeline
I found multiple tasks for SQL but can not find any required task where I can pass sql server , database name and login details. Is there any task available for this ?
If there is not any task available and only way to execute is powershell script any sample available script for this ?
You can use PowerShell to execute sql scripts. Example:
Invoke-Sqlcmd -InputFile "$(scriptfilepath)" -ServerInstance $(sqlserver) -Database $(dbname) -Username "$(username)" -Password "$(pwd)" -QueryTimeout 36000 -Verbose
Add custom variables (scriptfilepath, sqlserver, ...) and set values to them.
PowerShell task
Define variables
Invoke-Sqlcmd
You can use Invoke-Sql command like this
$SQLServer = "TestServerOne"
$db3 = "TestDB3"
$qcd = "PRINT 'This is output'"
Invoke-Sqlcmd -ServerInstance $SQLServer -Database $db3 -Query $qcd -Username "User" -Password "Password" -Verbose
Make sure SqlServer module is installed. It works also with Powershell Core
You can also try to use Run SQL Server Scripts Task extension
If you want to do this in Azure Release Pipeline (classic), you can use the '
Azure SQL Database deployment
' block which uses Invoke-Sqlcmd under the hood.
With that, you can configure it to execute an SQL script on a given database under one or your subscriptions or service connections.
Related
I am running into issues where I am trying to run an Invoke-Sqlcmd command to run create table SQL queries. I used different ServerInstances names such as the Synapse workspace name, the Dedicated SQL, Serverless SQL and Development endpoint. I have been testing these scripts in a PowerShell CLI on the Azure Portal before moving them into an ADO pipeline.
The command that is run is as below:
Invoke-sqlcmd -Query 'CREATE TABLE #DUMMYTABLE(USERNAME varchar(50));'
-SeverInstance {endpoints or Synapse Workspace resource name} `
-Database {dedicated SQL Pool name (not master db)}
-Username {Synapse Workspace username}
-Password {Synapse Workspace password}
-QueryTimeout 36000
-Verbose
I got an error code 25 or 35 or even just a general SQL Connection String error. Any support on this would be greatly appreciated. Using the Dedicated endpoint would result in a 'cannot log in username error'.
Cheers
Getting Error because of wrong credentials.
You can get connection string from here:
from this connection string use Servername as ServerInstance and Initial Catalog as Database
Invoke-sqlcmd -Query 'CREATE TABLE DUMMY (USERNAME varchar(50));' -ServerInstance <synapseworkspacename>.sql.azuresynapse.net -Database <Inital catlog name from connection string or dedicated sql pool name> -Username <username> -Password <password> -QueryTimeout 36000 -Verbose
Output
DUMMY Database created successfully.
I am trying to use Analysis Services Cmdlets to process partitions on a Power BI Premium Model. My PowerShell script works fine when run from ISE, the command line, and when scheduled using windows task scheduler; however, when I try to schedule the PowerShell script using a SQL Server 2019 Agent job using a step type of
Operating System (CmdExec)" the following error message is encountered.
Message Executed as user: MyDomain\MyUser. Invoke-ProcessPartition : The
connection string is not valid. At
C:\Users\MyUser\Desktop\PS1\SSAS\wtf.ps1:15 char:11 + $asResult =
Invoke-ProcessPartition -Credential $UserCredential -Server...+
CategoryInfo : NotSpecified: (:) [Invoke-ProcessPartition],
ConnectionException + FullyQualifiedErrorId :
Microsoft.AnalysisServices.ConnectionException,Microsoft.AnalysisServices.PowerShell.Cmd
lets.ProcessPartition.
I have followed the steps in this blog article to setup the job. The same windows user is used in all three run scenarios. The SQL server is my local development SQL server of which the windows user is SA on the SQL Server and Windows Admin. The same machine the SQL Instance is on is being used for successfully executing the other three ways of running the PS scripts, (ISE, Command Line, & Windows Task Scheduler)
If I run the following from the command line on the same machine as the SQL server, my local host, the PowerShell script runs successfully.
PowerShell -File "C\Users\MyUser\Desktop\PS1\SSAS\wtf.ps1"
Below is my PowerShell script modified to be as small as possible to demonstrate issue and of course sensitive information redacted. Thanks in advance for any help, I'm at a loss at how to proceed. I really need this to work from the SQL agent so I don't have to guess when steps that the processing depends on have completed.
$ErrorActionPreference=”Stop”
Import-Module "SqlServer"
$User = "MyUser#MyDomain.com"
$PWord = ConvertTo-SecureString -String "MyPassword" -AsPlainText -Force
$UserCredential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $User, $PWord
$server = "powerbi://api.powerbi.com/v1.0/myorg/MyWorkspace"
$db = "MyModel"
$asResult = Invoke-ProcessPartition -Credential $UserCredential -Server $server -PartitionName "DimDate" -TableName "DimDate" -Database $db -RefreshType "Full"
The step code in my job is:
powershell -File "C:\Users\MyUser\Desktop\PS1\SSAS\wtf.ps1"
UPDATE:
I have discovered that if I target an on-prem SSAS server, the exact same code, with the exception of the server variable, does work using the SQL Agent method, but of course switching to on-prem is not what I desire.
UPDATE 2:
When I right click on a job within SSMS, and click "Start PowerShell" this opens up a SQL Server PowerShell Window. If I try executing the script this way I get the below error message. The account being used does not have MFA enabled.
Unable to obtain authentication token using the credentials provided.
If your Active Directory tenant administrator has configured
Multi-Factor Authentication or if your account is a Microsoft Account,
please remove the user name and password from the connection string,
and then retry. You should then be prompted to enter your credentials.
If I don't pass a credential to the Invoke-ProcessPartition command using the SQL Server PowerShell window, I get prompted for credentials and the call works. Of course I can't use this as a workaround because I need it to run unattended.
I also tried opening PowerShell ISE as the account being used for authentication to the Workspace in the PS script and it also gives the same error as the SQL Server Agent Job does.
The connection string is not valid.
I have found a solution to the issue. The resolution was twofold.
First problem was that when PowerShell was being run from the SQL Server Agent, the Version of the SqlServer module was an older outdated version. I found this out by executing a ps1 file from the SQL Server Agent Job using the following code and looking in the job history results.
Get-Command -module sqlserver invoke*
I tried running
Install-Module sqlserver -AllowClobber -Scope AllUsers
as an admin but it did not update the SqlServer module the SQL agent was running. Instead I created a ps1 file that simply runs
Install-Module sqlserver -AllowClobber -Scope CurrentUser -Force
and called the script using a SQL Agent CMD task and this updated the SqlServer module version. After this I started getting a more useful error message:
Unable to obtain authentication token using the credentials provided.
If your Active Directory tenant administrator has configured
Multi-Factor Authentication or if your account is a Microsoft Account,
please remove the user name and password from the connection string,
and then retry. You should then be prompted to enter your
credentials
With this new error message, I decided to try an alternate way of supplying credentials by utilizing Azure Service Principals. This new method resulted in success from all PowerShell methods including SQL Server Agent Job. The steps implemented are outlined in this Power BI Documentation.
The final PS code looks like this.
$ErrorActionPreference=”Stop”
Import-Module "SqlServer"
$AppId = "AAD_App_Registration_Application_Client_Id"
$TenantId = "AAD_App_Registration_Directory_Tenant_Id"
$AppSecret = "AAD_App_Registration_CertificatesAndSecrets_ClientSecret"
$PWord = ConvertTo-SecureString -String $AppSecret -AsPlainText -Force
$Credential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $AppId, $PWord
Invoke-ProcessPartition -Server "powerbi://api.powerbi.com/v1.0/myorg/MyModel" -PartitionName "DimDate" -TableName "DimDate" -Database "MyModel" -RefreshType "Full" -ServicePrincipal -ApplicationId $AppId -TenantId $TenantId -Credential $Credential
I'd like to have a PowerShell script that uses as input;
-username
-password
-remote servername
And then lists the remote SQL databases. Thing is, it should work from any PC with powershell, even if there are no SQL client/tools used/installed/plugins loaded. Standard .NET framework call are accepted ofcourse.
I know it's possible, because I've seen 'sql-clientless' machines list databases remotely.
The following should work with an installed .NET Framework:
function Get-SqlDatabases
{
[CmdletBinding()]
Param
(
$Server,
$User,
$Password
)
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection "Data Source=$Server;User ID=$User;Password=$Password"
$sqlConnection.Open();
$sqlConnection.GetSchema("Databases") | select -expand database_name
$sqlConnection.Close();
}
Usage:
Get-SqlDatabases -Server 'myServer' -User 'myUser' -Password 'mySecret'
I am putting together some powershell scripts to execute SQL commands against SQL Server and need to specify the windows account to use in the connectionstring. I have tried the following which does not work.
$SQLServer = ".\sqlexpress"
$SQLDBName = "dbname"
$uid ="domain\user
$pwd = "password"
$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
$SqlConnection.ConnectionString = "Server = $SQLServer; Database = $SQLDBName; Integrated Security = false; User ID = $uid; Password = $pwd;"
$SqlCmd.Connection = $SqlConnection
$SqlConnection.Open();
.... Execute query...
I get the following error.
Exception calling "Open" with "0" argument(s): "Login failed for user
'Domain\user'.
I was lead to understand that if you specify false on integrated security then is would take the username and password and connect using the corresponding windows account.
For me this is failing to connect. Am I missing something here ?
Many thanks
Darren
Another way to tackle this would be to use the SQL PowerShell Module (SQLPS) which can be obtained by installing "Management Tools" from the SQL Management Studio Installer. By default, when using Invoke-Sqlcmd it will connect as the windows account that initiated the PowerShell instance. If you wish to run as another windows account other than the one you are signed in with, you would shift + right click PowerShell and choose option to "run as different user".
Once Module has been installed, you can import module by executing:
Import-Module SQLPS
Next you would execute your query like the following:
Invoke-Sqlcmd -ServerInstance "ServerName\Instance" -Database "DatabaseName" -Query "Some Query Here"
While entirely optional, I would recommend adding -Verbose as this will show you output just as you would see in Sql Management Studio.
Hope this helps.
I have the following Powershell command:
$DATABASE="adventureworks"
$SQLSERVER="PerfSQL02"
invoke-sqlcmd -Query "sp_whoisactive" -database $DATABASE -hostname $SQLSERVER
The database name and SQLServer are valid and I'm running as my local user, who is sysadmin on the instance and dbo of the database. Additionally, I can connect through SSMS and run sp_whoisactive quite nicely.
The error I'm receiving is:
Cannot open database "adventureworks" requested by the login. The login failed.
Login failed for user '<myusername>'
I'm sure I'm missing something excruciatingly simple, but it's been awhile since I've used Invoke-sqlcmd, what am I missing that is making it unable to connect?
It looks like I should have used -serverinstance rather than -hostname as the -serverinstance argument passes the server and instance name.