Exporting azure database in powershell with New-AzSqlDatabaseExport does not always return the OperationStatusLink, resulting in an exception - database

I am writing a powershell script to export an Azure database to a bacpac file using the New-AzSqlDatabaseExport command (following the documentation here.
When I run the powershell script, the results I get are inconsistent. When I open a new powershell window and run the export database script, everything runs as expected, and I get back an OperationStatusLink, so I can check the progress of the export as it progresses. However, once the export completes, if I try running the powershell script a 2nd time within the same window, the export will not return the OperationStatusLink. This will cause Get-AzSqlDatabaseImportExportStatus to fail with the following exception: Cannot bind argument to parameter 'OperationStatusLink' because it is null.
Below are the steps to reproduce, as well as a snippet of powershell script. Any suggestions as to what I could possibly try to ensure that New-AzSqlDatabaseExport always returns an OperationStatusLink would be greatly appreciated.
Steps to Reproduce:
Open powershell window
Log in to Azure
Run script to export database to bacpac
Expected Result: Export is successful and OperationStatusLink is provided
Actual Result: Export is successful and OperationStatusLink is provided
Run script to export database to bacpac
Expected Result: Export is successful and OperationStatusLink is provided
Actual Result: Export is successful and OperationStatusLink is not provided
Powershell script:
Connect-AzAccount
Select-AzSubscription -SubscriptionName 'subscription name'
BackupAzureDatabase.ps1 `
-DatabaseName "testDB" `
-ResourceGroupName "group1" `
-ServerName "testserver" `
-serverAdmin "admin" `
-serverPassword "********" `
BackupAzureDatabase.ps1:
Param(
[string][Parameter(Mandatory=$true)] $DatabaseName,
[string][Parameter(Mandatory=$true)] $ResourceGroupName,
[string][Parameter(Mandatory=$true)] $ServerName,
[string][Parameter(Mandatory=$true)] $ServerAdmin,
[string][Parameter(Mandatory=$true)] $ServerPassword,
)
Process{
# some code to get the storage info and credentials
$ExportRequest = New-AzSqlDatabaseExport `
-ResourceGroupName $ResourceGroupName `
-ServerName $ServerName `
-DatabaseName $DatabaseName `
-StorageKeytype $StorageKeytype `
-StorageKey $PrimaryKey `
-StorageUri $BacpacUri `
-AdministratorLogin $Creds.UserName `
-AdministratorLoginPassword $Creds.Password
$ExportStatus = Get-AzSqlDatabaseImportExportStatus `
-OperationStatusLink $ExportRequest.OperationStatusLink
# Get-AzSqlDatabaseImportExportStatus throws an exception, since OperationStatusLink is empty/null most of the time
}

This seems to be a regression in Az.Sql module introduced in 2.10.0 and it still active with the current version (2.11.0)
Symptoms:
when initiating export operation the following exception raised: New-AzSqlDatabaseExport: Missing the required 'networkIsolation' parameters for ImportExport operation.
The issue:
this should be optional parameter, and the parameter name is incorrect, it should be -UseNetworkIsolation instead.
Workaround:
target your script to older version of the module, 2.9.1 seems to be OK.
Long term solution:
The fix already committed, it should be available in the next releases on the module.
Source of information:
https://github.com/Azure/azure-powershell/issues/13097
Update 2020-11-04
The recent version of the module already contains the fix.
(2.11.1)
https://www.powershellgallery.com/packages/Az/5.0.0

Related

AD error while executing EXE

Using a PowerShell script (part of script Invoke-Command -ComputerName $n -FilePath $filepath -Credential ($Cred) -EnableNetworkAccess) to perform activities like unzip files and run EXE on remote VM.
I'm able to perform activities like extracting and calling EXE first part of the exe activities like DB creation… However, part of the EXE like installing frontend is not working where as exe might validating user in admin group.
Please note that user has admin group/access in both the machines.
Error message:
</Message>
<StackTrace>
at System.DirectoryServices.AccountManagement.PrincipalContext.DoLDAPDirectoryInit()
at System.DirectoryServices.AccountManagement.PrincipalContext.DoDomainInit()
at System.DirectoryServices.AccountManagement.PrincipalContext.Initialize()
at System.DirectoryServices.AccountManagement.PrincipalContext.get_QueryCtx()
at System.DirectoryServices.AccountManagement.Principal.FindByIdentityWithTypeHelper(PrincipalContext context, Type principalType, Nullable`1 identityType, String identityValue, DateTime refDate)
at System.DirectoryServices.AccountManagement.GroupPrincipal.FindByIdentity(PrincipalContext context, String identityValue)
at DataLabsXC.ManagementLib.Collections.XCUsers.ListUsers(String DomainName, String GroupName, String orgUnits)
at DataLabsXC.ManagementLib.XCSqlManager.AddGroups(XCSecGroups SecGroups, String DomainName, String ADAdmin, String ADPassword, Boolean LdapProvider, String OrganizationalUnits)
at DataLabsXC.Management.Setup.FrontEndInstaller.Execute()
at DataLabsXC.Management.BatchProcess.BatchProcessor.Install()
</StackTrace>
<CallingMethod>An operations error occurred.
The same script is working fine when executed directly on the targegeted VM instead of being calling from remote machine.

RM + DSC to node in untrusted domain

So I mention the untrusted domain aspect because I went through all the hoops around credential delegation and trusted hosts lists etc to allow me to successfully push a DSC configuration from my RM server to a target node (not using RM, just native DSC). I get that bit and it works, great.
Now when I use those same scripts in RM (with some minor edits for the format expected by RM), RM reports a successful deploy but all that has happened is the components bits have been copied to the target node to the default location for $applicationPathRoot (C:\Windows\DtlDownloads), there is no real evidence of an attempt to apply a mof file.
My RM server and target nodes are in different domains with no trust. Both servers are W2k8R2 (+ WMF4 of course). I'm running with Update 4 of RM server and client.
Here are the DSC scripts I'm running in RM:
CopyDSCResources.ps1
Configuration CopyDSCResource
{
param (
[Parameter(Mandatory=$false)]
[ValidateNotNullOrEmpty()]
[String] $ModulePath = "$env:ProgramFiles\WindowsPowershell\Modules")
#[PSCredential] $credential = get-credential
Node VCTSCFDSMWEB01
{
File DeployWebDeployResource
{
Ensure = "Present"
SourcePath = "C:\test.txt"
DestinationPath = "D:\temp"
Force = $true
Type = "File"
}
}
}
CopyDSCResource -ConfigurationData $configData -Verbose
# test outside of RM
#CopyDSCResource -ConfigurationData CopyDSCResource.ConfigData.psd1
#Start-DscConfiguration -Path .\CopyDSCResource -Credential $credential -Verbose -Wait
CopyDSCResource.ConfigData.psd1
##{
$configData = #{
AllNodes = #(
#{
NodeName = "*"
PSDscAllowPlainTextPassword = $true
},
#{
NodeName = "VCTSCFDSWEB01.rlg.test"
Role = "WebServer"
}
)
}
I'm afraid I cant seem to upload screenshots from my current location but in terms of RM, I have a vNext environment with a single server linked, a vNext release path with a single 'Dev' stage and a vNext release template with a single 'Deploy PS/DSC' action. The configuration of the action is:
ServerName - VCTSCFDSMWEB01
ComponentName - COpyDSCResource vNext
PSScriptPath - copydscresources.ps1
PSConfigurationPath - copydscresource.configdata.psd1
UseCredSSP - true
When I run a new release, the deploy stage reports success and when I view the Deployment log files I get the following:
Upload components - Successfully uploaded to the normalized store.
Deploy Using PS/DSC - Copying recursively from \vcxxxxtfs03\Drops\CorrespondenceCI\CorrespondenceCI20150114.1\Scripts to C:\Windows\DtlDownloads\CopyDSCResource vNext succeeded.
Finally the DSC event log has the following:
Job {CD3BE350-4072-4C8B-835F-4B4D1C46D65D} :
Configuration is sent from computer NULL by user sid S-1-5-18.
This compares markedly to the same event log entry when run outside of RM:
Job {34F78498-CF18-4F2A-9874-EB54FDA2D990} :
Configuration is sent from computer VCXXXXTFS01 by user sid S-1-5-21-1034805355-1149422947-1317505720-10867.
Any pointers appreciated
It would be good if I could see evidence of a mof file being created on the RM server for example, anybody know where I can find this??
Turns out the crucial element was that my DSC script had to use an environment variable for naming the node. So:
Node $env:COMPUTERNAME
No idea why but it works!

How to run a scheduled task to stop and start SSRS service with elevated permissions?

I have this SSRS latency issues on my site. So I have googled it and found out that it is the common issues for so many people. Here it is:
I have created a powershell script as follows:
Stop-Service "SQL Server Reporting Services (MSSQLSERVER)"
Start-Service "SQL Server Reporting Services (MSSQLSERVER)"
$wc = New-Object system.net.webClient
$cred = [System.Net.CredentialCache]::DefaultNetworkCredentials
$wc.Credentials = $cred
$src = $wc.DownloadString("http://example.com/Reports/Pages/Folder.aspx")
When i run this script from poweshell cmd it is throwing me an error says cannot open/access sql report server service. It seems like permissions issue. Then I came with this online solution, which invokes/elevates admin permissions to run the script to that perticular user.
function Invoke-Admin() {
param ( [string]$program = $(throw "Please specify a program" ),
[string]$argumentString = "",
[switch]$waitForExit )
$psi = new-object "Diagnostics.ProcessStartInfo"
$psi.FileName = $program
$psi.Arguments = $argumentString
$psi.Verb = "runas"
$proc = [Diagnostics.Process]::Start($psi)
if ( $waitForExit ) {
$proc.WaitForExit();
}
}
But I dont know how to run this function before running that script. Please suggest. I have added this function also to the same script file and added function-Admin() call at the top of the script to to execute this function before running the script as follows:
function-Admin()
Stop-Service "SQL Server Reporting Services (MSSQLSERVER)"
Start-Service "SQL Server Reporting Services (MSSQLSERVER)"
$wc = New-Object system.net.webClient
$cred = [System.Net.CredentialCache]::DefaultNetworkCredentials
$wc.Credentials = $cred
$src = $wc.DownloadString("http://example.com/Reports/Pages/Folder.aspx")
But is throwing following error:
Please specify a program
At C:\SSRS_Script\SSRSScript.ps1:3 char:39
+ param ( [string]$program = $(throw <<<< "Please specify a program" ),
+ CategoryInfo : OperationStopped: (Please specify a program:String) [], RuntimeException
+ FullyQualifiedErrorId : Please specify a program
You are getting that error because the function Invoke-Admin() was designed to have parameters passed for the program you wanted to run with elevated privledges. If you want your powershell script SSRSScript.ps1 to use this Invoke-Admin() you could convert it to a standalone script.
Take the code without the function declartion and outer brackets. Save this a file called Invoke-Admin.ps1
param ( [string]$program = $(throw "Please specify a program" ),
[string]$argumentString = "",
[switch]$waitForExit )
$psi = new-object "Diagnostics.ProcessStartInfo"
$psi.FileName = $program
$psi.Arguments = $argumentString
$psi.Verb = "runas"
$proc = [Diagnostics.Process]::Start($psi)
if ( $waitForExit ) {
$proc.WaitForExit();
}
With that created then you could try to elevate your script with the following:
C:\*pathtoscript*\Invoke-Admin.ps1 -program "Powershell.exe" -argumentString "-file C:\SSRS_Script\SSRSScript.ps1"
You should get the elevation prompt at that point and then, once accepted, will run another window with your script using admin rights.
This is by no means the only way to accomplish this goal.
Scheduler
You have this in the title but dont really cover it in the question. Running this as a scheduled task will not work since it requires user input. You could however just make a task with your script as is assuming it works unattended.
General Tab
Run whether user is logged on or not
Run with highest privileges
Action > New...
Action: Start a program Program/script: %SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe
Add arguments: -ExecutionPolicy Unrestricted -NoProfile -File C:\SSRS_Script\SSRSScript.ps1
Start in (optional): %SystemRoot%\system32\WindowsPowerShell\v1.0

invoke-sqlcmd - can I change the ApplicationName?

I'm running queries against SQL servers using invoke-sqlcmd and invoke-sqlcmd2.
Is there a way to change the ApplicationName that it runs as? When I run a profiler trace, I see the queries are run by ".Net SqlClient Data Provider", and I'd like to change that.
Any help greatly appreciated
Okay, futzed with the invoke-sqlcmd2 script by Chad Miller and came up with this:
Line 45, after "datarow" I added a comma, then:
[Parameter(Position=9, Mandatory=$false)] [string]$ApplicationName='Powershell'
Then modified the connection strings (about line 54):
if ($Username)
{ $ConnectionString = "Server={0};Database={1};User ID={2};Password={3};Application Name={5};Trusted_Connection=False;Connect Timeout={4}" -f $ServerInstance,$Database,$Username,$Password,$ConnectionTimeout,$ApplicationName }
else
{ $ConnectionString = "Server={0};Database={1};Integrated Security=True;Application Name={3};Connect Timeout={2}" -f $ServerInstance,$Database,$ConnectionTimeout,$ApplicationName }
The default Appname is now "Powershell", but can be changed by using the -ApplicationName parameter.
Include Application Name=MyAppName; in your connection string, and that will be what shows up in the profiler.

Invoke-SqlCmd QueryTimeout

Does anyone know how to set Invoke-SqlCmd QueryTimeout more than 65535?
Microsoft said that they have fixed it in Denali but we are still using SQL 2008 R2 with latest service packs.
http://connect.microsoft.com/SQLServer/feedback/details/551799/invoke-sqlcmd-querytimeout-0-still-times-out
Basically, we are trying to backup or restore the database using powershell. Some of our databases are very large so it takes more than 65535 to complete the job.
Some suggested that we should use ADO.NET with timeout in powershell. But I wonder if we have any workaround for Invoke-SqlCmd...
Invoke-Sqlcmd -query 'select * from largeDb' -QueryTimeout 0
-QueryTimeout 0 : will make your cmdlet from timing out.
AFAIK By soon, this bug will be resolved.
You can use the below function to customize your own Query time out and session time out limits (while defining the parameters you can give you own desired values)
function Invoke-SqlCommand
{
[CmdletBinding()]
param(
[Parameter(Position=0, Mandatory=$true)] [string]$ServerInstance,
[Parameter(Position=1, Mandatory=$false)] [string]$Database,
[Parameter(Position=2, Mandatory=$false)] [string]$Query,
[Parameter(Position=3, Mandatory=$false)] [string]$Username,
[Parameter(Position=4, Mandatory=$false)] [string]$Password,
[Parameter(Position=5, Mandatory=$false)] [Int32]$QueryTimeout=600,
[Parameter(Position=6, Mandatory=$false)] [Int32]$ConnectionTimeout=15,
[Parameter(Position=7, Mandatory=$false)] [ValidateScript({test-path $_})] [string]$InputFile,
[Parameter(Position=8, Mandatory=$false)] [ValidateSet("DataSet", "DataTable", "DataRow")] [string]$As="DataRow"
)
if ($InputFile)
{
$filePath = $(resolve-path $InputFile).path
$Query = [System.IO.File]::ReadAllText("$filePath")
}
$conn=new-object System.Data.SqlClient.SQLConnection
if ($Username)
{ $ConnectionString = "Server={0};Database={1};User ID={2};Password={3};Trusted_Connection=False;Connect Timeout={4}" -f $ServerInstance,$Database,$Username,$Password,$ConnectionTimeout }
else
{ $ConnectionString = "Server={0};Database={1};Integrated Security=True;Connect Timeout={2}" -f $ServerInstance,$Database,$ConnectionTimeout }
$conn.ConnectionString=$ConnectionString
#Following EventHandler is used for PRINT and RAISERROR T-SQL statements. Executed when -Verbose parameter specified by caller
if ($PSBoundParameters.Verbose)
{
$conn.FireInfoMessageEventOnUserErrors=$true
$handler = [System.Data.SqlClient.SqlInfoMessageEventHandler] {Write-Verbose "$($_)"}
$conn.add_InfoMessage($handler)
}
$conn.Open()
$cmd=new-object system.Data.SqlClient.SqlCommand($Query,$conn)
$cmd.CommandTimeout=$QueryTimeout
$ds=New-Object system.Data.DataSet
$da=New-Object system.Data.SqlClient.SqlDataAdapter($cmd)
[void]$da.fill($ds)
$conn.Close()
switch ($As)
{
'DataSet' { Write-Output ($ds) }
'DataTable' { Write-Output ($ds.Tables) }
'DataRow' { Write-Output ($ds.Tables[0]) }
}
}
Hope it HElps.
You can write your own version of Invoke-SqlCmd that directly uses the System.Data.SqlClient object and do anything that you want to with it. There are a bunch of examples of how to do this to be found, including invoke-sqlcmd2, which was specifically written to get around the QueryTimeout bug and is hosted on Microsoft's scripting gallery. If you don't want to deploy such a script, you can just integrate the relevant code directly into your backup script.
Alternatively, you should be able to use SMO to backup the database. IIRC, the querytimeout bug does not affect SMO.

Resources