I need your help with PowerShell today. I have a SQL Server instance (on a server named VMDEV-APP11) configured as a Central Management Server (CMS) in which I have registered all my Dev/Test/Prod SQL instances and I want to use PowerShell to read the SQL Server error logs from all my servers. I have a query which retrieves all my SQL Server instances but when I pipe the output of this into a call to ReadErrorLog, I get an error.
This is the code that retrieves my list of the SQL Server instances which are registered on my CMS (note that I exclude my SQL Server 2000 instances):
Set-Location D:\MSSQL11.MSSQLSERVER\CMS
# Define functions to query SQL Server and write data to a SQL table
. ./invoke-sqlcmd2.ps1
. ./write-datatable.ps1
Invoke-sqlcmd2 -ServerInstance "VMDEV-APP11" -Database dba -Query "select s.server_name from msdb.dbo.sysmanagement_shared_registered_servers s, msdb.dbo.sysmanagement_shared_server_groups g where s.server_group_id = g.server_group_id and g.name not like '2000%'"
server_name
-----------
INF-SRV14
VMDEV-APP15
NEX-SRV48
...
And this is what I thought would work and the error I actually get:
Invoke-sqlcmd2 -ServerInstance "VMDEV-APP11" -Database dba -Query "select s.server_name from msdb.dbo.sysmanagement_shared_registered_servers s, msdb.dbo.sysmanagement_shared_server_groups g where s.server_group_id = g.server_group_id and g.name not like '2000%'" | foreach-object { $_.server_name.ReadErrorLog() }
Error:
Method invocation failed because [System.String] does not contain a method named 'ReadErrorLog'.
At D:\MSSQL11.MSSQLSERVER\CMS\db_errorlog.ps1:38 char:284
+ ... reach-object { $_.server_name.ReadErrorLog() }
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : MethodNotFound
I suspect I need to convert the string returned by my query to another type of object (a server name perhaps?) in order to get the ReadErrorLog() call to work but I don't know how to do that.
Any suggestions?
Any help would be greatly appreciated.
Ken
You're trying to invoke the ReadErrorLog() method on a DataRow column value, which is a string.
What you need to do is something like this:
Import-Module sqlps -DisableNameChecking
Invoke-sqlcmd2 -ServerInstance "VMDEV-APP11" -Database dba -Query "select s.server_name from msdb.dbo.sysmanagement_shared_registered_servers s, msdb.dbo.sysmanagement_shared_server_groups g where s.server_group_id = g.server_group_id and g.name not like '2000%'" | `
foreach-object {
$server = $_.server_name
$logs = (get-item SQLSERVER:\sql\$server\default).ReadErrorLog()
# $logs is a DataTable so you can iterate the rows however you wish
}
This assuming all your servers are default instances, otherwise you may need to fiddle around a bit more to target specific instances.
The code I'm running on my SQL Central Management server is:
$logs = 0..6 | % { (get-item SQLSERVER:\sql\<servername>\default).ReadErrorLog($_) }
Originally, I got this to work by adding the service account that ran the PowerShell code to the local Administrators Windows group AND creating a SQL login with sysadmin privileges on the remote SQL server host. Not sursprisingly, my security officer had an issue with this. My initial attempts to reduce access resulted i the following error:
WARNING: Could not obtain SQL Server Service information. An attempt to connect
to WMI on 'NEX-SRV1' failed with the following error: SQL Server WMI provider
is not available on NEX-SRV1. --> Invalid namespace
After quite a bit of fooling around, I have what I think is the minimum security needed to read the SQL error logs. Perform the following grants on the remote server running the SQL Server database:
Local Windows Group
Add the service account to the "Distributed COM Users" group
WMIMgmt.msc
Add the service account to each of the following branches with all security options EXCEPT "Edit Security"
Root > cimv2
Root > cimv2 > ms_409
Root > Microsoft > SQLServer > ComputerManagement
SQL Server
Create a SQL login for the service account and add it to the "Security Admin" role.
After I had made these changes, I have the ability to monitor SQL Error logs from a central location without having to grant crazy levels of access to the service account.
Related
I am trying to run SQL Server DB index and statistics maintenance activating a stored procedure called AzureSQLMaintenance with a PowerShell Runbook in Automation Account.
Since I don't want to use standard SQL Server authentication, I am trying to use Managed Identities.
Online I found some Microsoft documentation getting quite close to the point on Microsoft Tech Community here and here, but both the threads are missing some pieces. A very good clue clue was given to me by this blog post but it was missing Managed Identity Authentication
I finally managed, after a couple of days of tests, to make it work, so I'll write the whole process down in case anybody will need to do the same:
Under Account Settings > Identity System Assigned Managed Identity must be set to On, and we'll need the Object (principal) Id, so remember to mark it down
In my Azure SQL Database Server, under Settings > Azure Active Directory, we'll need to check the value of the Azure Active Directory admin. In my case, this is a group
In Azure Active Directory, edit the group individuated on the previous step and add the Object (principal) Id obtained at the step 1 as a member of the group
A Powershell Runbook in Automation Account needs to be created
The powershell Runbook Code will need to look something like
Write-Output "Run started"
# Instantiate the connection to the SQL Database
$sqlConnection = new-object System.Data.SqlClient.SqlConnection
# Connect to the the account used the Managed Identity
Connect-AzAccount -Identity
# Get a Token
$token = (Get-AzAccessToken -ResourceUrl https://database.windows.net ).Token
# Initialize the connection String
$sqlConnection.ConnectionString = "Data Source=db-server-name.database.windows.net;Initial Catalog=<db-name>;Connect Timeout=60"
# Set the Token to be used by the connection
$sqlConnection.AccessToken = $token
# Open the connection
$sqlConnection.Open()
Write-Output "Azure SQL database connection opened"
# Define the SQL command to run
$sqlCommand = new-object System.Data.SqlClient.SqlCommand
# Allow Long Executions
$sqlCommand.CommandTimeout = 0
# Associate the created connection
$sqlCommand.Connection = $sqlConnection
Write-Output "Issuing command to run stored procedure"
# Execute the SQL command
$sqlCommand.CommandText= 'exec [dbo].[AzureSQLMaintenance] #parameter1 = ''param1Value'', #parameter2 = ''param2Value'' '
$result = $sqlCommand.ExecuteNonQuery()
Write-Output "Stored procedure execution completed"
# Close the SQL connection
$sqlConnection.Close()
Write-Output "Run completed"
At this point, run a test on the Runbook: in my case it worked perfectly, even if it took a while (that's the reason for the parameter $sqlCommand.CommandTimeout = 0)
In SSMS I tried adding encryption to columns. If I do this in the dialog box, the process works and the field is encrypted. If I try generating a PowerShell script to set the encryption, after many error messages I've finally got the code to execute without error but nothing happens in the database. This is the generated script with all the defaults and standard things and I'm running PowerShell as an administrator.
Import-Module SqlServer
# Set up connection and database SMO objects
$sqlConnectionString = "Data Source=MyPC\SQLEXPRESS;Initial Catalog=MyDatabase;Integrated Security=True;MultipleActiveResultSets=False;Connect Timeout=30;Encrypt=False;TrustServerCertificate=False;Packet Size=4096;Application Name=`"Microsoft SQL Server Management Studio`""
$smoDatabase = Get-SqlDatabase -ConnectionString $sqlConnectionString
# If your encryption changes involve keys in Azure Key Vault, uncomment one of the lines below in order to authenticate:
# * Prompt for a username and password:
#Add-SqlAzureAuthenticationContext -Interactive
# * Enter a Client ID, Secret, and Tenant ID:
#Add-SqlAzureAuthenticationContext -ClientID '<Client ID>' -Secret '<Secret>' -Tenant '<Tenant ID>'
# Change encryption schema
$encryptionChanges = #()
# Add changes for table [dbo].[MyTable]
$encryptionChanges += New-SqlColumnEncryptionSettings -ColumnName dbo.MyTable.MyField -EncryptionType Randomized -EncryptionKey "CEK_Auto1"
Set-SqlColumnEncryption -ColumnEncryptionSettings $encryptionChanges -InputObject $smoDatabase
Any ideas why the code runs but doesn't encrypt the columns?
https://learn.microsoft.com/en-us/sql/powershell/download-sql-server-ps-module?view=sql-server-2017
https://learn.microsoft.com/en-us/sql/relational-databases/security/encryption/configure-column-encryption-using-powershell?view=sql-server-ver15
Following the instructions above with some modifications
Run PowerShell as admin
To give permissions in this session for the file to be executed
This should give correct permission
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
To install SQL Server modules so it PowerShell can access it
Install-Module -Name SqlServer -AllowClobber
View available commands in SqlServer module
Get-Command -Module SqlServer
If New-SqlColumnEncryptionSettings isn't visible, copy files from
sqlserver.21.1.18218.nupkg.zip
downloaded from
https://www.powershellgallery.com/packages/SqlServer/21.1.18218
to
C:\Program Files\WindowsPowerShell\Modules\SqlServer\21.1.18218
Then run the powershell script using
.\test.ps1
I have just restored a .bacpac file into a local SQL server instance (64b v12.0.4213), the backup is from an azure sql instance.
It failed a few times with an OOM exception. I switched off everything on my machine and by the end of the restore the SQL server service instance was consuming 13GB of memory from a 700MB file!
The restore luckily finished, but it seems the memory is not being freed up/garbage collected. It's still sitting at 12GB as I write this.
Is it a known issue? Is there any way I can restore a .bacpac and select a table to ignore? You can to do this with a normal data restore, the most offensive table was a dbo.[Logs] table, obvs.
I had the same issue; amending the memory available to the server had no impact.
For me the resolution was to use the command line (PowerShell) to perform the import.
[string]$myBacpac = 'c:\temp\myBacpac123.bacpac'
[string]$connectionString = 'Data Source=.;Initial Catalog=MyNewCatalog; Integrated Security=true;'
[string]$action = 'Import'
[string[]]$commandParameters = #(
"/Action:`"$action`""
"/SourceFile:`"$myBacpac`""
"/TargetConnectionString:`"$connectionString`""
)
[string]$LatestSqlPackage = Get-Item 'C:\*\Microsoft SQL Server\*\DAC\bin\sqlpackage.exe' | %{get-command $_}| sort version -Descending | select -ExpandProperty source -First 1
if ($LatestSqlPackage) {
Write-Verbose "Found: $LatestSqlPackage"
& $LatestSqlPackage $commandParameters
} else {
Write-Error "Could not find SqlPackage.exe"
}
On my first attempt I received an error regarding an unsupported model version:
Importing to database 'MyNewCatalog' on server '.'. Creating deployment plan
Initializing deployment SqlPackage.exe : * Error importing
database:Could not read schema model header information from package.
At line:1 char:1
+ & $sqlPackage /Action:Import /SourceFile:"c:\temp\myBacpac123.bacpac" /T ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (* Error impor...n from package.:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError The model version '3.5' is not supported.
For that error I followed the guidance here: https://stackoverflow.com/a/40541210/361842; i.e. installed Microsoft SQL Server Data-Tier Application Framework (16.4). On rerunning all was successful.
To configure SQL Server's use of memory, open SQL Server Management Studio, connect to the server, right-click on the server in the Object Explorer window, click properties, and then click on the Memory tab of the Server Properties window.
As for the bacpac, you can't select which tables to restore during an import operation, but you can select which tables are exported. You can use SqlPackage.exe's export command with the /p:TableData parameter to specify which tables should be included in the bacpac. There's unfortunately no way to just specify which tables should be excluded. =^/
SqlPackage.exe documentation is available here: https://msdn.microsoft.com/en-us/hh550080(v=vs.103).aspx
Neither of the other answers worked for me, what did work was closing and restarting SSMS. This sounds like a silly suggestion, but I'd previously been running some large queries which must've caused memory issues.
Is there an equivalent of ping for checking connectivity to SQL Server?
I'm finding our BizTalk Admin Console that during some long operations, e.g. importing a big bindings file, the "connection" is being lost, i.e. the red box appears on the console. Eventually connectivity comes back. The SQL Server is on a different machine from BizTalk.
Also saw an issue where connection to SSO db was lost for a minute or so ... worse, this was production environment!
SQL DBA has checked and SQL is fine, showing no network issues ...
I can do a ping -t to see if anything happens to the connection between the two machines, but is there an equivalent function to check ongoing connectivity to SQL Server itself?
And if there is such a function, is there someway to automate its checking so I can have it flag any occurance of disconnect ... just sending an email to ops would be good first step
You can always use PowerShell
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
$query = "SELECT top 1* from bts_application"
Invoke-Sqlcmd -Query $query -ServerInstance '.' -Database 'BizTalkMgmtDb'
I am using a script that loads the following SQL Server 2008 R2 powershell plugins
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
I then user invoke-sql like this:
Invoke-Sqlcmd -Query "select * from table" -ServerInstance xyz -Database abc -username xxxxxx -password yyyyyyy
I am using method to run a number of upgrade scripts on our databases. I was quite happily using this in our dev\test environments but then we I tried it in production and it turns out we have a difference in server configurations. On our prod servers named pipes are disabled for security reasons (apparently worm attacks) and our DBA's don't want to enable.
This is the error I get and research says it is a named pipes problem - starts working when I enable them too.
INFO ERROR: Invoke-Sqlcmd : A connection was successfully
established with the server, but then an error occurred during the
login process. (provider: Shared Memory Provider, error: 0 - No
process is on the other end of the pipe.)
Does anyone know if there is some way to switch my script so that it does not require named pipes? Or is this the built in connection method for invoke-sqlcmd and I need to change tack (if so any suggestions).
Similar to Surreal's response to use LPC (local shared memory), for TCP/IP instead of named pipes you can also specify -ServerInstance tcp:foodb
This is an educated guess. But here goes:
I think you have to "override the default" by using the registry.
http://support.microsoft.com/kb/229929
Now, the easiest way to do this (IIRC) is to go through your
Control Panel / ODBC Data Source / System DSN.
Add a "Sql Server". (Not the native client ones).
The most important button is the "Client Configuration" where you can pick named-pipes or tcp/ip.
Try out the DSN method, and after completing the wizard, look at the registry entries under
HKEY_LOCAL_MACHINE\Software\Microsoft\MSSQLServer\Client\ConnectTo
.........
You might check out this:
http://sev17.com/2012/11/05/cloning-sql-servers-to-a-test-environment/
Look for this code.
sqlcmd -S myCMServerInstance -d msdb -Q $query -h -1 -W |
foreach { Set-ItemProperty -Path 'HKLM:SOFTWAREMicrosoftMSSQLServerClientConnectTo' -Name $($_ -replace 'TEST') -Value "DBMSSOCN,$_" }
}
You can change the connection method using prefixes to the instance name as for sqlcmd. With SQL Server 2012 and Powershell 4, this works for me:
Invoke-Sqlcmd -Query $sqlQuery -serverinstance "lpc:localhost" -Database "myDatabase"