In my SSDT project I have a post deployment script where I include a script file.
:r .\Data\Data.Content.sql
The file Data.Content.sql is a dump of the database (insert statements) and it contains content like 'var $sameHeightDivs = $(''.product-tile-region'');'. The database contains JQuery scripts. So I receive the following errors:
SQL72008: Variable document is not defined.
or
72006: Fatal scripting error: Incorrect syntax was encountered while parsing '$(''
I found that you can disable 'disable
variable substitution' with the argument -x.
But is there a way to define this somewhere? (post-deployment script? project setting?)
Or is there another way to solve this problem?
FYI: to create the dump I use Microsoft.SqlServer.Management.Smo.Scripter.
Kind regards,
bob
I posted the same question in the SQL Server Data Tools forum where someone came with a workaround.
After the script generation I do a search and replace for the $ char.
function SearchAndReplace($file) {
(Get-Content $file) |
Foreach-Object {$_ -replace "\$\(", "' + CHAR(36) + '("} |
Set-Content $file
}
I included the '(' to be sure to limit the scope (to JQuery selectors).
Related
I've setup a SQL Server 2016 agent job that has 2 steps that is executed by the same service account, Step 1 executes an SSIS package that writes 2 files out to a share. Step 2 executes embedded powershell code that combines these 2 files into a single file and creates a file on that same share where the 2 files were created by SSIS. When I execute the job, Step 1, the SSIS package completes successfully and creates the 2 files on this share. But when Step 2 executes I get the error below. Just a point of info, when I run this script locally from my windows 10 box, I have no issues.
I guess my question is, if SSIS can execute successfully with the same service account can this be a permissions issue for powershell scripts? I'm not a server admin so if this is a stupid questions please forgive my ignorance. Any help/direction anyone can provide would be appreciated.
Message
Executed as user: XXX\sqlsvc02. The job script encountered the following errors. These errors did not stop the script: A job step received an error at line 12 in a PowerShell script. The corresponding line is 'Get-Content $pathCounts, $pathDetails | Set-Content $outFile'. Correct the script and reschedule the job. The error information returned by PowerShell is: 'Cannot find path '\xxx\shared\MonthlyReports\BoundPoliciesAddedAtFaultEndorsementWithin10Days_Counts.csv' because it does not exist. ' A job step received an error at line 12 in a PowerShell script. The corresponding line is 'Get-Content $pathCounts, $pathDetails | Set-Content $outFile'. Correct the script and reschedule the job. The error information returned by PowerShell is: 'Cannot find path '\xxx\shared\MonthlyReports\BoundPoliciesAddedAtFaultEndorsementWithin10Days_Details.csv' because it does not exist. '. Process Exit Code 0. The step succeeded.
Here is my powershell script:
# Define variables
$path = '\\xxx\shared\MonthlyReports'
$pathCounts = Join-Path -Path $path -ChildPath 'BoundPoliciesAddedAtFaultEndorsementWithin10Days_Counts.csv'
$pathDetails = Join-Path -Path $path -ChildPath 'BoundPoliciesAddedAtFaultEndorsementWithin10Days_Details.csv'
$date = Get-Date
$dateStr = $date.ToString("yyyyMM")
# Define combined file name
$outFile = Join-Path $path -ChildPath("BoundPoliciesAddedAtFaultEndorsementsWithin10Days_" + $dateStr + ".csv")
# Execute combine files code
Get-Content $pathCounts, $pathDetails | Set-Content $outFile
Here is an image of what Step #2 looks like:
I have a Database project in Visual Studio that I am attempting to deploy automatically to a test environment nightly. To accomplish this I am using TFS which leverages a PowerShell script to run "SqlPackage.exe" to deploy any changes that have occurred during the day.
Some of my procs contain logic that is run inside of a script that is part of a agent job step and contains the following code(In dynamic SQL):
$(ESCAPE_SQUOTE(JOBID))
When deploying changes that affect this proc, I get the following issue:
SQL Execution error: A fatal error occurred. Incorrect syntax was
encountered while $(ESCAPE_SQUOTE( was being parsed.
This is a known issue, it appears as though that is not supported. It appears to be a function of the "SQLCmd" command misinterpreting the $( characters as a variable:
"override the value of a SQL command (sqlcmd) variable used during a
publish action."
So how do I get around this? It seems to be a major limitation of "sqlcmd" that you can't disable variables, I don't see that parameter that supports that...
Update 1
Seems as through you can disable variable substitution from "sqlcmd" by feeding it the "-x" argument(Source, Documentation):
-x (disable variable substitution)
Still don't see a way to do this from "SqlPackage.exe" though.
It seems that sqlcmd looks for the $( as a token, so separating those two characters is good enough. You can do this with a dynamic query that does nothing more than break the query into two strings:
DECLARE #query nvarchar(256) = N'... $' + N'(ESCAPE_SQUOTE(JOBID)) ...';
EXEC sp_executesql #query
One way to get around this is to refactor the "$(ESCAPE_SQUOTE(JOBID))" string into a scalar function, then setup a PowerShell script to directly invoke the "Sqlcmd" command with the "-x" parameter to "Create/Alter" said function before running "SqlPackage.exe".
Looks something like this in PowerShell:
$sql = #"
USE $database
GO
CREATE FUNCTION [dbo].[GetJobID] ()
RETURNS NVARCHAR(MAX)
AS
BEGIN
RETURN '$(ESCAPE_SQUOTE(JOBID))'
END
GO
"#;
Sqlcmd -S $servername -U $username -P $password -Q $sql -x;
This is a pretty poor workaround, but it does accomplish the task. Really hoping for something better.
I propose another workaround
my job has a step running : DTEXEC.exe /SERVER "$(ESCAPE_NONE(SRVR))"
I just have to add a SQLCMD variable before:
:setvar SRVR "$(ESCAPE_NONE(SRVR))"
this way the toked is considered as SQLCMD variables $(SRVR) and is replaced by the requested value
I am trying to implement Build automation using TFS (Version 12.0.31101.0)
This is the settings I am using, It build properly and publishes to the mentioned drop location:
For PreBuild I am trying to use the following batch script and this is not incrementing:
$path = "E:\Dev\ABC\My Project\AssemblyInfo.vb"
$pattern = '\<Assembly: AssemblyVersion\(.*\)\>'
(Get-Content $path) | ForEach-Object{
if($_ -match $pattern){
# We have found the matching line
# Edit the version number and put back.
$fileVersion = [version]$matches[1]
Write-Output "Major is $Matches[0] Minor is $Matches[1] Build is $Matches[2] Revision is [version]$matches[3]"
$newVersion = "{0}.{1}.{2}.{3}" -f $fileVersion.Major, $fileVersion.Minor, $fileVersion.Build, ($fileVersion.Revision + 1)
'<Assembly: AssemblyVersion("{0}")>' -f $newVersion
} else {
# Output line as is
$_
}
} | Set-Content $path
For 'post build script path' i want to zip the contents and put it into another folder, I am using the following script for this.
powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::CreateFromDirectory('\$(TF_BUILD_DROPLOCATION)\MySolution\_PublishedWebsites\ABC', 'ABC_Deploy.zip'); }"
On Executon it throws the following error:
Exception calling "CreateFromDirectory" with "2" argument(s): "Could not find
a part of the path 'C:\$TF_BUILD_DROPLOCATION\MySolution\_PublishedWebsites\ABC
At line:1 char:53
+ & { Add-Type -A 'System.IO.Compression.FileSystem';
[IO.Compression.ZipFile]::Cr ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : DirectoryNotFoundException
What changes should i make for both prebuild script and post build script to get this working?
For Prebuild:
Create a C# project(i used console project) which updates the assembly information..
Add RunScript to TFS build template before MSBuild.
Call the exe which is generated from c# project from your batch file, and pass the parameter as where your source code location.
Eg: (START "C:\UpdateAssemblyTest.exe" "\TFS-Server\TFSMapPath\ABC\")
PostBuild: I found a workaround for getting the TF_BUILD_DROPLOCATION. I modified the TFS Build Template and added a 'RunScript' tool and called my batch file from there and passed the drop location as an argument.
For 'Post Build Script' If build drop location is not accessible via the variable you are using; try using COPY command and copy the dropped folders to a known location on TFS Build server and then give that path in the ZIP Powershell command. (Though it is workaround for now. :) )
For 'Pre build Script', I will check and come back to you.
For 'post build script path', according to the Exception message it cannot identify the variable $TF_BUILD_DROPLOCATION.
As it's environment variable, So,please try to access the variables using $env:. The drop location would be
$env:TF_BUILD_DROPLOCATION for example.
For PreBuild, if you want to version the assemblies, you can try to put all custom Versioning work into a custom Version.proj MsBuild script and call it in TFS build definition before the .sln. The script injects Version into source code (SharedAssemblyInfo.cs, Wix code, readme.txt), and then solution build builds that source code. Please reference this thread :Versioning .NET builds
You can also reference this article :
https://www.stevefenton.co.uk/2012/11/automatically-updating-your-assemblyinfo-with-a-batch-file/
I have a problem with the processing of a SQL 2016 tabular model via a SSIS-Script task.
In the SSIS script task I invoke a powershell script the following way:
RunspaceConfiguration runspaceConfiguration = RunspaceConfiguration.Create();
Runspace runspace = RunspaceFactory.CreateRunspace(runspaceConfiguration);
runspace.Open();
// Configure Variables
Pipeline pipeline = runspace.CreatePipeline();
string query = Dts.Variables["XMLA"].Value.ToString();
//Read in powershell script
string script = readFile(#"folder");
script = script.Replace("plchldr_query", query);
pipeline.Commands.AddScript(script);
writeFile(#"folder", script);
try
{
pipeline.Invoke();
} catch(Exception e)
{
writeFile(#"folder", e.Message);
}
This powershell script only consists of two lines (plus some lines for debugging):
Import-Module SQLASCMDLETS -Verbose 4>&1 | Out-File "folder\filename.txt"
Invoke-ASCmd -Query "plchldr_query" -Server "Servername\TABULAR"
Unfortunately when I start the SSIS package the Invoke-ASCmd fails with the error message:
Could not load file or assembly
'Microsoft.AnalysisServices.PowerShell.Cmdlets' or one of its
dependencies. Invalid Pointer (Exception from HRESULT: 0x80004003
(E_POINTER))
When I start exactly the same script directly in powershell everything works fine.
For me it looks like the problem could be connected with different hosts. When I start the script directly in powershell the host is "Console Host" (Version 4.0). But when I start the script via SSIS-Task the host is "Default Host" (Version 4.0).
To check which modules are available I did execute the command:
Get-Module -ListAvailable | out-file "folder\file"
I thought that this list might be different, but it doesn´t matter wether I execute this command directly in powershell or via SSIS, the result is the same:
ModuleType Name ExportedCommands
---------- ---- ----------------
Manifest SqlServer {Invoke-ProcessCube, New-SqlCngColumnMas...
Manifest BitsTransfer {Complete-BitsTransfer, Add-BitsFile, Re...
Manifest CimCmdlets {New-CimSessionOption, Get-CimAssociated...
Script ISE {Import-IseSnippet, Get-IseSnippet, New-...
Manifest Microsoft.PowerShell.Diagnostics {New-WinEvent, Export-Counter, Get-WinEv...
Manifest Microsoft.PowerShell.Host {Start-Transcript, Stop-Transcript}
Manifest Microsoft.PowerShell.Management {Remove-WmiObject, Remove-EventLog, Add-...
Manifest Microsoft.PowerShell.Security {Get-Credential, Get-Acl, Set-Authentico...
Manifest Microsoft.PowerShell.Utility
Manifest Microsoft.WSMan.Management {Test-WSMan, Set-WSManInstance, Get-WSMa...
Manifest PSDiagnostics {Set-LogProperties, Enable-WSManTrace, S...
Binary PSScheduledJob {Get-JobTrigger, Add-JobTrigger, Get-Sch...
Manifest TroubleshootingPack {Get-TroubleshootingPack, Invoke-Trouble...
Manifest AppvClient
Binary ThirdPartyNotice
Binary UEV {Restore-UevUserSetting, Get-UevAppxPack...
Manifest SQLASCMDLETS {Add-RoleMember, New-RestoreFolder, Invo...
Manifest SQLPS {Backup-ASDatabase, Get-SqlInstance, New...
So, now I´m totally stuck. For me the only difference between the two types of executions is the host in which the script is executed. But both hosts have the same modules available. Do you maybe have another idea what could cause this different behaviour?
Thanks in advance!
Update:
I now checked the Import-Module SQLASCMDLETS call via -Verbose. When I execute this command in the powershell I get the following output:
Importing cmdlet 'Add-RoleMember'.
Importing cmdlet 'Backup-ASDatabase'.
Importing cmdlet 'Invoke-ASCmd'.
Importing cmdlet 'Invoke-ProcessASDatabase'.
Importing cmdlet 'Invoke-ProcessCube'.
Importing cmdlet 'Invoke-ProcessDimension'.
Importing cmdlet 'Invoke-ProcessPartition'.
Importing cmdlet 'Invoke-ProcessTable'.
Importing cmdlet 'Merge-Partition'.
Importing cmdlet 'New-RestoreFolder'.
Importing cmdlet 'New-RestoreLocation'.
Importing cmdlet 'Remove-RoleMember'.
Importing cmdlet 'Restore-ASDatabase'.
The output for execution with SSIS is:
Loading module from path 'C:\Program Files (x86)\Microsoft SQL
Server\130\Tools\PowerShell\Modules\SQLASCMDLETS\SQLASCMDLETS.psd1'.
Loading module from path 'C:\Program Files (x86)\Microsoft SQL
Server\130\Tools\
PowerShell\Modules\SQLASCMDLETS\Microsoft.AnalysisServices.PowerShell.Cmdlets.dl
l'.
I checked that the files are present in the folders. But the messages seem for me like that the loading of the modules happens differently. Could that be possible? If yes, does anybody knows why? Or do I misinterpret the different outputs?
UPDATE: Modified the script to work within the bounds of PS1 as required by SQLPS.
Changed:
IF($property.Value -match $regex){
$currentBadLine = (ConvertTo-Csv $_ -NoTypeInformation -Delimiter $delimiter);
$badLines += $currentBadLine[1,2]
};
To:
IF($property.Value -match $regex){
$badLines += $_ | Select-Object | ft -autoSize;
};
Prints a new header for each bad line, but it's not the end of the world and not worth the effort to prevent.
I have a Powershell script that pre-processes CSV files before they have a chance to screw up my data imports.
On two servers in a row now I have confirmed that the PS Major Version at least 2, and that the following code snippet runs fine in Powershell ISE. The purpose of the code is to read in each line of the CSV, and then loop through the columns looking for the regex pattern in the $regex variable. When it finds one I want it to keep track of the error before fixing it so I can write an error log later before outputting a cleaned up file ready for import.
%{
Foreach($Property in $_.PSObject.Properties){
IF($property.Value -match $regex){
$currentBadLine = (ConvertTo-Csv $_ -NoTypeInformation -Delimiter $delimiter);
$badLines += $currentBadLine[1,2]
};
$property.Value = $property.Value -replace $regex;
}
$_
}
But once I put that code into an agent job the agent complains:
'The term 'ConvertTo-Csv' is not recognized as the name of a cmdlet,
function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and
try again. '
The question then is this: Is the Agents Powershell subsystem using a different version of Powershell than the rest of the system? If so, how do I find out which version the Subsystem is using and if possible upgrade it so I can fix this.
The server is running:
Windows Server 2008 R2 Enterprise
PS Major version 2, Minor 0 Build -1 revision -1
SQL Server 2008 R2 SP1 10.5.2500.0 64 bit
Thanks!
Yes, proper PowerShell support isn't really implemented until SQL Server 2012 (an even that is a bit flakey as to what cmdlets it supports)
In 2008 and R2 the agent's powershell implementation is actually a minishell, created by the now (thankfully) deprecated make-shell.exe utility, which only allows v1 cmdlets to run, and disallows the use of Add-PSSnapin so you can't add anymore cmdlets.
To get proper powershell support, you either need to shell out and call an external script, or run the job as a windows scheduled task rather than an agent job.
The following article explains a lot about why powershell support in 2008 R2 doesn't work like you think it should:
The Truth about SQLPS and PowerShell V2
One work-around: Export-CSV to a file, then Get-Content from the file.
$rows = ,(New-Object PSObject -prop #{a=7; b='stuff';});
$rows +=,(New-Object PSObject -prop #{a=77; b='more';});
#To run from SQL Job, change from this:
$csvrows = $rows | ConvertTo-CSV -NoType | % {$_.Replace('"','')};
write-output $csvrows;
#to this:
$rows | Export-CSV -NoType "C:\Temp\T.csv"
$csvrows = (Get-Content "C:\Temp\T.csv") | % {$_.Replace('"','')};
write-output $csvrows;