The process: an Azure agent that runs on a Windows 10 32bit pro machine with SQL Server 2014 Express installed.
The pipeline is built and runs successfully with PowerShell scripts as follows:
Create blank database
Create tables needed
C# application runs and populates the tables executed via a PowerShell script
Cross reference tables to update data needed.
Build a SSIS package
After result from SSIS package is success perform a backup
Command:
Backup-SqlDatabase -ServerInstance "$env.ComputerName" -Database "RealDB"
-BackupAction Database -BackupFile $Path -Blocksize 4096
This all works with one exception the actual backup I get is missing the data from the SSIS package run. BUT if I log into the machine and restore the backup used from $Path it is missing the data.
When I query the database after this process the data is there in the database.
There is only one database so its not backing up a different one.
I can run this command in powershell on the machine and my backup has the missing data that the powershell command from the agent does not.
Also interesting enough if I remove the -Blocksize 4096, it works as I expect and the backup has the data in it. I am considering abandoning the powershell due to this but thought I would ask to see if anyone experienced this or no.
Any help or thoughts are appreciated.
Thank you
Thank you #user19702 I was so consumed looking at the backup command with the added -BlockSize that I completely ignored the fact that my data has increased (thanks random process I never heard of before today) and even though the powershell is written to start the backup AFTER the SSIS package the process was not done. To find it I started task manager on the machine while the build was running and watched the process stay in memory for a few seconds when the backup started. I added a powershell command to make it wait a few seconds before processing the backup and its working.
In case anyone is wondering this is the command
$result = $package.Execute("false", $null)
Write-Host "Package ID Result: " $result
Start-Sleep -Seconds 10
Backup-SqlDatabase -ServerInstance "$env:ComputerName" -Database "RealDB" -BackupAction Database -BlockSize 4096 -BackupFile $Path
Thank You!!
Related
I have a fairly straightforward Powershell script that is ran as part of a Bamboo deployment which includes a call to Invoke-DbaQuery to run sql scripts against a database.
After an incident with a bad script left our deployment hanging for hours, I am attempting to implement a timeout which I would expect to cause the Invoke-DbaQuery to fail, and thus the script to fail, and likewise the Bamboo deployment after a set amount of time.
However, Powershell seems to be ignoring the -queryTimeout parameter of Invoke-DbaQuery. Running the following commands calling an intentionally long running script and an arbitrarily short timeout, the query continues to execute hours after the timeout.
To eliminate variables I am testing this directly on Powershell on the server with the sql instance.
$path = "c:\somescript.sql"
$server = "localhost"
Invoke-DbaQuery -file $path -SqlInstance $server -Database MyDatabase -QueryTimeout 30
I need to create a CSV file from a stored procedure that runs everyday on SQL Server with shipping details.
What would be the best way to go about this?
The solution we found best was to call the stored procedure using Powershell and have a task running in windows scheduler everyday which runs the Powershell.
Another option is BCP, or bulk command protocol. You essentially can either build this into the stored proc, or exclusively in the sql server agent job that runs it, or in an SSIS package. Wherever you prefer.
Here’s a link to Microsoft’s BCP utility:
https://learn.microsoft.com/en-us/sql/tools/bcp-utility?view=sql-server-ver15
It is important to note you may need to enable some more advanced options in SQL Server. It is highly configurable though, including the ability to dump entire tables to a csv file.
I don't know about Oracle but for SQL Server the easiest method I found was to use PowerShell.
$hostname = hostname
Invoke-Sqlcmd -ServerInstance $hostname -Database master -Query "select * from sys.sysdatabases" | Export-Csv "d:\result.csv" -NoTypeInformation
gives a CSV file at desired location.
If your SQL Server is at remote location, please make sure your server can connect to said SQL Server and if any certificate/encryption is in place then you have all that is necessary.
I'm trying to automate executing SQL scripts using Invoke-SQLCmd powershell command. As a source I have a one script generated by SSDT Schema Compare (4000 lines, all inside transaction). I'm triggering this script in the following way:
Invoke-SQLCmd -InputFile "$Path" -ServerInstance "$Server" -Database "$DbName" -ErrorAction "Stop"
As an output I'm receiving the following error:
Invoke-SQLCmd : Could not allocate a new page for database 'TEMPDB' because of insufficient disk space in filegroup 'DEFAULT'. [...]
I know that this problem is probably related to lack of free space on testdb disk, but what is interesting the script has been completed successfully when it was executed from SQL Server Management Studio on the same database.
The question is how is it possible and how can I make this script working correctly from PowerShell.
This error is due to insufficient Transaction log space...
You can use below command to find out offending transaction and take care of it
Dbcc opentran
This may be also due to your script generating a lot of transaction log and you dont have enough Transaction log space left...if this is the case, you might need to break up your command into multiple commands or increase your tlog space
I'm trying to write a PowerShell script which will execute tsql query to only one remote server using invoke-sqlcmd. Those tsql queries are simple one like backup/restore database, create a user, etc.
Below is an extract of it :
# 5 # Create clientdb database on secondary server by restoring the full backup for primary
Try {
Invoke-sqlcmd -ServerInstance 'REMOTESQLSRV'`
-Username 'ts_sql' -Password 'somepassword'`
-InputFile "$LScltid\__01_On_Secondary_CreateDB2_srv2.sql"`
-ErrorAction Stop
Write-Host " clt_$id is now restored to secondary server "`
-ForegroundColor White -BackgroundColor Green
} Catch {
Write-Host " Restore operation for clt_$id did not succeed. Check the error logs " -ForegroundColor Black -BackgroundColor Red
}
My scripts always break here. For some reasons that i could not put my head on, invoke-sqlcmd does not use the variable "$LScltid" to resolve the path where it will find the .sql script.
Everytime i had to run it, it change the current directory to either the SQLERVER:\ provider or some other causing the script to failed at this step.
Am I doing this the right way? If so, how should i adapt the command to perform as I expect it to.?
UPDATE
Forgot to mention, if i run the script with the variables values hard-coded I'm able to get the result i need (in this case restoring a database from device).
Thanks for your feedbacks.
Odd thing, the command is now working. I don't really know what i've done wrong previously, but the exact same command is now working.
Just a heads up for those facing the same issue:
If you have to use Invoke-Sqlcmd in your scripts, beware of the provider change especially if the commands coming after Invoque-Sqlcmd are regulars one (get, set, copy, new, etc....)
In my case, somewhere in my script between two Invoke-sqlcmd commands i had to copy files from local to remote server. Everytime the command failed because the provider changed. As a workaround I set-location before copy-item command execution and that maneuver seemed to do the trick (don't know if its re-commanded tough.
Thanks Stackoverflow Team
I've been using following backup script for a while to create .bacpac files for MSSQL databases. The script working ok in general but fails for some databases. Those databases are not too much different to others, maybe a bit bigger. As this is used only for dev databases the average size is not big, the bacpac file size is ~200Mb.
The weird thing the script successfully running when executed from PowerShell ISE but failed when running from the command line. Please note that script only failing for some databases and work for others. The error message is not really helpful:
WARNING: Exception occurred: Exception calling "ExportBacpac" with "2" argument(s): "Could not export schema and data from database."
We using MSSQL 2104 and the database can be exported to backup from MSSQL Studio without problems.
The script:
Param(
# Database name to backup e.g 'MYDB'
$databaseName,
# Database connection string "server=server ip;Integrated Security = True;User ID=user;Password=pass"
$connectionString,
# Path to the directory where backup file should be created
$backupDirectory
)
add-type -path "C:\Program Files (x86)\Microsoft SQL Server\120\DAC\bin\Microsoft.SqlServer.Dac.dll";
try {
$dacService = new-object Microsoft.SqlServer.Dac.DacServices $connectionString
# build backup filename
$backupFileName = $backupDirectory+"\"+$dataBaseName+[DateTime]::Now.ToString("yyyyMMdd-HHmmss")+".bacpac"
# perform backup
$dacService.exportBacpac($backupFileName, $dataBaseName);
Write-Output "Database backup file created $backupFileName"
} catch {
Write-warning "Exception occurred: $_"
throw "Database backup haven't been created, execution aborted."
}
Does anyone come across this issue? I know PowerShell and PowerShell ISE a bit different but I don't understand why script execution produces different results.
[EDIT]
Tried to add event listener for DacService and print output to get more debug information
register-objectevent -in $dacService -eventname Message -source "msg" -action { out-host -in $Event.SourceArgs[1].Message.Message } | Out-Null
The output is
Dac Assembly loaded.
Extracting schema (Start)
Gathering database options
Gathering users
WARNING: Exception occurred: Exception calling "ExportBacpac" with "2" argument(s): "Could not export schema and data
from database."
Gathering roles
Gathering application roles
Gathering role memberships
Gathering filegroups
Gathering full-text catalogs
Gathering assemblies
Gathering certificates
....
Processing Table '[dbo].[file_storage_entity]'. 99.74 % done.
Processing Table '[dbo].[file_storage_entity]'. 100.00 % done.
Exporting data (Failed)
[EDIT 2]
As a workaround, I've created C# program to do the same. Runs fine in any environment. The code virtually the same as in PowerShell script.