How to deploy SQL CLR stored procedure to multiple servers - sql-server

I have inherited a SQL CLR project as part of a code maintenance project that I'm working on for a client. I'm pretty new to SQL CLR, admittedly, so I'm trying to figure out how it works.
I noticed that the database connection string is stored in the project's Properties, so I know how to change it if I need to. The one question I have though is this: is it possible to set multiple connection strings for deployment to multiple SQL Server instances? In my case I have a local dev machine, a staging server, and a production server (with a separate copy of the target database on each server). I'd like to be able to deploy the SQL CLR assembly to all 3 without having to change the connection string and re-build for each one.

You should not deploy to anywhere but development via Visual Studio, hence the connection string in the Project should always point to your dev environment.
Once you have the code tested in the development server, you can script out the Assembly in SSMS by right-clicking on the Assembly in question and do "Script Assembly As..." then "Create To..." and then "New Query Window". This will give you the basic script that should be used to deploy to QA, Staging, and Production.
The general format is:
USE [DBName]
GO
CREATE ASSEMBLY [AssemblyName]
AUTHORIZATION [dbo]
FROM 0x0000...
WITH PERMISSION_SET = SAFE
You do not really need to propagate the Assembly Files to the other environments, though if you want to it does not hurt.
If you want to automate that, once you have that basic script you can always grab the updated Assembly code (what is noted as 0x0000 above) via:
SELECT Content FROM sys.assembly_files WHERE name = 'AssemblyName'
Edit:
For the sake of completeness, as Jeremy mentioned in a comment below, the above info only describes deployment of the Assembly itself, not of the wrapper objects to access the code within the Assembly. A full deployment process would:
Drop existing wrapper objects (Stored Procs, Functions, Triggers, Types, and Aggregates)
Drop the Assembly
Create the new Assembly
Create the wrapper objects

When you deploy the code to your development server, Visual Studio creates a .sql file in the bin/Release folder.
This can useful for deployment, it requires some cleaning.
Here is a perl script I'm using to get a deployment script from the script created by VS.
It's closely linked to my needs and the file format (I'm using VS 2010 SP1, SQL 2008 R2, perl within cygwin), consider this as an example it may not work automagically for everyone.
use strict;
use warnings;
use Text::Unidecode 'unidecode'; # http://search.cpan.org/dist/Text-Unidecode/
sub ProcessBlock($)
{
my $lines = $_[0];
if ($lines =~ "Deployment script for") { return 0; }
if ($lines =~ "^SET ") { return 0; }
if ($lines =~ "^:") { return 0; }
if ($lines =~ "^USE ") { return 0; }
if ($lines =~ "^BEGIN TRANSACTION") { return 0; }
if ($lines =~ "extendedproperty") { return 0; }
if ($lines =~ "^PRINT ") { return 0; }
if ($lines =~ "#tmpErrors") { return 0; }
if ($lines =~ "^IF \#\#TRANCOUNT") { return 0; }
my $drop = $lines;
if ($drop =~ m/^DROP (FUNCTION|PROCEDURE) ([^ ]+);/m)
{
printf("if OBJECT_ID('$2') IS NOT NULL\n");
}
elsif ($drop =~ m/^DROP ASSEMBLY \[([^ ]+)\];/m)
{
printf("IF EXISTS (SELECT 1 FROM sys.assemblies WHERE name = '$1')\n");
}
printf($lines);
printf("GO\n");
my $create = $lines;
if ($create =~ m/^CREATE PROCEDURE (\[[^]]+\])\.(\[[^]]+\])/m)
{
printf("GRANT EXECUTE ON $1.$2 TO PUBLIC\nGO\n");
}
elsif ($create =~ m/^CREATE FUNCTION (\[[^]]+\])\.(\[[^]]+\]).*RETURNS .* TABLE /ms)
{
printf("GRANT SELECT ON $1.$2 TO PUBLIC\nGO\n");
}
elsif ($create =~ m/^CREATE FUNCTION (\[[^]]+\])\.(\[[^]]+\])/m)
{
printf("GRANT EXECUTE ON $1.$2 TO PUBLIC\nGO\n");
}
}
my $block="";
while (<>)
{
my $line = $_;
$line = unidecode($line);
if ($line =~ "^GO")
{
ProcessBlock($block);
$block = "";
}
else
{
$block .= $line;
}
}
Usage:
perl FixDeploy.pl < YourAssembly.sql > YourAssembly.Deploy.sql

Look here: The difference between the connections strings in SQLCLR I think you should use context connection if possible. That way you don't have to reconfigure.
If you need different credentials or something, you can query a settings table that holds those settings. Use the context connection to connect, query the settings table to get the login details and then use them to connect again.
Also: the connection string is in the properties, but as I understand the settings.xml does not get deployed so you'd always be getting the default values hardcoded into settings class.

Related

Issue with scripting out subscriptions using Windows PowerShell

I am trying to script out the replication objects via PowerShell using Microsoft.SqlServer.Rmo.dll. The replication type is transactional with push subscriptions.
I have been able to script out publication, articles, PALs but not able to script out publisher side subscriptions.
Reference
[reflection.assembly]::LoadFrom("c:\\sql\\Microsoft.SqlServer.Rmo.dll") | out-null
ScriptOptions
$scriptargs =[Microsoft.SqlServer.Replication.ScriptOptions]::Creation `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludeCreateLogreaderAgent `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludeCreateMergeAgent `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludeCreateQueuereaderAgent `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludePublicationAccesses `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludeArticles `
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludePublisherSideSubscriptions` #one way tried to get the subscriptions
-bor [Microsoft.SqlServer.Replication.ScriptOptions]::IncludeGo
foreach($replicateddatabase in $repsvr.ReplicationDatabases)
{
if ($replicateddatabase.TransPublications.Count -gt 0)
{
foreach($tranpub in $replicateddatabase.TransPublications)
{
**[string] $myscript=$tranpub.script($scriptargs)** #Errors out here
writetofile $myscript $filename 0
}
}
}
The other way I tried is exclude IncludePublisherSideSubscriptions from Scriptoptions and tried to script out directly using the following statement
foreach($replicateddatabase in $repsvr.ReplicationDatabases)
{
if ($replicateddatabase.TransPublications.Count -gt 0)
{
foreach($tranpub in $replicateddatabase.TransPublications)
{
[string] $subs=$tranpub.TransSubscriptions.script($scriptargs) #another way but same error
writetofile $subs $filename 0
}
}
}
The third way I tried:
$repsvr.ReplicationDatabases.TransPublications.TransSubscriptions.Script($scriptargs)
Of all, I found the following link to be very helpful where my code is mostly based on but just got stuck in the scripting out of publisher side subscriptions. I appreciate your help.
Microsoft.SqlServer.Replication has been deprecated since SQL Server 2012 (I think), and as of SQL Server 2017, it's no longer available. Follow the link and try switching the version to 2017 or 2019.
If you're using server 2016 and below, you may just need to load the rmo assembly from the GAC (Global Assembly Cache), this will do that for you.
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.Rmo')
If for whatever reason, you don't want to, or can't use the GAC, do this
Add-Type -Path 'C:\path\to\dll\Microsoft.SqlServer.Rmo.dll'
I'll dig, and see if there's a different way to go about doing what you're trying to do.
Edit: Not sure if I'm misunderstanding the docs about RMO, or if they're just out of date
you take this link and it shows you how to configure publishing & distribution using RMO on Server 2019. Step 2 references this which shows the ReplicationServer class. The problem is, that's for SQL Server 2016. I'm not entirely sure if this is supported past 2016 or not.

Convert dacpac into folder structure of database objects with powershell

I'm working on integrating SQL Server databases into our in-house version control/deployment utility, which is built with powershell,and uses Github as a repository.
Using the excellent sqlpackage.exe utility, I have been able to add a process whereby a developer can extract their current changes into a dacpac and store it in Github, then do the opposite in reverse when they want to get the latest version. However, because the .dacpac is a binary file, it's not possible to see differences in git. I have mitigated this somewhat by unzipping the dacpac before storing in in source control, so contained xml files are added instead. However, even though these files are text-based, they are still not easy to look through and find differences.
What I would like to do, is convert the dacpac into a folder structure similar to what would be seen in SSMS (with all the database objects such as triggers, sprocs etc in their respective folders), store that in Github, and then convert it back into a dacpac when a client checks out the code. However, there doesn't seem to be any function in sqlpackage.exe for this, and I can't find any documentation. Is there any command line tool I can use to this through Powershell?
Using the public APIs for DacFx you can load the dacpac, iterate over all objects, and script each one out. If you're willing to write your own code you could write each one to its own file based on the object type. The basic process is covered in the model filtering samples in the DacExtensions Github project. Specifically you'll want to do something like the ModelFilterer code that loads a dacpac, queries all objects, scripts them out - see the CreateFilteredModel method. I've put a sample that should mostly work below. Once you have this, you can easily do compare on a per-object basis.
using (TSqlModel model = new TSqlModel(dacpacPath))
{
IEnumerable<TSqlObject> allObjects = model.GetObjects(QueryScopes);
foreach (TSqlObject tsqlObject allObjects)
{
string script;
if (tsqlObject.TryGetScript(out script))
{
// Some objects such as the DatabaseOptions can't be scripted out.
// Write to disk by object type
string objectTypeName = tsqlObject.ObjectType.Name;
// pseudo-code as I didn't bother writing.
// basically just create the folder and write a file
this.MkdirIfNotExists(objectTypeName);
this.WriteToFile(objectTypeName, tsqlObject.Name + '.sql', script);
}
}
}
This can be converted into a powershell cmdlet fairly easily. The dacfx libraries are on nuget at https://www.nuget.org/packages/Microsoft.SqlServer.DacFx.x64/ so you should be able to install them in PS and then use the code without too much trouble.
Based on the other post I was able to get a script working. Caveat is you'll have to try the types till you get what you want... The way it is no it trys to put the full http or https value for some of the objects.
param($dacpacPath = 'c:\somepath' Debug', $dacpac = 'your.dacpac')
Add-Type -Path 'C:\Program Files (x86)\Microsoft SQL Server\120\DAC\bin\Microsoft.SqlServer.Dac.dll'
add-type -path 'C:\Program Files (x86)\Microsoft SQL Server\120\DAC\bin\Microsoft.SqlServer.Dac.Extensions.dll'
cd $dacpacPath
$dacPack = [Microsoft.SqlServer.Dac.DacPackage]::Load(((get-item ".\$dacpac").fullname))
$model =[Microsoft.SqlServer.Dac.Model.TSqlModel]::new(((get-item ".\$dacpac").fullname))
$queryScopes = [Microsoft.SqlServer.Dac.Model.DacQueryScopes]::All
$return = [System.Collections.Generic.IEnumerable[string]]
$returnObjects = $model.GetObjects([Microsoft.SqlServer.Dac.Model.DacQueryScopes]::All)
$s = ''
foreach($r in $returnObjects)
{
if ($r.TryGetScript([ref]$s))
{
$objectTypeName = $r.ObjectType.Name;
$d="c:\temp\db\$objectTypeName"
if(!(test-path $d ))
{
new-item $d -ItemType Directory
}
$filename = "$d\$($r.Name.Parts).sql"
if(! (test-path $filename))
{new-item $filename -ItemType File}
$s | out-file $filename -Force
write-output $filename
}
}

sqlsrv drivers slow in codeigniter?

I have installed the latest version of CI 2.1.3
Now after running a query, I am getting a very slow response time for something very simple such as:
function Bash(){
$sql = “SELECT * FROM Contacts”;
$stmt = sqlsrv_query( $conn, $sql );
if( $stmt === false) {
die( print_r( sqlsrv_errors(), true) );
}
after querying a remote database. (Sql server 2008)
When I run this same query in a simple PHP script against the same remote database. I get results instantly.
a) Has anyone else experienced this problem with the sqlsrv drivers in codeigniter?
If so, how did you solve it?
Here is my connection string:
$db['default']['hostname'] = "xxxxx,1433";
$db['default']['username'] = "xx";
$db['default']['password'] = "xxxxxx-xx";
$db['default']['database'] = "xxxxxxxxx";
$db['default']['dbdriver'] = "sqlsrv";
$db['default']['dbprefix'] = '';
$db['default']['pconnect'] = TRUE;
$db['default']['db_debug'] = TRUE;
$db['default']['cache_on'] = TRUE;
$db['default']['cachedir'] = '';
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
$db['default']['swap_pre'] = '';
$db['default']['autoinit'] = TRUE;
$db['default']['stricton'] = FALSE;
UPDATE:
I have found the following from running the profiler.
DATABASE: database QUERIES: 1 (Hide)
0.0659 select * from Contacts
Loading Time: Base Classes 0.0428
Controller Execution Time ( Welcome / AzureBash ) 58.2173
Total Execution Time 58.2602
It seems as though the query is executing in 0.06 secs but the controller is taking a minute to load.
No idea why this is happening.
Solution
The active records interface for the latest SQLSRV drivers are buggy.
So, download and overwrite the existing interface with these (overwrite your sqlsrv folder in the database folder in CI):
http://www.kaweb.co.uk/blog/mssql-server-2005-and-codeigniter/
Note: These have been tested with SQL Azure and works.
$query->num_rows(); does not work with these drivers, so I suggest you use count instead. Or create your own wrapper.
In addition date is now a date object type in your result set.
I hope this helps.
Solution 2
If for whatever reason you find a bug that makes this completely unusable. Revert back to the sqlsrv interface originally provided. You will find what is causing the problem is the way the original interface are executing the query, thus, create a database helper class; use $sql = $this->db->last_query(); to get the query you was about to execute and then within the database_helper class execute it yourself:
function MakeDbCall ($sql)
{
$serverName = "xxxxx-xxxx-xxx,1433"; //serverName\instanceName
$connectionInfo = array( "Database"=>"xxx", "UID"=>"xx", "PWD"=>"xxxxx","ConnectionPooling" => "1");
$conn = sqlsrv_connect($serverName,$connectionInfo);
$stmt = sqlsrv_query($conn, $sql);
while( $row = sqlsrv_fetch_array( $stmt, SQLSRV_FETCH_ASSOC) ) {
$result_array[] = $row;
}
return $result_array;
}
Create one for row_array.
You should be able to call this function directly, from anywhere in your app. Whilst taking advantage of the way active_records constructs your query.
Not an ideal solution, but until codeigniter sort their SQLSRV class, there is not a lot we can do.
Adding an answer to this after the answer has already been accepted because I found a different solution. I was having the same problem ... looping through the result set was very very slow. i opened system/database/drivers/sqlsrv/sqlsrv_driver.php and found the connection function. i noticed that is was using the SQLSRV_CURSOR_STATIC option. i changed this to SQLSRV_CURSOR_CLIENT_BUFFERED and my slowness problems went away. See documentation for this here:
http://msdn.microsoft.com/en-us/library/hh487160(v=sql.105).aspx
I honestly have no idea what the sql server driver for php is doing, however, given the speed up, etc i can guess that the driver might be using a cursor by default. this seems like an awful idea. i also am assuming that by choosing client_buffered the data for the query would b e read without a cursor and accessed in memory on the client as if it were a cursor. If this is the case, bad things might happen if you try to execute a query that has many many rows to read. Perhaps another option (SQLSRV_CURSOR_FORWARD?) can be used to read data without a cursor - but i'm sure the methods used to access the query will be more limited (e.g. not using result_array())
-Don
Solution
The active records interface for the latest SQLSRV drivers are buggy.
So, download and overwrite the existing interface with these (overwrite your sqlsrv folder in the database folder in CI):
http://www.kaweb.co.uk/blog/mssql-server-2005-and-codeigniter/
Note: These have been tested with SQL Azure and works.
$query->num_rows(); does not work with these drivers, so I suggest you use count instead. Or create your own wrapper.
In addition date is now a date object type in your result set.
Solution 2
If for whatever reason you find a bug that makes this completely unusable. Revert back to the sqlsrv interface originally provided. You will find what is causing the problem is the way the original interface are executing the query, thus, create a database helper class; use $sql = $this->db->last_query(); to get the query you was about to execute and then within the database_helper class execute it yourself:
function MakeDbCall ($sql)
{
$serverName = "xxxxx-xxxx-xxx,1433"; //serverName\instanceName
$connectionInfo = array( "Database"=>"xxx", "UID"=>"xx", "PWD"=>"xxxxx","ConnectionPooling" => "1");
$conn = sqlsrv_connect($serverName,$connectionInfo);
$stmt = sqlsrv_query($conn, $sql);
while( $row = sqlsrv_fetch_array( $stmt, SQLSRV_FETCH_ASSOC) ) {
$result_array[] = $row;
}
return $result_array;
}
Create one for row_array.
You should be able to call this function directly, from anywhere in your app. Whilst taking advantage of the way active_records constructs your query.
Not an ideal solution, but until codeigniter sort their SQLSRV class, there is not a lot we can do.
What is your connection string? You can specify the "network protocol" explicitly, which somtimes can affect speed.
http://www.connectionstrings.com/articles/show/define-sql-server-network-protocol
"Provider=sqloledb;Data Source=190.190.200.100,1433;Network Library=DBMSSOCN;Initial Catalog=pubs;User ID=myUsername;Password=myPassword;"
By specifying the IP address, the port number (1433) and the Network Library, you are providing a very granular connection string.
Your details may vary of course.
Alot of times, you don't need this. But I've been on a few client trips where this was the magic dust.
You might want to turn db_debug to FALSE which should save time debugging the database.
Also, would suggest to turn cache_on to FALSE and specify cachedir and use $this->db->cache_on(); for queries that are less dynamic, i.e. does not change frequently.
For speed up fetch up to 3 times please use "MultipleActiveResultSets"=>'0' in your sqlsrv_connect connection options.
Ex:
$db = sqlsrv_connect('127.0.0.1', array('Database'=>'dbname','UID'=> 'sa','PWD'=> 'pass',"CharacterSet" =>"UTF-8","ConnectionPooling" => "1"
,"MultipleActiveResultSets"=>'0'
));

Perl: why aren't my environment variables being set?

I am trying to connect to an Oracle DB using perl-DBD-Oracle. I need to have the following environmental variables set:
export LD_LIBRARY_PATH=/home/x/lib/ora10gclient
export TNS_ADMIN=/home/x/lib/ora10gclient
export PATH=/home/x/lib/ora10gclient:/usr/kerberos/bin:/home/x/bin:/home/x/bin64:/usr/local/sbin:/sbin:/usr/sbin:/usr/local/bin:/bin:/usr/bin
export ORACLE_HOME=/home/x/lib/ora10gclient
This script is run by a headless user and I have placed those lines in the .bash_profile file for that headless user.
The problem is, the headless user ssh's into the machine where this script is run and the bash profile isn't sourced. So I thought I would just set those environment variables in the BEGIN block of the script like so:
BEGIN {
$ENV{'LD_LIBRARY_PATH'} = '/home/x/lib/ora10gclient';
$ENV{'TNS_ADMIN'} = '/home/x/lib/ora10gclient';
$ENV{'PATH'} = '/home/x/lib/ora10gclient:/usr/kerberos/bin:/home/x/bin:/home/x/bin64:/usr/local/sbin:/sbin:/usr/sbin:/usr/local/bin:/bin:/usr/bin';
$ENV{'ORACLE_HOME'} = '/home/x/lib/ora10gclient';
}
However, the script fails claiming it can't find Oracle.pm, something that it doesn't do when those environment variables are set.
I've sprinkled print statements throughout the script to confirm that it does indeed set the environment variables. And even printed the environment variables out the line before attempting to create a handle to the DB (which is where the script fails) and confirmed that they are still set to the correct values.
I'm not spawning any child processes either. Below is a simplified version of my script which fails in the same manner.
#!/usr/local/bin/perl
use warnings;
use strict;
BEGIN {
$ENV{'LD_LIBRARY_PATH'} = '/home/x/lib/ora10gclient';
$ENV{'TNS_ADMIN'} = '/home/x/lib/ora10gclient';
$ENV{'PATH'} = '/home/x/lib/ora10gclient:/usr/kerberos/bin:/home/x/bin:/home/x/bin64:/usr/local/sbin:/sbin:/usr/sbin:/usr/local/bin:/bin:/usr/bin';
$ENV{'ORACLE_HOME'} = '/home/x/lib/ora10gclient';
}
use DBI;
testConnect();
sub testConnect {
my $orclPort = 1521;
my $orclHost = '<oraclehost>';
my $orclUser = '<user>';
my $srvName = '<servicename>';
my $db = "DBI:Oracle:service_name=$srvName;host=$orclHost;port=$orclPort";
my $orclDBHndl = DBI->connect($db, $orclUser, $orclUser) or die "could not connect: $DBI::errstr\n";
}
Any ideas on what might be the problem?
LD_LIBRARY_PATH needs to be set before perl starts. Write a shell wrapper that sets the variables and then launches your script.
Also see this thread for alternatives and other links to explanations.

zend framework: models cannot interact with database on the server

I have just finished my first site built with zend framework and all works great on my local machine.
Then I uploaded it to the server (godaddy) and all works except any connection my models do with the database. I have made a connetion to the database with regular PDO with the credentials in my application.ini and it worked, and I can interact with the model if it's not returning anything from the database (and again all the models work great on my local machine).
My models looks like this:
class Default_Model_picture extends Zend_Db_Table_Abstract
{
protected $_name = 'pictures';
protected $_primary = 'id';
public function getPicturesByCategory($category)
{
$query = $this->select()->from(array('pictures'), array(
'pictures.id', 'pictures.pic_name', 'pictures.pic_desc',
'pictures.pic_category', 'pictures.pic_date_added',
'pictures.pic_larger', 'pictures.pic_url'));
$query->where('pic_category = ?', $category);
$query->order('pic_date_added ASC');
$result = $this->fetchAll($query);
return $result;
}
}
this is an example for a model, obviously i did not added lots of methods.
i have no idea what to do next.
I am assuming you set up the db connection correctly into $db. Afterwards you must set it as the default adapter for Zend_Db_Table.
Zend_Db_Table::setDefaultAdapter($db);
I am just assuming this is what went wrong. But it is a common problem, so I decided to go ahead and answer anyway.
Since your script works fine on your local machine, the first thing I check is if you have got the database connection params setup correctly in your application.ini
Try to write a test script that uses the pdo functions on itself (without zend framework). see if you get any errors at all
try {
$dbh = new PDO('mysql:host=YOURHOST;dbname=YOURDBNAME', $YOURUSERNAME, $YOURPASSWORD);
foreach($dbh->query('SELECT * from FOO') as $row) {
print_r($row);
}
$dbh = null;
} catch (PDOException $e) {
print "Error!: " . $e->getMessage() . "<br/>";
die();
}

Resources