I currently have a Base SQL Server project. This project contains shared SQL tables/procs/functions that are used by various different modules/API's.
When I create a new module, I have to host it in the same Git repo, and then add the SQL scripts by reference to the new "child" module. This ensures that all scripts are the same amongst all my modules.
This solution isn't ideal, as I now have a huge unmaintainable Git repo. What I would like to do is create a "shared" SQL project, that I then create a nuget package of and the other modules will then use this "SQL Class library" as a nuget package.
I am unable to find a project of this type, and wondering if there's right/other way of approaching this problem
Related
I use EPPlus.dll library for generating Excel files on the fly for attachment and generate mailer in Script Task of SSIS package.
When there is new requirement for change in mailer comes, I do the change in Script Task on my local machine and send the built package file ( .dtsx ) to DBA team for deployment.
Now everytime I have to ask the DBA team to share production server screen with me where I:
Open the file in Visual Studio Data Tools solution
Browse to the dll location
Add Reference to the dll in Script Task.
Then they import the package file in MSDB from where scheduled job references and executes the package.
If I dont do the above step, the Script Task throws error of reference not found.
Error 1 The type or namespace name 'OfficeOpenXml' could not be found
(are you missing a using directive or an assembly reference?)
I overcame challenge of installing DLL inside GAC that is being referenced in Script Task by dynamically loading the assembly as below'
public void Main()
{
AppDomain.CurrentDomain.AssemblyResolve += new ResolveEventHandler(CurrentDomain_AssemblyResolve);
}
private System.Reflection.Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
{
return System.Reflection.Assembly.LoadFrom(System.IO.Path.Combine(strDLLPath, "EPPlus.dll"));
}
But I am, not able to find steps to avoid manually browsing and adding DLL reference. Please help as DBA team is reluctant/ avoids sharing screen.
Alternatively, what is the correct/best practice way to get package file deployed on server in which external dll is used if I dont have direct access to production server.
I don't think there is a direct way to do that from integration services packages, because the only workaround you can do it is - (what you have done using CurrentDomain_AssemblyResolve function) - loading an assembly from a location instead of installing it into GAC.
There are 3 ways for using custom dll's on deployment:
Assigning DLL's to the GAC
Using the AssemblyResolve Function
Copy all Dll's to the sql server DTS Assemblies folder (example for SQL Server 2008: C:\Program Files\Microsoft SQL Server\100\DTS\Binn) and to the .Net framework assemblies folder.
if the problem is to ask for screen sharing, you can create a small installation wizard that copy these dlls to the specific location and ask the dba team to execute it.
Workaround
When searching for this issue, i found an interesting workaround by using a Web Service instead of direct dll, so you can create a web service which contains the methods you are using and add a Web reference instead of a Local assembly
Side Note: i didn't try this approach before, but i am only trying to help
Useful Links & References
SSIS custom DLLs during deployment
How to load an Assembly in a SSIS script task that isn’t in the GAC
SQL Server SSIS Custom DLL Folders
Using Custom DLL's in an SSIS Script Task
SSIS Script Task cant find reference to assembly
Referencing Other Assemblies in Scripting Solutions
Access WebService via SSIS Script Component
Calling a secure webservice in SSIS through script task
Consume Webservice via SSIS Script Component
HOW TO: Write a Simple Web Service by Using Visual C# .NET
I have been doing file deployment and database deployment in sql server 2008.I have gone through the ssis(on sql 2014) and seen only catalogue based deployment.
Just wondering ,is there file deployment still available in ssis 2014.
can not find any file deployment option in deployment wizard.
There is an option called convert to package deployment.Does it mean the file deployment is only available under the package deployment option .
If i select the "Convert to package deployment" option,then the project deployment is not going to work in the future!
Also,Is there any concept of configuration in ssis 2014!
In ssis 2008 ,we have a file called configuration file and it contains all the
connection information.After the package deployment,we can change all the package connection information in this single file ,instead of opening and doing it in multiple files
There are two deployment models available to SSIS from the 2012 release going forward: Project Deployment Model (new, default) and Package Deployment Model (classic).
Which deployment model am I using?
The easiest way is to look at Solution Explorer. If you see a node under there specifying Project.params, then you are in the Project Deployment Model (left half of the image). Otherwise, you should see a "Data Sources" folder along with the project title's giant hint (package deployment model) (right half of the image)
Is File Deployment still a thing?
Package Deployment Model allows you to keep your process if you like your process. Happy deploying to the file system? Great, keep it up. Enjoy explicitly turning on logging and configuration - it's all yours. Just convert to the package deployment model and it's all there.
So SSIS 2012 requires all your packages and your project to have the same ProtectionLevel. We have a workflow where we can't store secrets in our packages but configure all our connection strings at runtime. T
The tricky part is debugging these packages in Visual Studio requires you to either exclude all packages you are not working on and manually setting ProtectionLevel on the working packaging and project, or the more cumbersome approach of setting ProtectionLevel on all packages every time. In a project with 20 packages like we have this is a chore. Anybody have any good workflow for debugging SSIS packages while maintining a DoNotSaveSensitive ProtectionLevel?
I would create Project Parameters to store the connection strings:
http://technet.microsoft.com/en-us/library/hh213214.aspx
Are you working in project deployment model?
https://msdn.microsoft.com/en-us/library/hh213290.aspx
It should just work this way.
All packages same level and you set the connection strings in the environment on the server. I deploy mine to prod with localhost connection using Windows auth and the server magically does the rest.
Can you just create a test project with just the package you want to test? Then you don't have to change protection level for each package.
The simple way is to have two projects in Visual Studio: one with DontSaveSensitive and another one with the protection level desired.
I have found two modules that save their module specific settings without using sql server.
After the module finish installing the installation log shows that there was no sql provider scripts and when I check the DNN database tables there is no new table created.
One of the modules in question is a free module to create a popup window.
http://www.codingstaff.com/products/dotnetnuke-modules/free-dnn-pop-up
So my question is: Is there any form of storage inside DotNetNuke that a module can use to persist their settings other than SQL Server?
thank you.
Modules can store their settings in ModuleSettings or TabModuleSettings tables that are part of the DNN installation.
But to answer your question, there is no other form of persisting settings in DotNetNuke besides using sql.
I'm just trying to understand SSIS packages a bit better and how they are deployed. Correct me I'm wrong but for any deployment, I believe there needs to be at least two files a .SSISDeploymentManifest and a .dtsx. The .SSISDeploymentManifest acts as the equivalent windows installer package which points to the .dtsx. The dtsx is the actual package of "stuff" that is referenced as an external file some how when you run the installer. When you install it, the package gets added to a list of ssis packages for that instance.
My further questions:
If i wanted to keep previous version of the same package, can I just copy the bin directories with the two above files and keep separately should I need to roll back to a previous package?
Where are these packages installed to? How does SSIS know where the packagess are?
Correct me I'm wrong but for any deployment, I believe there needs to
be at least two files a .SSISDeploymentManifest and a .dtsx. The
.SSISDeploymentManifest acts as the equivalent windows installer
package which points to the .dtsx. The dtsx is the actual package of
"stuff" that is referenced as an external file some how when you run
the installer. When you install it, the package gets added to a list
of ssis packages for that instance.
Your assumptions are mostly correct. You don't need the deployment manifest, but it can be handy. Also, you don't need to deploy to the SQL Server instance. You have the option to deploy to the file system as well. I'll explain both below.
Regarding your 1st question:
Version Control:
Make sure you're developing and checking in your dtsx packages via visual studio. Label your releases in sourcesafe or whatever version control you're using. If you are checking in and labeling, then you should be able to easily roll back to a previous version. As you mention, you also can just save a copy of your old bin directory but naturally put them in dated subfolders or something. However, this does not take the place of proper version control.
Regarding your 2nd question:
Deployment:
As the other poster states, you first have a decision to make:
a) Deploy packages to the file system
b) Deploy packages to MSDB
There are benefits to each, and everyone has their preference. I have used both, but I prefer the filesystem because it's more transparent, however there is more to maintain.
See this post for much more on this: http://blogs.conchango.com/jamiethomson/archive/2006/01/05/SSIS_3A00_-Common-folder-structure.aspx
The code is in the dtsx package. Generally,in order to make your packages portable you also abstract your connection strings and other configurable information into a config file (.dtsconfig) or environment variable (no file needed). See BOL to learn more about configuration.
The manifest file contains metadata about which dtsx and config files to install. If you open one, you'll see it's a simple readable xml file.
The manifest file makes it easy to hand over to a DBA to deploy (ask them to double-click the manifest file and follow directions, but they'll need instructions.
To me, the manifest file is more useful for deploying to SQL Server than to the file system. Really, all it does is make a copy of the dtsx and config files and puts them where you tell it. You could just as easily instruct the DBA to copy your dtsx files to a common folder on the server, and the config files to another folder on the same server.
Then when you schedule your jobs using SQL Agent, you specify that you're going to run an SSIS package that is stored on the file system and browse to where it's located. If you're using configurations, then there's a tab to specify where the config file is located.
There is so much to know about configuring/deployment/versioning of SSIS packages. But hopefully this will get you started on the right path.
When you export your DTS packages using the Import/Export Wizard in SQL Server you have the option of saving them to SQL Server or locally on the file system.
Regarding the versions of your SSIS packages, you need to query SSISDB to extract the version numbers. It's annoying this kind of info isn't shown directly in the Management Studio but, until it is, someone may find this useful:
SELECT prj.[name] as Project
,pkg.[name] as Package
,pkg.[version_major]
,pkg.[version_minor]
,pkg.[version_build]
FROM [SSISDB].[internal].[packages] as pkg
JOIN [SSISDB].[internal].[projects] as prj
ON pkg.[project_id] = prj.[project_id]
ORDER BY prj.[name]