Existing SSIS Packages Suddenly fail to Validate custom components XML - sql-server

We have been using and running SSIS packages for SQL Server 2014 with custom components that have been used since before I arrived at my work place. We upgraded to Visual Studio 2017, and have been fine doing so for months. Recently--maybe in line with a recent update--Visual Studio ssdt fails to load these packages because of errors regarding validating the xml behind these components. The packages continue to run on the server, and creating new packages with these custom components still works as expected. The packages also still open in Visual Studio 2013. We've gone back to earlier versions of the same package in code control. Reinstalled earlier versions of Visual Studio, installed the ssdt tools as a stand-alone, but nothing seems to work.
There is a slew or errors regarding being unable to load values that are part of the definition of these components, and bad references to these in the precedence constraints because of the failure to load these components in the first place, like:
Error loading CCMI Call Import.dtsx: Error loading value "<DTS:ForEachEnumerator xmlns:DTS="www.microsoft.com/SqlServer/Dts" DTS:CreationName="FolderEnumerator" DTS:DTSID="{FACFAC12-F5E1-4BFE-9768-BF73D8053550}" DTS:ObjectName="{FACFAC12-F5E1-4BFE-9768-BF73D8053550}"><DTS:PropertyExpression DTS:Name="Directory">#" from node "DTS:ForEachEnumerator". C:\Users\..Call Import.dtsx 1
When I remove all references to the custom components in the precedence constraints, it will attempt to load the package without the "corrupted" pieces, but because these custom components are containers, in some cases this leaves out most of the work that's been done. I've removed some of these "bad" values it can't load on a lark, but it keeps erroring on the next attribute still associated with the container after I remove the one previously referenced in the error.
I can't open the package, so I cant copy and paste the components into a new package.
Anyone have any ideas?

I was able to fix some of the issues after some study of the XML of these custom components, and wanted to share in case it helped someone, although I doubt the exact same issues I came across would affect many other people.
I wasn't getting any useful error messages for a custom foreach loop container, but when I tried to copy it from a working VS 2013 screen into VS 2017, I did finally get an error stating something like (lost the exact quote) "missing attribute for the server target version". I then found that the working examples of the custom component that worked had this node that was missing from the XML that was "broken":
<TargetServerVersion
Type="3"
Value="120" />
I added the missing attribute and changed the value from 140 to 120 (the version we were actually targeting) and it began to work just fine.
Fixed another custom destination through a similar process, although the error was entirely different. In this scenario, the package would actually open without error, but the custom destination would not function properly (and also no longer had the custom icon). The working package I created with the custom component had an XML that was similar to the broken package's XML in this case. It wasn't until I created a new package in a new project where I immediately set the correct target server that I could see any change in the XML between the broken and working package's code, and it was this:
The 2017 version of the component had a much longer definition of the "UserComponentTypeName" property:
<properties>
<property
dataType="System.String"
name="UserComponentTypeName">Konesans.Dts.Pipeline.TrashDestination.Trash, Konesans.Dts.Pipeline.TrashDestination, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b2ab4a111192992b</property>
</properties>
The correct version for 2016 (I know, different from the last package) had a shorter, much simpler definition:
<properties>
<property
dataType="System.String"
name="UserComponentTypeName">Konesans.TrashDestination</property>
</properties>
It worked correctly when I replaced the property with the shorter definition.
I don't know how frequently these exact issues would come up, but I hadn't seen anything like this while searching for solutions to my problem and thought if someone had a different but related problem with XML changing (perhaps to someone accidentally upgrading a package version?) that it might help add a spark to the creative process and save some time.

Related

SQL Server Database Project dbml file is always modified [duplicate]

I'm running into a small but weird annoyance that seems to be happening to other people, too (for example, check out the revision history of SEDE). I have a SQL Server 2008 database project in Visual Studio 2010 that works properly. When I go to commit/checkin to source control, I'm told that my DB project's .dbmdl file has changed, even when I've made no changes to the project!
I'm not sure if the changes are triggered by building my solution (which also includes an ASP.NET MVC application and a unit test project) or by simply opening the DB project, but this is getting kind of annoying and is creating clutter in source control.
Is it possible to stop these changes from occurring, or get rid of the .dbmdl file whatsoever?
If you delete the .dbml file, it is rebuilt without errors or warnings, so I think it's just a cache file for references, intellisense, etc. I'm going to exclude it from source control.
I believe Visual Studio serializes the dbmdl file every time the project is opened. The only possible work around would be to keep the project open.
See related question here. The .dbmdl file is unique per user (and some kind of cache, as said above) and so the right solution is indeed to exclude it from source control.

SSIS Control Flow Diagram is now empty

I had a working SSIS package last week. I came back into the office this morning, opened up the package and tried to open up one of the Data Flow tasks. Double clicking did nothing. I closed the project and reopened it and received an error message that there was a "Catastrophic Error" and that the layout could not be displayed. The Control Flow diagram now only showed my two Sequence Containers.
I made a copy of the dtsx file then went into it manually with a text editor and found that some of the XML was duplicated. I removed the "extra" copy and reopened my package. The empty Sequence Containers are now gone, but I still can't see any of my components. I looked at the Package Explorer and everything is still there, but it's just not displaying in the designer. I was even able to run the package successfully.
Is there any way to rebuild the package so that I can see everything in the designer? Any other suggestions?
Thanks!
I'm not sure why BIDS has decided the layout is junk but you can get around the issue by removing the layout and forcing BIDS to regenerate that information. The graphical layout stuff is a cool idea for conceptualizing how a package is organized but the implementation of storing that information, serialized XML inside XML, is pants.
There are various articles available about how to remove the XML, but this is a more recent example of how to use do this. http://joshrobi.blogspot.com/2012/04/editing-existing-ssis-package-via-ezapi.html
As always, ensure you have a good copy of the package before editing the XML directly.
Do you have a backup of your package - try and restore that to another location first to make sure that is not corrupted.
How are you accessing the package, are you connecting remotely to a server hosting SSIS, or are you using tools located locally on the server?
I've encountered issues when trying to access packages across networks before.
Also, as a word of warning it's not good practice to edit package contents before carrying out further diagnosis!

Using a test/dev/prod database strategy with Entity Framework

When working in Ruby (specifically, in Rails), I can automatically run my tests using a testing database, and also choose easily between a development or production database. Still new to Entity Framework (WPF), but it seems less than simple.
My Entity assembly has an App.Config file that holds a reference to the database, and I need to copy this App.Config file to all runnable projects (e.g. questions 1113361 and 2233897).
If I use Test->Database Test Configuration without explicitly copying App.Config from my EF project, I get
System.ArgumentException: The
specified named connection is either
not found in the configuration, not
intended to be used with the
EntityClient provider, or not valid.
which is the same as if I have a missing or different App.Config file.
Strangely even if I do change all the connection stings in my Entity project and testing project to the same database, I still get that error.
Is there a step I'm missing when telling Visual Studio (2010 Ultimate) that I want my tests to run using a different testing database, or is this just not supported with EF 4?
Also, Is there some way to change database context other than copying App.Config files back and forth? Seems like a serious way to ignore separation of concerns if not, so I think I'm missing something.
Strangely, as I was playing around after doing this, it started working. Somebody might comment on this, but it looks like I was using Test->Database Test Configuration at the wrong time.
To wit:
Clean project, no App.Config, use Test->Database Test Configuration to set the test database. It generates an app.config (lowercase, notably). You get an error. Even if you check all the connection strings (weird)
Clean project, first copy the App.Config from from you Entity project. Now when you use Test->Database Test Configuration, it will populate the App.Config that you copied, and it works.
I imagine it was some issue I was creating when copying the connection strings.
I've also confirmed that this works with my Entity project using a "Main" database while my testing project uses a "Test" database.
Wonder if someone can confirm/clarify this?

Why does my DB project's .dbmdl file change even when I make no changes to the project?

I'm running into a small but weird annoyance that seems to be happening to other people, too (for example, check out the revision history of SEDE). I have a SQL Server 2008 database project in Visual Studio 2010 that works properly. When I go to commit/checkin to source control, I'm told that my DB project's .dbmdl file has changed, even when I've made no changes to the project!
I'm not sure if the changes are triggered by building my solution (which also includes an ASP.NET MVC application and a unit test project) or by simply opening the DB project, but this is getting kind of annoying and is creating clutter in source control.
Is it possible to stop these changes from occurring, or get rid of the .dbmdl file whatsoever?
If you delete the .dbml file, it is rebuilt without errors or warnings, so I think it's just a cache file for references, intellisense, etc. I'm going to exclude it from source control.
I believe Visual Studio serializes the dbmdl file every time the project is opened. The only possible work around would be to keep the project open.
See related question here. The .dbmdl file is unique per user (and some kind of cache, as said above) and so the right solution is indeed to exclude it from source control.

What does this WCF error mean: "Custom tool warning: Cannot import wsdl:portType"

I created a WCF service library project in my solution, and have service references to this. I use the services from a class library, so I have references from my WPF application project in addition to the class library. Services are set up straight forward - only changed to get async service functions.
Everything was working fine - until I wanted to update my service references. It failed, so I eventually rolled back and retried, but it failed even then! So - updating the service references fails without doing any changes to it. Why?!
The error I get is this one:
Custom tool error: Failed to generate code for the service reference
'MyServiceReference'. Please check other error and warning messages for details.
The warning gives more information:
Custom tool warning: Cannot import wsdl:portType
Detail: An exception was thrown while running a WSDL import extension:
System.ServiceModel.Description.DataContractSerializerMessageContractImporter
Error: List of referenced types contains more than one type with data contract name 'Patient' in
namespace 'http://schemas.datacontract.org/2004/07/MyApp.Model'. Need to exclude all but one of the
following types. Only matching types can be valid references:
"MyApp.Dashboard.MyServiceReference.Patient, Medski.Dashboard, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" (matching)
"MyApp.Model.Patient, MyApp.Model, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" (matching)
XPath to Error Source: //wsdl:definitions[#targetNamespace='http://tempuri.org/']/wsdl:portType[#name='ISomeService']
There are two similar warnings too saying:
Custom tool warning: Cannot import wsdl:binding
Detail: There was an error importing a wsdl:portType that the wsdl:binding is dependent on.
XPath to wsdl:portType: //wsdl:definitions[#targetNamespace='http://tempuri.org/']/wsdl:portType[#name='ISomeService']
XPath to Error Source: //wsdl:definitions[#targetNamespace='http://tempuri.org/']/wsdl:binding[#name='WSHttpBinding_ISomeService']
And the same for:
Custom tool warning: Cannot import wsdl:port ..
I find this all confusing.. I don't have a Patient class on the client side Dashboard except the one I got through the service reference. So what does it mean? And why does it suddenly show? Remember: I didn't even change anything!
Now, the solution to this was found here, but without an explanation to what this means. So; in the "Configure service reference" for the service I uncheck the "Reuse types in the referenced assemblies" checkbox. Rebuilding now it all works fine without problems. But what did I really change? Will this make an impact on my application? And when should one uncheck this? I do want to reuse the types I've set up DataContract on, but no more. Will I still get access to those without this checked?
I found my answer here: http://www.lukepuplett.com/2010/07/note-to-self-don-let-wcf-svcutil-reuse.html
Long story short: I unchecked Reuse types in reference assemblies from the Advanced menu.
I don't know if this matters but i'm not using MVC, but Web Forms.
When you add a service reference, there are two ways the types that are used by the service can be handled:
The types are stored in a dll, and that dll is referenced from both the client and the server application.
The types are not in a dll referenced by the client. In that case the tool that creates the service reference, will create the types in the references.cs file.
There are many things that can go wrong. We have found that if the tool crashes, it is sometimes faster to delete the service reference and start again.
We have stopped using service reference. For projects where we have control of the client and the service, we use the method described in this screencast.
I also had this issue Today. It took me one entire day to find my mistake. Hope it helps.
My class that weren't able to be imported has a cutom enum type property. This property is marked as DataMember and the Enum is also marked as DataContract. Everything fine so far.
I just forgot to mark every enum member as EnumMember.
So i changed
[DataContract]
public enum SortMethodType
{
Default = 0,
Popularity = 1,
ReleaseDate = 2,
PublishedDate = 3,
TranslatedTitle = 4,
OriginalTitle = 5,
UserRating = 6,
Duration = 7
}
To this:
[DataContract]
public enum SortMethodType
{
[EnumMember]
Default = 0,
[EnumMember]
Popularity = 1,
[EnumMember]
ReleaseDate = 2,
[EnumMember]
PublishedDate = 3,
[EnumMember]
TranslatedTitle = 4,
[EnumMember]
OriginalTitle = 5,
[EnumMember]
UserRating = 6,
[EnumMember]
Duration = 7
}
And it finally worked!
Go to Advanced properties while adding reference and remove "System.Window.Browser" from the checklist, It solves the problem.
that might sound weird, but I got it fixed by deleting the references, then closing Visual Studio, and reopening it again, and finally adding the references again.
I think the custom tool thing needed to be restarted or something.
I constantly run across this error while it works on another developers machine. Even though I'm a full admin everywhere in my virtual machine, I tried closing Visual Studio, and re-opening with 'Run As Administrator' and it magically worked.
Good luck.
I got the warning after upgrading my solution from Visual Studio (VS) 2010 to 2013 and changing each project's .NET Framework from 4 to 4.5.1. I closed VS and re-opened and the warnings went away.
One downside of turning off 'reuse types in referenced assemblies' is that it can cause issues with ambiguous references. This is due to the service reference creating those objects again in the reference .cs file, and your code implementing the service may be referencing them from the original namespace.
When this scenario occurs I find it useful to check the 'reuse types in specified referenced assemblies' which allows me to choose the ones with ambiguous references only, which resolves the issue quickly that way.
Hope it helps someone else.
My interfaces of the WCF service are in an assembly, the implementation is in an another and the service reference is in yet another assembly, separate from the clients of the service reference. I got the error message right after I applied the DataContract to an enum. After I applied EnumMember to the fields of the enum, the issue resolved.
If in doubt that your service doesn't have any problems (such as problems with enums, or non-serializable classes as mentioned by others) then try to create a new project with a new reference.
I am using Silverlight 5 and I had tried to delete and recreate the reference several times. The reference.cs file just came up completely empty each time and it had been literally years since I'd created it so trying to figure out what had changed in the service was out of the question.
I noticed that the error contained references to 2.0.5.0. Now I don't even know if this is actually relevant to the Silverlight version, but it made me think of just creating a brand new project and then suddenly everything worked.
Warning 2 Custom tool warning: Cannot import wsdl:portType Detail: An
exception was thrown while running a WSDL import extension:
System.ServiceModel.Description.DataContractSerializerMessageContractImporter
Error: Could not load file or assembly 'System.Xml, Version=2.0.5.0,
Culture=neutral, PublicKeyToken=7cec85d7bea7798e' or one of its
dependencies. The system cannot find the file specified. XPath to
Error Source: //wsdl:definitions[#targetNamespace='']/wsdl:port
Type[#name='IShoppingCart']
I was looking over my project and I was having this same issue. It turned out to be different versions of the same DLL on the WCF vs. Web Site.
Web site had a newer version of the DLL and the service was referencing an older version of the DLL. Once they were all in sync all worked well.
I experienced the same error. I struggled for almost a day trying to find out what was going wrong. The clue for me were the warnings that VS was throwing. It was trying to do some kind of mapping to Yahoo.Yui.Compressor.dll, a library I had added and removed (because I decided not to use it) a couple of days before. It was shocking because the library wasn't there, but somehow it was trying to reference it.
Finally, I restore this dll from the Trash, and then I could update my service reference successfully.
For anyone here in the future, I had the same error but caused by version issues, in two different ways.
I have two WCF services and two client applications that talk via the service references. I updated a nuget package on both sides and tried to update the service reference and got this error.
Deleting didn't help. Unchecking "reuse assemblies" is not desired as I need to reuse them - that's the whole point.
In the end, there were two separate issues:
1) The first issue, I believe, was a visual studio caching issue. I meticulously went over all of the references and found no issues but it still reported being unable to find the previous version of the file. I uninstalled all of the nuget packages, restarted visual studio, and reinstalled them. Updating the service reference worked.
2) The second issue was caused by a dependency issue. I updated the nuget package on both sides and everything appeared correct, but an unmarked dependency was out of sync. Example:
Package Foo v1 references Bar v1.
It is possible to update Foo and Bar to v2 independently without updating the reference.
If you install both Foo and Bar v2 the service reference tool will scan Foo v2, see the reference to Bar v1, and fail because it can't find the older version.
This is only reported correctly if you update the version numbers of your dll for every package.
Visual Studio and MSBuild will have no problem building the application but the service reference will have a terrible time trying to resolve everything.
I hope this helps someone.

Resources