I am trying to create a package containing several Data Flow tasks in them. The tasks are fairly similar in nature, but contain fairly important differences.
I have tried to copy the task, then change the things which need changing.
When I run the task by itself it runs fine, however when I run it with all the other tasks in the package I get the below error:
Object reference not set to an instance of an object. at
ScriptMain.Input0_ProcessInputRow(Input0Buffer Row) at
UserComponent.Input0_ProcessInput(Input0Buffer Buffer) at
UserComponent.ProcessInput(Int32 InputID, String InputName,
PipelineBuffer Buffer, OutputNameMap OutputMap) at
Microsoft.SqlServer.Dts.Pipeline.ScriptComponent.ProcessInput(Int32
InputID, PipelineBuffer buffer) at
Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.ProcessInput(Int32
inputID, PipelineBuffer buffer)
Not the most friendly error message.
Can anyone tell me what this is and how to fix it? I assume that there is some variable or other attribute which is being repeated, but which one?
Note that many of the columns over the several data flow tasks will have the same column names.
I figured it out in the end. The reason was that the second level objects need to be explicitly declared.
I had
public class Level2
{
public string Somevalue { get; set; }
}
public class RootAttributes
{
public Level2 lvl2 { get; set; }
public string Somevalue2 { get; set; }
}
It should have been
public class Level2
{
public string Somevalue { get; set; }
}
public class RootAttributes
{
public Level2 lvl2 = new Level2;
public string Somevalue2 { get; set; }
}
The weird thing was that the top method worked in several other places.
I had this issue with my send mail task.
As I was using a file connection for my body of the email, I had to create a new file connection and it worked fine.
Use the package run report in vs 2012 to identify in which DFT task package got failed.
Use the view code option of the package to see the whether all the DFT task having proper input.
Related
I used mongoexport to export the collection in to csv file along with fields. However when I tried to import the .csv in sql server using SSIS. I got errors and the data in preview section before executing the package was wrong. Can any one please guide me how can I export the data properly which can be easily imported into sql server. I am ready for minor tuning like adding id column or changing data types etc.
So it looks like you are getting a json formatted object for each row in the file. It may be different than that, and if that's the case, how you iterate through each might change a little. But here is something to get you started:
1 - There is no out of the box JSON parser in .NET, but this seems to be a popular utility: http://www.newtonsoft.com/json
2 - If you download the parser above, you'll have to go through a little pain to get it in a useable state with SSIS:
Go to the source folder and open the solution for net40
Sign the assembly
comment out the lines that cause build errors
//[assembly: InternalsVisibleTo("Newtonsoft.Json.Schema")]
//[assembly: InternalsVisibleTo("Newtonsoft.Json.Tests")]
install the assembly to the gac
3 - Once all that is out of the way, add a file connection manager to your package and point it to the mongdb file
4 - Add a dataflow and then add a script component source to the dataflow
5 - In the script component, configure the connection manager that you created in step three, I gave mine the friendly name "MongoDbOutput"
6 - In the Inputs and Outputs section, go to Output0 and add a column for each field in the JSON object, setting datatypes to string as they will be int by default
7 - Open the script and add a reference to Newtonsoft.Json and System.IO
8 - The script below shows how to access the connection string to the file in the connection manager, read the file with a streamreader one line at a time and then parse each JSON object for each address. One line is added for every address and the name and ssn are repeated for each line.
Note also, that I added a person and address class. The json.net is pretty cool, it will take that json and push it into the object - provided all the fields match up. Goog luck!
using System.IO;
using Newtonsoft.Json;
public override void CreateNewOutputRows()
{
object MongoDbPath = Connections.MongoDbOutput.AcquireConnection(null);
string filePath = MongoDbPath.ToString();
using (StreamReader fileContents = new StreamReader(filePath))
{
while (fileContents.Peek() >= 0)
{
var contents = fileContents.ReadLine();
Person person = JsonConvert.DeserializeObject<Person>(contents);
foreach (address address in person.addresses)
{
Output0Buffer.AddRow();
Output0Buffer.name = person.name;
Output0Buffer.ssn = person.ssn;
Output0Buffer.City = address.city;
Output0Buffer.Street = address.street;
Output0Buffer.country = address.cc;
}
}
}
}
public class Person
{
public string name { get; set; }
public string ssn { get; set; }
public address[] addresses
{ get; set; }
}
public class address
{
public string street { get; set; }
public string city { get; set; }
public string cc { get; set; }
}
I'm developing a windows service that reads information from the app.config at start-up which should allow us to change internal thread configuration without redeploying the service.
I created some custom configuration sections and elements as follows (implementation omitted):
public class MyConfigurationSection
{
[ConfigurationProperty("threads")]
[ConfigurationCollection(typeof(MyThreadCollection), AddItemName="addThread")>
public MyThreadCollection threads { get; }
}
public class MyThreadCollection
{
protected override void CreateNewElement();
protected override object GetElementKey(ConfigurationElement element);
}
public class MyThreadElement
{
[ConfigurationProperty("active", DefaultValue=true, IsRequired=false)>
public bool active { get; set; }
[ConfigurationProperty("batchSize", DefaultValue=10, IsRequired=false)>
public int batchSize { get; set; }
[ConfigurationProperty("system", IsRequired=true)>
public string system { get; set; }
[ConfigurationProperty("department", IsRequired=true)>
public string department { get; set; }
[ConfigurationProperty("connection", IsRequired=true)>
public MyThreadConnectionElement connection { get; set; }
}
public class MyThreadConnectionElement
{
[ConfigurationProperty("server", IsRequired=true)>
public string server { get; set; }
[ConfigurationProperty("database", IsRequired=true)>
public string database { get; set; }
[ConfigurationProperty("timeout", DefaultValue=15, IsRequired=false)>
public int timeout { get; set; }
}
Then I add some elements to the app.config as follows:
<configurationSection>
<threads>
<addThread
active="True"
batchSize="50"
system="MySystem1"
department="Department1">
<connectionString
server="MyServer"
database="Database1" />
</addThread>
<addThread
active="True"
batchSize="30"
system="MySystem2"
department="Department2">
<connectionString
server="MyServer"
database="Database2" />
</addThread>
</threads>
</configurationSection>
Everything works - configuration is read, threads are created, and the processes run.
The problem is, I would like both these threads to have the same system name/value -- both should be MySystem -- but when I do that and run the program, I get a The entry 'MySystem' has already been added. exception.
I figured it might be because a property has to be explicitly configured to allow duplicates, but I don't know how and I couldn't find a property of the ConfigurationProperty class that might allow that, other than IsKey, but from its description it didn't seem like the answer, and trying it didn't solve the problem. Am I on the right track here?
Initially the system property was named name and I though that just maybe any property named name is treated as a unique identifier, so I changed it to system but it didn't change anything.
I tried the <clear /> tag as some other, similar posts suggested, without success.
Do I need to add another hierarchy to the configuration section -- Config -> Department -> Thread instead of Config -> Thread? I'd prefer to not take this approach.
Thanks for any and all input.
I actually found the problem and solution quite some time ago, but forgot to post the answer; thanks #tote for reminding me.
When implementing the ConfigurationElementCollection class, the GetElementKey(ConfigurationElement) method can be overridden. Without immediately realising what the method is for I overrode it and simply returned the system property value, and, since more than one configuration element had the same system name, technically they had the same key, which is why the error occurred.
The solution for me was to return the system and the department values as system.department which resulted in unique keys.
I am trying to learn EF, C# and MVC at the same time so I apologise if this is a bit basic.
I am trying to get a result set from a SQL Server stored procedure into a model.
I have created a EDMX file and have stored procedure imported and a result set exists in a cs file - see below.
public partial class prBGGetYourTasks_Result
{
public int CompanyTaskID { get; set; }
public int TaskID { get; set; }
public string TaskName { get; set; }
public string TaskDescription { get; set; }
public System.DateTime StartDate { get; set; }
public System.DateTime EndDate { get; set; }
public string TimeLeft { get; set; }
public string Status { get; set; }
}
I have then created a class in models folder with the exact definition called Task.
I am trying to populate the model with results of the SP with this code:
public class TaskList : List<Task> {
public TaskList GetTasks()
{
BGEntities BGDB = new BGEntities();
TaskList task_list = new TaskList();
task_list = BGDB.prBGGetYourTasks(HttpContext.Current.User.Identity.Name).ToList();
return task_list;
}
}
However it is coming up "cannot implicitly convert type 'Genric.List to 'Task.TaskList'. I have tried trying to a variable of the result type and also casting into result set type.
Please advise if I am also going about it the wrong way.
I have previously used a sql reader and loop to populate the model but I figure there has to be a better way.
Your issue is here:
TaskList task_list = new TaskList();
task_list = BGDB.prBGGetYourTasks(HttpContext.Current.User.Identity.Name).ToList
You're trying to assign a List<Task> to a TaskList. Inheritence does not work this way.
List<Task> tasks = new List<Task>();
TaskList taskList = new TaskList();
//This will compile
tasks = taskList;
//This will not
taskList = tasks;
You may want to get rid of TaskList altogether and just use List<Task>, it doesn't seem like you're adding any a new members of behavior to it.
My guess is your Function Import within the Data Model is set to return a collection of ComplexTypes not your Entity type as you are wanting.
To fix. Double click your function(stored procedure) in Data Model-> Function Imports. Within the dialog box, you will find a section entitled "Returns a collection of". Select your entity type from the dropdown Entities:. Click OK. This one frustrated me for a while too.
Hopefully this helps someone looking for an answer. Really hope the original post has found a work around that works for them!
Is there a way to convert data in an existing database into objects that can easily be put into a seed method?
Essentially I'd like to let some people add lines to my DB via a form and then convert that into an object that I can re-seed the DB anytime I need to make changes.
The database itself is created via code-first using the following model:
public class Combo
{
public int Id { get; set; }
public string MainPrefix { get; set; }
public string MainDescriptor { get; set; }
public string MainDish { get; set; }
public string Connector { get; set; }
public string SecondaryDescriptor { get; set; }
public string SecondaryDish { get; set; }
}
High level untested idea:
Create a new T4 template which will use either your context or direct SQL to query database and generate either C# static class with method (which will return collection of object created from database data) or SQL insert commands. Depending on what you choose you will either call static method in Seed to get all entities you want to insert or simply load the created SQL script and execute it.
You can also do the same without T4 by using Code Dom (if you want to generate C# code) and creating a new custom tool for visual studio.
These features can be part of your project or the result (C# code) can be compiled in separate external assembly.
I am struggling with returning a complex type from my services layer. It doesnt seem to be accessible from my object context.
This is the query in the service layer. All compiling fine.
public IQueryable<USP_GetPostsByThreadID_Result> uspGetPostsByThreadID(int ThreadID)
{
return this.ObjectContext.USP_GetPostsByThreadID(ThreadID).AsQueryable();
}
When I try and call it from my client, the ForumContext is not seeing it. I checked the client generated file and nothing similar is being generated. Help!!!
The name of your method may not meet the expected convention for queries. Try one or both of the following:
Add the [Query] attribute
Rename the method to GetUspPostsByThreadID
Result:
[System.ServiceModel.DomainServices.Server.Query]
public IQueryable<USP_GetPostsByThreadID_Result> GetUspPostsByThreadID(int ThreadID)
{
return this.ObjectContext.USP_GetPostsByThreadID(ThreadID).AsQueryable();
}
Its very common to have a stored procedure returning data from multiple tables. The return type doesn't fit well under any of the Entity Types(Tables). Therefore if we define Complex Type as the return collection of objects from Stored Procedure invocation, it becomes quite a powerful tool for the developer.
Following these steps I have achieved successfully the configuration of complex type on a sample AdventureWorks database.
1. Refer the picture and ensure the Stored procedure and function import is done.
2. Add the Domain Service name it as AdventureDomainService.
3. Now its time to define the tell the RIA services framework to identify my Complex Type as Entity Type. To be able to do this, we need to identify a [Key] DataAnnotation. Entity types provide data structure to the application's data model and by design, each entity type is required to define a unique entity key. We can define key on one property or a set of properties in metadata class file AdventureDomainService.metadata.cs
First define the class then add MetadatatypeAttribute like :
[MetadataTypeAttribute(typeof(CTEmployeeManagers.CTEmployeeManagersMetadata))]
public partial class CTEmployeeManagers
{
internal sealed class CTEmployeeManagersMetadata
{
private CTEmployeeManagersMetadata() { }
[Key]
public int EmployeeID { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public int ManagerID { get; set; }
public string ManagerFirstName { get; set; }
public string ManagerLastName { get; set; }
}
}
Define the Domain service method to return the collection of objects/entities for populating the Silverlight Grid or any other data consuming controls.
public IQueryable<CTEmployeeManagers> GetEmployeeManagers(int empId)
{
return this.ObjectContext.GetEmployeeManagers(empId).AsQueryable();
}
We define IQueryable if we are to fetch the records from datasources like SQL, whereas we define IEnumerable if we are to fetch the records from in memory collections,dictionaty,arrays.lists, etc.
Compile the server side to generate the client proxy.
In the Silverlight side open the MainPage.xaml or wherever the datagrid is put, then add following namespaces :
using System.ServiceModel.DomainServices.Client;
using SLBusinessApplication.Web;
using SLBusinessApplication.Web.Services;
..
Load the data and display:
public partial class MyPage : Page
{
AdventureDomainContext ctx = new AdventureDomainContext();
public MyPage()
{
InitializeComponent();
LoadOperation loadOp = this.ctx.Load(this.ctx.GetEmployeeManagersQuery(29));
myGrid.ItemsSource = loadOp.Entities;
}
// Executes when the user navigates to this page.
protected override void OnNavigatedTo(NavigationEventArgs e)
{
}
}
That is all that is needed to do.
It has to be part of an entity. Complex types cannot be returned by themselves