I am using SSIS 2012 - project deployment model, utilizing the out of the box SSISDB logging.
However, I was wondering how to log custom messages to the SSISDB, i.e. "hello world" when the package start or when a certain event is raised. Is this possible?
Or do I have to fallback to a custom log table, which seems to defeat the advantage of the SSISDB logging.
Thanks!
Of course it is. What message would you like to log? http://msdn.microsoft.com/en-us/library/ms136054.aspx
This is an example from a Script Task (Control Flow)
bool fireAgain = false;
Dts.Events.FireInformation(0, "Test", "I am the description", string.Empty, 0, ref fireAgain);
This is an example from a Script Component (Data Flow)
bool fireAgain = false;
ComponentMetaData.FireInformation(0, "Test too", "I am data flow description", string.Empty, 0, ref fireAgain);
You can then look in your reports or write custom queries against catalog.operation_messages in SSISDB. When we were loading our data warehouse, if I found employee ids that the data didn't support, I'd fire off OnInformation events with the words Research in the title and then parse out the bits I needed (employee id and the date).
Related
In my SSIS Script Component Task (NOT the Script task, which can be added to a Control Flow, but the Script Component Task, which is added within Data Flows), I added some error handling in the catch part of a try, catch block:
// No connection was created. Exit gracefully
bool cancelOnError = false;
ComponentMetaData.FireError(ErrorCode: 10, SubComponent: "SubComponent",
Description: "Couldn't set up the connection. This could be because an invalid host was
provided, or due to a firewall blocking the connection.", HelpFile: "", HelpContext: 0,
pbCancel: out cancelOnError);
This all works fine, with the script task catching the error I raise in my code. I can also see the error in the logs:
Error: 0xA at DFT Extract, SubComponent: Couldn't set up the connection.
This could be because an invalid host was provided, or due to a firewall
blocking the connection.
However, the icon on on the task is green, and a subsequent task I defined to process results gets fired (there is nothing to process, though, as this particular error occurs prior to processing any data):
The Data Flow correctly shows a red cross icon. Is there any way I can change the icon on the Script Component, or a better way for me to (elegantly) simulate a showstopping error?
I found this page on Microsoft, showing the difference between Script Task and Script Components, which also states:
The Script component runs as a part of the Data Flow task and does not
report results using either of these properties.
That doesn't have me very hopeful, but I am hoping someone might have a work around. I primarily think that showing the green icon is somewhat misleading when we trap an error.
Why not just changing the cancelonerror value to true
bool cancelOnError = true;
Or you can use Throw exception method
throw new Exception("Couldn't set up the connection. This could be because an invalid host was
provided, or due to a firewall blocking the connection.");
Workaround
If your goal is to ignore rows that was not processed correctly, the best way is to add a Column to the OutputBuffer (of type DT_BOOL), and set this column value to False if there is an error , else it's value must be True.
And add a conditional split after the script component to filter rows based on this Column. With a similar expression:
[FlagColumn] == True
It's easy to use SQL Server Profiler to trace stored procedures activity. But how to trace SQL queries issued by LINQ via Entity Framework? I need to identify such queries (LINQ code) that consume a lot of time, are called most frequently and therefore are the first candidates for optimization.
Add this key to your connection string:
Application Name=EntityFramework
And filter by this in Profiler
Adding #ErikEJ's answer : if you are using .net Core, so you are using EFCore. There are no Database.Log property. You should use OnConfiguring override of your DbContext class and then
optionsBuilder.LogTo(Console.WriteLine);
Sample :
public class AppDbContext : DbContext
{
public DbSet<User> Users { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.LogTo(Console.WriteLine);
}
}
I've found useful DbContext.Database.Log property.
MSDN article Logging and Intercepting Database Operations
The DbContext.Database.Log property can be set to a delegate for any method that takes a string. Most commonly it is used with any TextWriter by setting it to the “Write” method of that TextWriter. All SQL generated by the current context will be logged to that writer. For example, the following code will log SQL to the console:
using (var context = new BlogContext())
{
context.Database.Log = Console.Write;
// Your code here...
}
What gets logged?
When the Log property is set all of the following will be logged:
The approximate amount of time it took to execute the command. Note that this is the time from sending the command to getting the result object back. It does not include time to read the results.
SQL for all different kinds of commands. For example:
Queries, including normal LINQ queries, eSQL queries, and raw queries from methods such as SqlQuery
Inserts, updates, and deletes generated as part of SaveChanges
Relationship loading queries such as those generated by lazy loading
Parameters
Whether or not the command is being executed asynchronously
A timestamp indicating when the command started executing
Whether or not the command completed successfully, failed by throwing an exception, or, for async, was canceled
Some indication of the result value
I am facing two issues in my SSIS package. The SSIS package I have has an active task that formats an excel sheet as per my requirement and saves it as a different file, modified.xlsx. This file is then used in my data flow task to process and upload data to database tables.
This package works perfectly in my local system, but when I created a scheduled job on my SQL server to run this package it fails with the generic error message "Microsoft (R) SQL Server Execute Package Utility Version 11.0.5058.0 for 64-bit Copyright (C) Microsoft Corporation. All rights reserved. Started: 12:06:55 PM Error: 2016-04-01 12:06:57.06 Code: 0x00000001 Source: Script Task Description: Exception has been thrown by the target of an invocation. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 12:06:55 PM Finished: 12:06:57 PM Elapsed: 1.563 seconds. The package execution failed. The step failed."
To get a more detailed error message i tried to set up logging for the active task.
I configured logging to write log entries for events to a CSV file as in the below screen shot.
I enabled logging for package and checked individuals tasks as well. In active tasks, I added Dts.Log("",0,bytes); to keep track of any exception if any also to log each steps.
public partial class ScriptMain:Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
byte[] bytes = new byte[0];
public void Main()
{
LogMessages("");
LogMessages("Update Bug package execution started at :: " + DateTime.Now.ToLongTimeString());
LogMessages("Loading package configuration values to local variables.");
FileName = Convert.ToString(Dts.Variables["User::ExcelFileName"].Value);
SourceFileLocation = Convert.ToString(Dts.Variables["User::SourceFileLoc"].Value);
SourceFileName = Path.Combine(SourceFileLocation, FileName);
saveLoc = Path.Combine(SourceFileLocation, "ModifiedExcel.xlsx");
var excel = new Excel.Application();
var workbook = excel.Workbooks.Open(SourceFileName);
try
{
foreach (Excel.Worksheet tempSheet in workbook.Worksheets)
{
LogMessages("For loop to check sheet names");
if (((Excel.Worksheet)(tempSheet)).Name.Contains("Test"))
{
if (File.Exists(saveLoc))
{
File.Delete(saveLoc);
}
//File.Create(saveLoc);
tempSheet.Select();
workbook.SaveAs(saveLoc);
}
System.Runtime.InteropServices.Marshal.ReleaseComObject(tempSheet);
}
workbook.Save();
workbook.Close();
excel.Quit();
LogMessages("Quit Excel sheet");
System.Runtime.InteropServices.Marshal.ReleaseComObject(workbook);
System.Runtime.InteropServices.Marshal.ReleaseComObject(excel);
LogMessages("Release excel objects");
}
catch(Exception ex)
{
LogMessages("Exception: " + ex.InnerException);
System.Runtime.InteropServices.Marshal.ReleaseComObject(workbook);
System.Runtime.InteropServices.Marshal.ReleaseComObject(excel);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
#region ScriptResults declaration
/// <summary>
/// This enum provides a convenient shorthand within the scope of this class for setting the
/// result of the script.
///
/// This code was generated automatically.
/// </summary>
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
#region Log messages to package log files/table.
public void LogMessages(string strLogMsg)
{
Dts.Log(strLogMsg, 0, bytes);
}
#endregion
}
But when i run the package the log file is not updated. The log file just contains the following :
Fields: event,computer,operator,source,sourceid,executionid,starttime,endtime,datacode,databytes,message
Can someone please help me understand what I am missing here for logging? Also, what could be the issue for the job to fail in SQL server?
Why is it not logging?
Here's the fun part, as best as I've been able to divine over the years of dealing with SSIS. Dts.Log is pretty useless, at least if you want it to show up in the Logging facility built into SSIS.
Instead, change out your Dts.Log calls to Dts.Events.Fire e.g.
bool fireAgain = false;
Dts.Events.FireInformation(0, "This gest logged", "My long description here", string.Empty, 0, ref fireAgain);
Then, in your Details tab above, ensure that you have checked the OnInformation event (this also presumes you've configured the package to track it all)
Finally, if you don't actually click the button in Providers and Logs tab, it won't log to the table
Why is it not working?
The package isn't working because you're dealing with Excel and the error message specifies you are running in 64 bit mode.
Microsoft (R) SQL Server Execute Package Utility Version 11.0.5058.0 for 64-bit
Unless you've done something to explicitly make 64 bit Excel work on this server, it's not going to work. Instead, in the SQL Agent job, you need to specify that this task runs in 32 bit mode.
See also
Why does my ODBC connection fail when running an SSIS load in Visual Studio but not when running the same package using Execute Package Utility
Have hard time to get push notifications from server when using MSSQL scalling.
Message flow is one way from server to subscribers (are distributed by groups)
Push messages are not received on UI with the following host configuration:
GlobalHost.DependencyResolver.UseSqlServer(new SqlScaleoutConfiguration(ConfigurationManager.ConnectionStrings["SignalR"].ConnectionString));
Messages are created in SignalRDb and records are present in tables. However they do not reach UI. With SQL scaling disabled all messages propagates to UI successfully.
Here is my Notification code:
public void OnNext(ResultModel value)
{
Clients.Group(group).notify(value);
}
OnNext method always executes witho or without Scalling also group and value are correct. No exceptions thrown on UI or beckend. Below is UI part:
var hubProxy = $.connection.visitsHub;
hubProxy.client.notify = function (updatedResult) {
console.log(updatedResult.Id);
});
};
Any help is appreciated.
I am using Linq to SQL in a two tier project where the server tier abstracts the DB and is using Linq to SQL.
In my client tier I construct an object and I send to the server.
I have a Task, which has a relationship to Reporter (who reported this task), so Task has a ReportedID column in the database, which is a FK to Reporter.ID.
In the Linq abstraction, my Task has a Reporter property and a ReportedID property.
To save new Tasks, I would like to use the ReportedID, so I have this code:
//Populate the object with the info
Task task = new Task();
task.Title = tbTitle.Text;
task.Description = tbDescription.Text;
task.Severity = ((Severity)lbSeverities.SelectedItem);
//the first state: "open"
task.StateID = 1;
//TODO - Set ReporterID
task.ReporterID = 1;
//Save the task
client.SaveTaskCompleted += new EventHandler<SaveTaskCompletedEventArgs>(client_SaveTaskCompleted);
client.SaveTaskAsync(App.Token, task);
So, the object is constructed and sent to the server, where it is saved using this code:
public Task SaveTask(string token, Task task)
{
TrackingDataContext dataConext = new TrackingDataContext();
//Saves/Updates the task
dataConext.Tasks.InsertOnSubmit(task);
dataConext.SubmitChanges();
return task;
}
The problem is that I get an exception: "An attempt was made to remove a relationship between a Reporter and a Task. However, one of the relationship's foreign keys (Task.ReporterID) cannot be set to null.".
If I use the Reporter property, it works.
What am I doing wrong?
Thank you,
Oscar
I made some refactory in my code and this error doesn't anymore. It may be a logic error but I can't tell exactly what.