In my SSIS Script Component Task (NOT the Script task, which can be added to a Control Flow, but the Script Component Task, which is added within Data Flows), I added some error handling in the catch part of a try, catch block:
// No connection was created. Exit gracefully
bool cancelOnError = false;
ComponentMetaData.FireError(ErrorCode: 10, SubComponent: "SubComponent",
Description: "Couldn't set up the connection. This could be because an invalid host was
provided, or due to a firewall blocking the connection.", HelpFile: "", HelpContext: 0,
pbCancel: out cancelOnError);
This all works fine, with the script task catching the error I raise in my code. I can also see the error in the logs:
Error: 0xA at DFT Extract, SubComponent: Couldn't set up the connection.
This could be because an invalid host was provided, or due to a firewall
blocking the connection.
However, the icon on on the task is green, and a subsequent task I defined to process results gets fired (there is nothing to process, though, as this particular error occurs prior to processing any data):
The Data Flow correctly shows a red cross icon. Is there any way I can change the icon on the Script Component, or a better way for me to (elegantly) simulate a showstopping error?
I found this page on Microsoft, showing the difference between Script Task and Script Components, which also states:
The Script component runs as a part of the Data Flow task and does not
report results using either of these properties.
That doesn't have me very hopeful, but I am hoping someone might have a work around. I primarily think that showing the green icon is somewhat misleading when we trap an error.
Why not just changing the cancelonerror value to true
bool cancelOnError = true;
Or you can use Throw exception method
throw new Exception("Couldn't set up the connection. This could be because an invalid host was
provided, or due to a firewall blocking the connection.");
Workaround
If your goal is to ignore rows that was not processed correctly, the best way is to add a Column to the OutputBuffer (of type DT_BOOL), and set this column value to False if there is an error , else it's value must be True.
And add a conditional split after the script component to filter rows based on this Column. With a similar expression:
[FlagColumn] == True
Related
A continuation to this : Flink : Handling Keyed Streams with data older than application watermark
based on the suggestion, I have been trying to add support for Batch in the same Flink application which was using the Datastream API's.
The logic is something like this :
streamExecutionEnvironment.setRuntimeMode(RuntimeExecutionMode.BATCH);
streamExecutionEnvironment.readTextFile("fileName")
.process(process function which transforms input)
.assignTimestampsAndWatermarks(WatermarkStrategy
.<DetectionEvent>forBoundedOutOfOrderness(orderness)
.withTimestampAssigner(
(SerializableTimestampAssigner<Event>) (event, l) -> event.getEventTime()))
.keyBy(keyFunction)
.window(TumblingEventWindows(Time.of(x days))
.process(processWindowFunction);
Based on the public docs, my understanding was that i simply needed to change the source to a bounded one. However the above processing keeps on failing at the event trigger after the windowing step with the below exception :
java.lang.IllegalStateException: Checkpointing is not allowed with sorted inputs.
at org.apache.flink.util.Preconditions.checkState(Preconditions.java:193)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.init(OneInputStreamTask.java:99)
at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:552)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:537)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:764)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:571)
at java.base/java.lang.Thread.run(Thread.java:829)
The input file contains the historical events for multiple keys. The data for a given key is sorted, but the overall data is not. I have also added an event at the end of each key with the timestamp = MAX_WATERMARK to indicate end of keyed Stream. I tried it for a single key as well but the processing failed with the same exception.
Note: I have not enabled checkpointing.
I have also tried explicitly disabling checkpointing to no avail.
env.getCheckpointConfig().disableCheckpointing();
EDIT - 1
Adding more details :
I tried changing and using FileSource to read files but still getting the same exception.
environment.fromSource(FileSource.forRecordStreamFormat(new TextLineFormat(), path).build(),
WatermarkStrategy.noWatermarks(),
"Text File")
The first process step and key splitting works. However it fails after that. I tried removing windowing and adding a simple process step but it continues to fail.
There is no explicit Sink. The last process function simply updates a database.
Is there something I'm missing ?
That exception can only be thrown if checkpointing is enabled. Perhaps you can a checkpointing interval configured in flink-conf.yaml?
I have a simple wpf application that continues to run even after I explicitly call it to shut down.
It integrates with a third party application and needs to check that a few documents of a certain type and with specific content are open as it initializes.
Here is a portion of the initialization code:
try
{
ActiveProductDoc = Automation.CATIA.ActiveDocument as ProductDocument;
}
catch
{
InvalidAssemblyShutdown("You must have an assembly open before you run the app");
}
if(ActiveProduct == null)
InvalidAssemblyShutdown("You must have one assembly open (not a part)");
ActiveProduct = ActiveProductDoc.Product;
And here is the InvalidAssemblyShutdown method:
private void InvalidAssemblyShutdown(string message)
{
MessageBox.Show(message);
Close();
Application.Current.Shutdown();
}
I have set the application's ShutdownMode property to OnMainWindowClose.
I am currently doing a use case test where the user has the wrong type of document open and so the ActiveProduct field is null. The InvalidAssemblyShutdown method is called as expected but despite this the line in the initialization method following the shutdown call still runs and throws an exception.
Any ideas what's going on?
Should I throw exceptions instead and use a global exception handler?
If you have a look at the source code for Application.Current.Shutdown (link to source), you'll see that it uses Dispatcher.BeginInvoke() to initiate the shutdown. In other words, the shutdown gets queued on the UI thread. It doesn't take effect during that precise method call, so the following code keeps executing.
You'll need to exit the code right after the call to Application.Current.Shutdown if you don't want some code to run while the shutdown request gets processed. Something like:
if(ActiveProduct == null)
{
InvalidAssemblyShutdown("You must have one assembly open (not a part)");
return; // prevent further code execution.
}
For what it's worth, this.Close() works in a similar way. So if you have proper flow control, you won't need to invoke Application.Current.Shutdown at all. Your call to this.Close() should be enough.
I try to setup connection to SQL Server and catch the error
var
GFireDACConnProp : TSQLDBFireDACConnectionProperties;
GFFireDACConn: TSQLDBFireDACConnection;
begin
try
GFireDACConnProp := TSQLDBFireDACConnectionProperties.Create('MSSQL?Server=server','dbname','user','pass');
GFFireDACConn := TSQLDBFireDACConnection.Create(GFireDACConnProp);
// OR I := GFireDACConnProp.Execute('Select * from Station', []);
GFFireDACConn.Connect;
....
Error message:
Project app_.exe raised exception class Exception with message 'Object factory for class {3E9B315B-F456-4175-A864-B2573C4A2101} is
missing. To register it, you can drop component [TFDPhysXXXDriverLink]
into your project'.
What is correct way to connect to SQL Server and expose REST service?
FireDAC is more helpful than some other frameworks in that when things go wrong the exception messages often say how to fix the problem.
So, in your case, given that the message says "you can drop component [TFDPhysXXXDriverLink] into your project" the thing to try first would be to drop the relevant DriverLink component onto your form/datamodule. As you're using Sql Server, the the driver link to choose would be the TFDPhysMSSqlDriverLink, which is on the FireDAC Links tab of the Component Palette.
If you're creating a Console application, obviously there's no form or datamodule to drop the link on. In that case, create it in code:
FDPhysMSSQLDriverLink := TFDPhysMSSQLDriverLink.Create(Nil);
I know its an old thread, but if other come by, it can be fixed just by adding the FireDAC's units to your uses clause e.g.
uses
FireDAC.Phys.MSSQL, FireDAC.Stan.Def, FireDAC.Stan.Pool, FireDAC.DApt, FireDAC.Stan.Async; // Not sure if you need them all, FireDAC will ask for them
I'm trying to create a connection to an Interbase ToGo database, at runtime, in a unit that does not have a form.
MyConnection := TSQLConnection.Create(nil);
strTestPath := TPath.Combine(TPath.GetDocumentsPath, 'ISHMAEL.GDB');
MyConnection.Params.Add('Database=' + strTestPath);
MyConnection.DriverName := 'IBToGo';
MyConnection.Params.Add('User_Name=SYSDBA');
MyConnection.Params.Add('Password=masterkey');
MyConnection.Open();
Everything seems fine as I step through, until I try and open the connection. The open raises an I/O error, saying that database.gdb isn't found. I called the database ISHMAEL, so is something else happening, or is the error message misleading?
Ideally, I'd like to get all of my database calls (and really, most if not all of the business logic) out of the forms and into some data access and business object units, but am having difficulty finding examples.
All of the examples and tutorials I have found focus on dropping controls on a form, and using visual live bindings.
Can this be done inside a non-form unit?
Thanks!
I have a SQL Sever 2008 R2/64-bit database server for which I'm writing some fairly basic scripting utilities with the Sql Server Management Objects (SMO). My project is a 32-bit VS2010 executable written in C#.
Most of the effort has been fairly simple and successful. The only problem I'm having is in the firing of my custom event handler that should be called in response to a Scripting Error.
The Scripter object exposes a ScriptingError event, which I have attempted to leverage thusly:
//srv contains a valid server name
Scripter scrp = new Scripter(srv);
//scrp_ScriptingError is my handler
scrp.ScriptingError += new ScriptingErrorEventHandler(scrp_ScriptingError);
My handler is declared thusly:
static void scrp_ScriptingError(object sender, ScriptingErrorEventArgs e)
{
// my handler goes here, just printing e.Current.Urn to the console
// This is merely representative, have had other things here, but
// the handler never fires
Console.WriteLine(e.Current.Value);
}
All this compiles cleanly.
My code is invoked via simple scrp.Script(urns); where urns is just an array of the database objects being scripted out. Nothing fancy:
try
{
sc = scripter.Script(urns);
}
catch (Exception e)
{
WriteLog(String.Format("Failure during scripting: {0}: Inner exception message (if any): {1}",e.Message,((e.InnerException==null)?"[None]":e.InnerException.Message)));
}
using (System.IO.StreamWriter file = new System.IO.StreamWriter(fileName,true))
{
foreach(String currentLine in sc)
{
file.WriteLine(currentLine);
file.WriteLine("GO");
}
}
The problem is that, no matter what I've tried so far, when errors occur during scripting, my ScriptingError handler never fires.
Even in debug mode within VS2010, when I set a breakpoint within the handler, and fire my scripting code, and know an error will occur, only an exception will be thrown, but the breakpoint in my ScriptingError handler never trips.
I'm in trees-for-forest mode now, not sure what I've done wrong. Am I expecting the wrong things for the ScriptingError handler?
I have searched the MSDN docs on the SMO objects and found virtually nothing on ScriptingError handlers other than the basic API calls themselves, and precious few examples on the Internet. It seems incredibly simple and straightforward to me - just assigning an event handler to the event - but there's some battery-not-included notice I've failed to note.
Pointers to my error are greatly appreciated, with a polite request for minimal brickbats if the error is exceptionally stupid on my part :)
I am not at a pc right know, but I think you should try to set the ContinueScriptingOnError option. Otherwise there would be no reason for SMO to invoke the event, but rather through an exception instead.