I am trying to use Jenkins Conditional step plugin, to trigger a shell script if the current job has a certain pattern in its log, say "Keep one snapshot per week".
I guess I need to use the "Regular expression match" from conditional step plugin, but how should I configure the Expression and Label values ?
Thanks
The first step is to get the log content, that will be the input for the regex to match. This is done by setting Label with :
${BUILD_LOG_REGEX,regex="Keep one snapshot per week"}
(note : after selecting "regular expression match" in the combo box when configuring the plugin, you can click on the question mark to get the list of existing tokens like BUILD_LOG_REGEX that are available to use)
when this will be executed, if the log actually contains the string, the returned value will be something like :
Label=[[...truncated 1515 lines...]
[INFO] [05:10:55.472] -> Keep one snapshot per week between 2016-03-10 and 2017-02-09
[...truncated 22 lines...]
]
So now, we need to find the regex that matches exactly the label content, ie whatever is between the external brackets, including the blank characters. In the case above, we can configure Expression with :
^.*\s.*Keep one snapshot per week.*\s\[.*\s*
"\s" matches any space, tab or newline character.
Then you should see in the job logs something like this, confirming it's matching :
Regular expression run condition: Expression=[^.*\s.*Keep one snapshot per week.*\s\[.*\s*], Label=[[...truncated 1515 lines...]
[INFO] [05:10:55.472] -> Keep one snapshot per week between 2016-03-10 and 2017-02-09
[...truncated 22 lines...]
]
Run condition [Regular expression match] enabling perform for step [Execute shell]
Related
I'm fairly new to SSIS.
I have a pretty complex SQL query (that I don't want to replicate & maintain 3 times), whose result is a table with 2 columns:
emailAddress, formLetterNumber
What I want to do is
for each row returned:
if formLetterNumber == 1
send form letter #1 to that emailAddress
if formLetterNumber == 2
send form letter #2 to that emailAddress
if formLetterNumber == 3
send form letter #3 to that emailAddress
For various business reasons, I don't expect there to be more than 3 form letters.
What I've done:
On the Control Flow, I've put an Execute SQL component. (so far, so good)
I connected it to a Foreach component
In the Foreach component, I put a Send Mail Task
enter image description here
This works if there's only 1 type of form letter. This single-form-letter package runs without any problem.
But now I want to do a Conditional Split so I can divide the flow to 3 different Send Mail Tasks, depending on the formLetterNumber returned by the SQL. But the Conditional Split is on the Data Flow toolbox, and the Send Mail Task is only on the Control Flow toolbox, and I can't figure out how to move the data from one component to another. See image for what I'm trying to do: enter image description here, enter image description here
I've looked on this site, and on YouTube, and either the solution is not there, or I'm asking my question the wrong way. C# and scripting is an option, but purchasing other software is not. Thank you in advance for your suggestions.
It is possible to use Precedence Constraints in the SSIS Control Flow.
Precedence constraints support expressions. If we choose expressions in SSIS precedence constraints, it evaluates the expression and if the expression is true, then the constrained executable runs.
So for your case you would need three precedence constraints with an expression along the following line:
#formLetterNumber == 1
Next precedence constraint expression would be
#formLetterNumber == 2
Good article on the subject: Overview of SSIS Precedence Constraints
As end result, it is possible to compose the process as follows:
Add an empty Sequence Container inside Foreach Employee Container.
Out of Sequence Container add three Precedence Constraints described
above.
Each Precedence Constraint ends up with its own Send Email task for a specific type of form letter.
I have a time series graph in Google Data Studio that connects to Search Console. In the graph I have a filter to only show search "queries" featuring a certain phrase. It works fine when I use one query. In the image I've blanked out the term as it's a clients brand name.
However when I add an OR element and then a second query the time series chart breaks.
When I add the second query it shows the error "Data Studio Cannot Connect to Your Data Set".
query 2:
Error message:
Tried it out earlier today using a Filter, and REGEXP_MATCH with a Pipe operator
|
For example, if you are searching for a Query that contains X or Y or Z, this does the trick:
Include Query RegExp Match .*(X|Y|Z).*
I can't tell you why - but I can tell you what's causing it and how to get round it.
The Search Console data source doesn't recognise the 'or' condition in the filter. To have an 'or' condition you need to add it as regex in your first condition
While creating Kinesis Analytics application it successfully discovered my schema based on the data. However, when I hit save and continue, I get following error
Error updating application There was an issue updating your
application. Error message: 1 validation error detected: Value 'C' at
'input.inputSchema.recordColumns.2.member.name' failed to satisfy
constraint: Member must satisfy regular expression pattern:
[a-zA-Z][a-zA-Z0-9_]+
my sample record is below
{"reported": {"timestamp": "1482231365", "C": "40", "id": "D_aa-bb"}}
My bad, I overlooked the error message. Found the solution, hope it might help someone.
The auto detected schema name was the issue. From the sample record, the auto detected column name was C and the regex says it should contains atleast two characters. After editing the schema manually with two characters it succeeded.
There was another issue though, the auto detected column name timestamp is a reserved keyword, which we need to change.
I'm trying to create a simple script that I can paste into the RethinkDB data explorer to initialize a database.
I can insert records fine with something like this:
r.db('testdb').table('cars').insert([{ "model" : Honda }])
But how can that be followed by a second insert into a related table like:
r.db('testdb').table('engines').insert([{ "cars_id" : <what goes here> }])
I could imagine writing some javascript to extract the newly created id's, but it doesn't appear that the data explorer supports things like variables.
Ideally there would be a way to write a large script that would work directly in data explorer for when needed for convenience.
The data explorer can take multiple queries separated by a semi colon.
Note that queries will be sequentially executed, and only the result for the last query will be returned.
The data explorer supports some javascript to some extent, the requirements (as far as I know) are:
the last query must be a ReQL query
Any instructions must ends with a semi colon (except the last one)
That being said, you cannot use the result of the first query in the second query using JavaScript (you don't get access to the result). You can however nest queries (or chain with do), something like this should work I think:
r.db('testdb').table('cars').insert([{ "model" : Honda }]).do(function(result) {
r.db('testdb').table('engines').insert([{ "cars_id" : <you can use the result of the insert here> }])
})
I am getting to know SSIS, I apologize if the question is too simple.
I got a set of tasks inside a foreach-loop-container.
The first task needs only to get executed on condition that a certain user variable is not null or empty.
Otherwise, the flow should skip the first task and continue to the second one.
How would I go about realizing this (in detail) ?
Issue 1: There are two ways to interpret your logic: "...a certain user variable is not null or empty":
The (Variable is Not Null) OR the (Variable is Empty).
The (Variable is Not Null) OR the (Variable is Not Empty).
It's all about the object(s?) of the word "not". The differences are subtle but will impact when the first task in the Foreach loop executes. For demonstration purposes, I am assuming you intend #1.
Issue 2: The first task can no longer be first. In order to accomplish what you desire using SSIS inside the BIDS environment, you need to place another task ahead of the task formerly known as "the first task". This is so you can set a Precedence Constraint on the former first task from the new first task.
It is possible to accomplish what you desire by designing your SSIS dynamically from managed code, but I don't think this issue warrants the overhead associated with that design choice.
I like using an empty Sequence Container as an "Anchor" task - a task that exists solely to serve as the starting endpoint of a Precedence Constraint. I heavily document them as such. I don't want anyone deleting the "unnecessary empty container" and roaming the halls for days shaking their heads and repeating "Andy, Andy, Andy..." but I digress.
In the example below, I have two precedence constraints leaving the empty Sequence Container. One goes to the task that may be skipped and the other to the task following the task that can sometimes be skipped. A third precedence constraint is required between the task that can sometimes be skipped and the task following. It is important to note this third precedence constraint must be edited and the Multiple Constraints option set to OR. This allows the task following to execute when either of the mutually exclusive previous paths are taken. By default, this is set to AND and will require both paths to execute. By definition, that will not - cannot - happen with mutually exclusive paths.
I test the value of an SSIS String variable named #MyVar to see if it's Null or Empty. I used the Expression Only Evaluation Option for the constraints leaving the empty Sequence Container. The expressions vary but establish the mutual exclusivity of the expression. My Foreach Loop Container looks like this:
I hope this helps.
:{>
The best thing can be to use the 'Disable Property' in expressions and giving the expression as per the condition. Just search how to use the disable property.
How about a simple solution instead of some of the more complex ones that have already been given. For the task you want to conditionally skip, add an expression to the disabled property. Any expression that produces a true or false result will work, so for the question example you could use:
ISNULL(#[User::MY_VAR]) || #[User::MY_VAR]==""
The only downside is that it may not as visible as some of the other solutions but it is far easier to implement.
I would create a For Loop Container around the task that needs the condition with the following conditions (#iis the loop counter, #foo is your user variable that you want to test):
InitExpression: #i=0
EvalExpression: #i<1 && !ISNULL(#Foo) && #Foo!=""
AssignExpression: #i=#i+1
there is no need to create a "script"
I think the best (and simpler) approach is to add a blank script task inside your loop container before your "first task", drag the green arrow from it to your "first task" (which obviously will become the second) and use the precedence constraint to do the check.
To do that, double click the arrow, select "expression" on the "evaluation operation" and write your expression. After hitting OK the arrow will become blue indicating that it isnt a simple precedence constraint, it has a expression assigned to it.
Hopefully I didn't misunderstand the question but a possible solution can be as written below.
I created a sample ForEach loop. The loop itself is an item enumerator. It enumerates the numbers 1, 2, 3. The acutal value is stored in a variable called LoopVariable.
There is another variable named FirstShouldRun which is a Boolean variable showing the first task in the foreach loop should be runned or not. I set this variable's EvaluateAsExpression property to true, and its expression is (#[User::LoopVariable] % 2) == 0. I would like to demonstrate with this that every second time the first task should be started.
The two tasks do nothing much but display a MessageBox showing the task has been started.
I started the package and first and the third time the first task didn't started. In the second loop the MessageBox (showing "First started") appeared.
After that you should set FirstShouldRun variable as you like.
As I mentioned in my first comment to the OP, this solution is based on the idea of Amos Wood written in another answer.
That's a bit tricky.
You have to create a Script Task and check if your variable is not null in there.
So first you have the script task in which you will have the following code in your Main() function:
public void Main()
{
if (Dts.Variables["User::yourVariable"].Value != null)
{
Dts.TaskResult = (int)ScriptResults.Failure;
}
else
{
Dts.TaskResult = (int)ScriptResults.Success;
}
}
Then you create two connections from your script task, one to the task that needs to be executed when your variable is not null, and one to the next task (or to another script, if you need to check again, if the variable is not null).
Then you right-click on the (green) arrow of your first connection and select "Failure". Right-click the connection to the next task / script and set it to "Completion".
It should then look something like this:
That's it.