I'm trying to generate a random release-name by using the Bamboo Server's deployment project / plan.
I was able to generate a dynamic version number by using variables (defined the bamboo.release_number, bamboo.release_major, etc.)
resulting a release name of "StaticName-1.2"
I would like to define on each release that the StaticName will be generated randomly, based on predefined array or list.
The wanted result is by giving it the list of: ["NameA", "NameB", "NameC"]
Bamboo will generate something else on every run, like:
"NameB-1.3"
"NameC-1.4"
"NameA-1.5"
Any ideas how can I preform such thing?
Generally, you can add a build task to write the release name to a file in the project in the format of KEY=VALUE. You would then add an inject variables task to read that file, adding the variables to the plan. You can then reference that variable when creating the release just as you did for the build number.
Related
I wrote my model in the GUI and want to run it with repetitions in the headless mode in a cluster.
The model has a go command that is repeated until we reach a specified step (at each go procedure, the year variable is incremented and when we reach 2070, the model stop running). At the end of the go procedure, the world is exported (and analysed in R).
If I run multiple repetitions on parallel cores, how can I export the worlds so they have different names ?
So far, I export the world with the following lines (when running only one time) :
let file-name (word scenario "_NETLOGO_" year ".csv")
export-world (file-name)
But if the model is run at the same time on several cores, there will be overlap and I would not know which file is coming from which repetition (assuming that the name would change with an extra (1)). I thought about creating folders to save the worlds,is that possible ? Is so, how is it possible to pimp the folder name according to the number of repetitions ?
Started a new SSIS project, and forgot to set the default protection level to "Don't save sensitive" (our standard) Now midway through the project and made the changes (at project level and in each package.) When checked all packages are Don't Save Sensitive and the project is Don't Save Sensitive, however when attempting a build, I get
Project consistency check failed. The following inconsistencies were
detected: PACKAGE1.dtsx has a different ProtectionLevel than the
project. PACKAGE2.dtsx has a different ProtectionLevel than the
project. ... PACKAGE(N).dtsx has a different ProtectionLevel than
the project.
(it lists every package in the project even though they all match the Project level protection.)
I suspect you ran into the same issue I did. I corrected all my packages via the API so that they all indicated they were DTS:ProtectionLevel="0" which is unprotected.
The project (.dtproj) file also has a protection level which gets set to DontSaveSensitive. <SSIS:Project SSIS:ProtectionLevel="DontSaveSensitive" xmlns:SSIS="www.microsoft.com/SqlServer/SSIS">
The mismatch for me was that inside the project file, it keeps track of far too much information about each package so if you scroll down, you'll see an entry per package like
<SSIS:Property SSIS:Name="ProtectionLevel">3</SSIS:Property> or whatever the default number is. Make that 0 in the file (search and replace). Save the project file and it'll now build.
You might need to perform a Build All to get it to build. I suspect that VS/SSDT is trying to use the extra data it stores in the .dtproj file to determine whether it needs to validate all the packages in a project. Since we hand edited the file, it didn't trip whatever sensor would normally be flipped to signal a full recompile was needed.
billinkc's answer didn't work for me because changing the value with a text editor doesn't change it correctly. The following MSDN page explains that there is a cmd line tool to manage this:
http://technet.microsoft.com/en-us/library/cc879310.aspx
Like so:
for %f in (*.dtsx) do dtutil.exe /file %f /encrypt file;%f;2;strongpassword
It will change every module in the project to the protection level specified in the second to last value. If it is 0, don't store values, then you don't need the password and can get rid of the last semicolon and everything after.
The following MSDN article has a table with the numbers for each protection level, as used with dtutil:
http://technet.microsoft.com/en-us/library/ms141747.aspx
Microsoft have broken this so that even using the DTUTIL utility to change package protections doesn't fix the project file metadata.
I had to apply to manual fix to the project file to change metadata storing a copy of the Package Protection level of the packages to the same as that of the project and packages.
Thought through? Probably not.
Get list of packages (that have already been deployed and create DTUTIL statements. Put them in a batch file and execute from a command line.
This only works for deployed packages as we are looking at SSISDB and not the Project folder
USE SSISDB
DECLARE #projName VARCHAR(250) = 'Sales'
DECLARE #FolderPath VARCHAR(1000) = 'E:\ssis_' + #projName
DECLARE #DtutilString VARCHAR(1000) =
'"C:\Program Files\Microsoft SQL Server\130\DTS\Binn\dtutil.exe"/file "'+ #FolderPath +'\XXX" /encrypt file;"'+ #FolderPath +'\XXX";0 /quiet'
SELECT DISTINCT
REPLACE(#DtutilString, 'XXX', pack.[name])
-- SELECT *
FROM internal.packages AS pack
INNER JOIN
[internal].[projects] AS proj
ON pack.project_id = proj.project_id
WHERE proj.name = 'ssis_' + #projName
I have a simple String variable with the following value: "C:\Test.txt".
Now I would like to edit the variable to point to a different file.
I cannot find a way to do that. I can change the Name, Data Type, but not the value itself!
Do I need to delete the variable and create the new one?
Update: The problem was caused by "ReadOnly" property set to "True". For typical scenarios, see the accepted answer below.
As #Yuck and #devarc have noted, there are two different and distinct values a Variable holds. The Design-time value is the value you assign when the variable is first created. In your case, the variable holds C:\Test.txt as the design-time value. Everytime you open the package, it would show C:\Test.txt until you change it in the
To make the value of a variable change while the package is running, your options are either to set the value or calculate it. Here I have created a package-level variable CurrentFile with the value of C:\Test.txt
One thing that often trips people up is that they have correctly changed the run-time value but when they run it in BIDS, they see the "old" value. The value displayed in the Variables window does not change during package execution.
During package execution, my Variables window still shows the design-time value (C:\Test.txt) but the true value is reflected in the Locals window (C:\Test2.txt)
Setting a value
The value of most anything in SSIS can be established at run-time through a set of verbose command-line options or through configuration sources. The biggest difference in my mind is that this approach is that the value will always be the value for the entire lifetime of package execution. Sequential or parallel invocations of a package can change that value but for that execution the value would remain constant (barring an explicit modification of the value.
/SET
Command-line execution (dtexec.exe), right clicking on a package and running from the filesystem (dtexecUI.exe) or creating a SQL Agent job step of SQL Server Integration Services all allow for providing a run-time value through the SET command. Using the above variable, the following command would set the run-time value to C:\Test2.txt
dtexec /file C:\Generated.dtsx /set \Package.Variables[User::CurrentFile].Properties[Value];"C:\Test2.txt"
Configuration
SSIS offers an option to create configuration sources to provide run-time values to packages. The article I linked to above does a much better job describing the pros and cons of the configuration options than I will do here. I will say that I typically use both - my SET command configures a connection manager which is then used by the package to find the "full" set of package configurations.
Calculating a value
There are a variety of tasks in SSIS that can change the value of a variable as well as the use of Expressions to change a value. I see these as things that operate on value whilst the package is in flight.
Tasks
A Script Task is one of the most commonly used mechanisms for those starting out but I find other tools in the SSIS toolkit usually better suited for changing variable values.
Foreach Loop Container and Execute SQL Task are two of the other big Tasks you should look at for assignment of a variable value.
Expressions
Expressions are the most glorious candy in the SSIS toolbox. Most every "thing" in SSIS exposes properties for configuration. That's helpful, but using assigning an expression to build those properties is outstanding.
For example, imagine 3 variables RootFolder, FileName and ComputedCurrentFile with values of C:\, File2.txt and empty string. On the Properties window for ComputedCurrentFile we'd change the value for EvaluateAsExpression from False to True and then use an expression like #[User::RootFolder]+ "\\" +#[User::FileName] That simply concatenates the value the first two variables together. This can be helpful if the file name for processing was standard but the source folder changed often. Or if we're talking about output, it's common to use expressions to build an output file name using the date and possibly time of when the package is running.
Finally, there is nothing that prevents a mixing and matching of these approaches. I typically use a configuration to point a file enumerator at the correct starting folder and then use calculated values to identify the current file for processing.
If you want to change it in designer just right click on free space and --> Variables.
But if you want to change it at runtime I suggest you to:
create script task
choose language
add your variable to ReadWriteVariables.
Edit script.
For example in VB:
Dts.Variables("myVariable").Value = #"C:\Test2.txt";
Dts.TaskResult = ScriptResults.Success
Found an easy way to handle this. Remove the Variable from Expression which will enable Value Box to edit. Once it is edited, add the Variable back in the Expression should get the updated value. Hope this helps.
I was also facing the same issue like you where once the variable is declared and define (for eg:var1=text1.csv)in SSIS Variable window I was not able to update the variable value(for eg: var1=text2.csv) in SSIS Variable Window by clicking on the variable value field.
Applied below fix:-
I noticed that I was using var1 variable as a Expression by using expression builder so to update the value(for eg:-var1=text2.csv) I used expression builder window.once you done using the expression builder,you can see the text2.csv is got mapped to var1.
I too need a way to find out, how to get and parse the protection table using the P4.net DLL. I am trying to run "p4 protects" command using P4.Net DLL. I just need a way to find out which AD Group has access to which branch of perforce tree.
You can use the code
P4Form group = p4.Parse_Form("group", groupString);
groupString is a string argument that will be written filling all the required fields of a group.
I'm tring to create an SSIS package to import some dataset files, however given that I seem to be hitting a brick
wall everytime I achieve a small part of the task I need to take a step back and perform a sanity check on what I'm
trying to achieve, and if you good people can advise whether SSIS is the way to go about this then I would
appreciate it.
These are my questions from this morning :-
debugging SSIS packages - debug.writeline
Changing an SSIS dts variables
What I'm trying to do is have a For..Each container enumerate over the files in a share on the SQL Server. For each
file it finds a script task runs to check various attributes of the filename, such as looking for a three letter
code, a date in CCYYMM, the name of the data contained therein, and optionally some comments. For example:-
ABC_201007_SalesData_[optional comment goes here].csv
I'm looking to parse the name using a regular expression and put the values of 'ABC', '201007', and
'SalesData' in variables.
I then want to move the file to an error folder if it doesn't meet certain criteria :-
Three character code
Six character date
Dataset name (e.g. SalesData, in this example)
CSV extension
I then want to lookup the Character code, the date (or part thereof), and the Dataset name against a lookup table
to mark off a 'checklist' of received files from each client.
Then, based on the entry in the checklist, I want to kick off another SSIS package.
So, for example I may have a table called 'Checklist' with these columns :-
Client code Dataset SSIS_Package
ABC SalesData NorthSalesData.dtsx
DEF SalesData SouthSalesData.dtsx
If anyone has a better way of achieving this I am interested in hearing about it.
Thanks in advance
That's an interesting scenario, and should be relatively easy to handle.
First, your choice of the Foreach Loop is a good one. You'll be using the Foreach File Enumerator. You can restrict the files you iterate over to be just CSVs so that you don't have to "filter" for those later.
The Foreach File Enumerator puts the filename (full path or just file name) into a variable - let's call that "FileName". There's (at least) two ways you can parse that - expressions or a Script Task. Depends which one you're more comfortable with. Either way, you'll need to create three variables to hold the "parts" of the filename - I'll call them "FileCode", "FileDate", and "FileDataset".
To do this with expressions, you need to set the EvaluateAsExpression property on FileCode, FileDate, and FileDataset to true. Then in the expressions, you need to use FINDSTRING and SUBSTRING to carve up FileName as you see fit. Expressions don't have Regex capability.
To do this in a Script Task, pass the FileName variable in as a ReadOnly variable, and the other three as ReadWrite. You can use the Regex capabilities of .Net, or just manually use IndexOf and Substring to get what you need.
Unfortunately, you have just missed the SQLLunch livemeeting on the ForEach loop: http://www.bidn.com/blogs/BradSchacht/ssis/812/sql-lunch-tomorrow
They are recording the session, however.