I have a basic understanding problem with datastage. I am new to this field. It is about the implementation of loops. First I get several rows of a select query using the connector-stage. Now I would like to do several more steps for each row. The result of each row should now be used as a variable in further stages. How can I do that? I know the loop possibility in the transformer stage, but does not seem to solve my problem.
Should i work with the loop stage in the jop sequence? If yes how?
the Problem:
foreach($selectQueryResults as row) {
// do something with the row-value
}
Thanks!
I have to admit, I've not got access to DataStage anymore nor the code I once had.
This is to propagate stage variables at a Job Level
However the routine activity to accomplish your task would be as follows.
Propagate SQL as a variable
Use DSExecute to execute a command line function (to call SQLPlus, nzsql or whichever your command line is) and pass through your SQL variable
Return it's results into another variable
With that variable you can split the contents, first by line and then by delimiter using loop / for statement.
Use the DSSetParam to map the key value pairs to your parameters of a specific job by using the DSAttachJob function, or just propagate them as outputs from the routine activity
BASIC Language Reference
DSSetParam
Remember that error handling and commenting is important within the BASIC routines otherwise
Subsequently, this is to propagate stage variables within a transformer and is a very powerful tool once it's been mastered.
Here is the documentation and examples for defining stage variables within a Transformer model and aggregating the output. Please note, the order of the stage variables is extremely important.
TransformerLoops
Within a transformer you can define incoming columns as stage variables, use those stage variables to aggregate, concatenate (strings), split, subtract... basically you can do a ridiculous amount once you get your head around it.
I would suggest going through the Transformer Examples first as I suspect that this may be what you're looking for.
Remember, it doesn't all have to be completed in a single Transformer stage, you can get the initial cleansing done in the first transformer and then do the complex loop in the second, break the steps down for what works for you.
Related
I am trying to create a new reference containing another reference as in ${var${randnum}}.
Ultimately, I want to create a variable which refers to a two times two randomized set of variables.
As the above approach did not work, I developed it further with below result.
In the calculate field I write
concat('$','{','trust',${rand_no2},'_' ,${rand_no3_1},'}')
Which should result in
${trust1_1}
and respective combinations.
Without line 11 (name=ref2) the file compiles and I can start it in ODK Collect (v.2.4) on my phone. When I reach line 10 (in ODK Collect), however, I receive the message:
"Error Occured
Dependency cycle in s; recursion limit exceeded!!"
(I included line 11 to show what I want to do in the end.)
I am writing the file in Excel and compile it with ODK xlsform offline. (For testing I transfer it via cable to my phone.)
The xls file for reproduction can be found here:
https://forum.getodk.org/t/concatenate-references-to-create-new-reference-var-randnum/34968
Thank you very much in advance!
You're mixing up some things related to the ${q} syntax, question names and question values.
Note that ODK Collect does not actually understand the ${q} syntax (which is XLSForm-only). It's helpful to look at the actual form format that ODK collect understands which is called XForm, an XML format that XLSForm converts into. However, even if ODK Collect understood the ${q} syntax, your approach still wouldn't work since you're creating a string value for the ref question (using concat). This wouldn't magically be evaluated as a reference / formula. You cannot dynamically create a reference or formula.
At the moment (until ODK supports something like the local-name() function), maybe the best approach is to use position and put the calculated values inside a group. Something like //group/calc[number(${pos})] perhaps. Note that positions are 1-based (so the first item is position 1) and casting the position to a number or integer is required.
The problem:
I need to read a row of values from an excel spreadsheet into a parameter array and use these values to set the initial values of stocks.
The specifics:
a. I can successfully set the scalar parameters default values from excel using ExcelControlName.getCellNumericValue("ExcelSheetName", RowNumber, ColumnNumber).
b. Trying to set default values for array parameters with ExcelControlName.readHyperArray(DestinationArrayName,"ExcelSheetName",RowNumber, ColumnNumber, false) returns a "Cannot return a void result" error on build.
c. I can read the parameter arrays from a function called from the Agent actions "On startup:" section using ExcelControlName.readHyperArray(DestinationArrayName,"ExcelSheetName",RowNumber, ColumnNumber, false).
d. Stocks with their initial values set to the parameter array that was successfully loaded by the function are all zero even though the parameter array shows values (when run). Initial value: ParameterArrayName.
e. When I set the parameter array values through the value editor the stocks initialize correctly.
My suspicion:
I'm thinking that the issue has something to do with the internal timing within the entrails of the model generated by Anylogic such that the function to load the parameters is executed after the stocks get initial values - but that could just be the delirium caused by repeatedly smashing my forehead into the wall. But, if this is indeed the case, how can I sneak the function in earlier or, better yet, how would someone who actually knows what they're doing accomplish this?
What I'm trying to accomplish:
As a septuagenarian with lots of time on my hands and a vague recollection of dynamic modeling using Dynamo from a Systems Science program in the early seventies (and not used since), I thought I'd take a whack at age-based modeling of the COVID-19 pandemic. I wanted to see, among other things, whether establishing elder-prisons (in now vacant Club-Meds, Sandals... I'm sure) would be an economically advantageous strategy. Getting there requires dis-aggregating classic SIR approaches into age-specific chains of causality. So far, I'm at 27 age-specific parameters for eight age-groups and 24 scalar parameters. As much as I'd like to type and retype... all this data I'm really hoping there is a better way.
I must say that I am amazed at how far modeling has come in only 50 or so years and I am enthralled with Anylogic's application - even though it's a bit java-ish for me.
Thank you ever so much,
Carl Cottrell
Equivalent parameter arrays - one with values entered in edit the other read from excel file through a function
Not sure if I understand but here it goes:
I understand that you have the following structure:
and you want to initialize the stock as follows.
To do that, on the initial value of your parameter use a function that returns an HyperParameter:
the function getInitialValue should return an HyperParameter and I think this code should work for you (you have to choose RowNumber according to whatever is on your excelfile, MyDimension is the name of your dimension, and ExcelControlName is the excel in which you hold the values of the initial values of the stock)
HyperArray x=new HyperArray(MyDimension);
for(int i=0;i<=numColumns;i++){
x.set(ExcelControlName.getCellNumericValue("ExcelSheetName", RowNumber, i), i);
}
return x;
We can use vpi_scan in the following way:
vpiHandle iter = vpi_iterate(property, handle);
if (iter)
while ( entry = vpi_scan(iter) )
/*code*/;
iter will be freed when vpi_scan() returns NULL.
But what if I need to scan through the loop several times? Is there a way to tell vpi_scan not to free the iterator, so that I could vpi_free_object when I'm done?
I think I can solve the problem using an additional container (to collect and keep all entries), but is it really needed?
EDIT:
1. I would not like to call vpi_iterate more than once, since it can be expensive.
2. Suppose I go with an additional container solution. Is there a way to find out the number of entries in the vpi_scan loop without actual scanning through the loop? This could make implementation simpler.
ut what if I need to scan through the loop several times? vpi_iterate returns an initialized pointer to the iterator. Every vpi_scan removes an element from the list and frees it. If vpi_scan did not run till the end, you'd better use vpi_free_object to clean the rest of the iterator list. If you need to rescan the same object again, you can call vpi_iterate again and it will return a new iterator object which you can re-scan.
s there a way to tell vpi_scan not to free the iterator, so that I could vpi_free_object when I'm done? No, (1) is the only mechanism which can be used to access and free iterator elements. There is no other exist in the standard.
I think I can solve the problem using an additional container (to collect and keep all entries), but is it really needed? -- this is a good idea if you want to re-scan your data structs. It could be much better from the performance point of view. Verilog simulator usually has a separately generated vpi database which it needs to consult, probably multiple times to provide you with vpi access information.
Is there a way to find out the number of entries in the vpi_scan loop without actual scanning through the loop? not really. there are no defined way to query this information from the iterator. There might be a work-around with using vpi_handle(vpiUse, iterator) but it depends on the underlying data and the type of iteration. It is much easier to use linked lists created by the first scanning loop.
you can get all additional information for LRM or a verilog pli handbook.
In FMI 2.0, array parameters are serialized to scalar variables.
Importing tools can display them as arrays, but their size is fixed and their handling is inefficient.
Better array support is currently in development by a working group of the FMI project, but I would like to know about workarounds how to handle array parameters in the meantime.
Ideas are to
hard code them (disadvantage: the are no paramters any more ...)
put them in a CSV file in the resources folder and read them at the start of the simulation (disadvantage: no parameter mask support, complicated)
put them in a string parameter and parse it at simulation start (disadvantage: limited length of strings, complicated)
Are there other ideas / workarounds? Thanks in advance.
Combinations of the ideas outlined in your question are also possible.
Hard code with selector parameter
Here the idea is to hard code several variants of your array and allow the user to select one with a parameter.
I did this in a recent project where a user needed to choose between different spatially resolved initial conditions (e.g. temperature profiles). We used a model to generate more than 100 different sets of spatially resolved initial conditions (each representing a different "history" of the modeled object), hard coded them as FORTRAN arrays (the inner core of the FMU was in FORTRAN), and used a single integer parameter to select which profile he wanted to use.
It worked very well and the user has no way of breaking it.
Shorten the array and interpolate
If the data in your array is smooth, you might be able to dramatically reduce the number of values you actually need to pass to your simulation - which would make serialization into scalar parameters less painful.
Within the FMU, interpolate to get the resolution you need.
String parameter to select csv file
You can use a string parameter to provide the path to a user-provided csv-file. I would not recommend this, because the user will most likely break it.
I have two dimensions - Invoice_In and Invoice_Out. I need to create a new dimension Invoice which combines both of these. Is there any easy way to do this with a TI process (or any other way using TI or Performance Modeler)? Thanks.
Have you consulted the Reference Guide (TM1 TurboIntegrator Functions chapter) about this?
You could use the All subsets of the two dimensions as a data source and iterate through both in the Metadata tab using two processes (or a master process which calls the same process and passes it parameters) but it would be just as easy (and more importantly you could keep it in one process by) doing this in the Prolog tab with a data source of None:
Use DimensionExists as an argument to an If() block to determine
whether the dimension Invoice exists;
If it doesn't, use DimensionCreate to create it. Add any consolidations that you want to add using DimensionElementInsert statements.
Use the DimSiz Rules function to get the number of elements in Invoice_In and Invoice_Out and store both in variables;
Your first loop iterates through InvoiceIn using a While block to count from 1 to the DimSiz value.
In your loop you would obtain the existing element using DimNm(). (You will also need to use ElLev or DType if you want to obtain only the N level elements.) You insert each element into Invoice through DimensionElementInsert. You may also need to use DimensionElementComponentAdd to add it to any top level consolidation.
Your second loop would do exactly the same but for Invoice_Out.
Where you may run into issues is if you have the same element names in both dimensions. DimensionElementInsert won't spit the dummy over that but it will ignore the insertion when it's encountered the second time.
Do NOT call any other processes which are intended to refer to this new dimension in the Prolog. You need to cross the Metadata boundary to ensure that the new dimension is registered with the server.
Export both Elements, copy and paste both list into one sheet.
Use the sheet as a source then use one line of code DimensionElementInsert in your TI.
DimensionElementInsert(DimName, InsertionPoint, ElName, ElType);
Alternatively, use the existing dimensions as a source. Then you don't need to construct a file.
You can set the datasourcename and cycle through N amount of dimensions.
(note: The new dimension needs to exist. Or you can create a new dimension within your TI. Depends how much you want to code. But I gave you the solution with the least coding).