-=-=-=-=-
Edit #3
Okay, this is just baffling as heck.
I've narrowed this down to one single line...
A sample of my code, with three examples:
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
// line 1 (LookUp(Master_Transaction_Log, Result in Scan_Code).Scan_Code) in Result,
// line 2 (LookUp(Master_Transaction_Log, "Comms Lockoff" in Transaction_Type).Scan_Code) in Result,
// line 3 (LookUp(Master_Transaction_Log, "Comms Lockoff" in Transaction_Type And Result in Scan_Code).Scan_Code) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
My database has one single entry that fits the above criteria.
When I uncomment only line 1, the routine runs great. SC_OnCommsHold receives one record, and SC_Remainder_1 gets the other three.
When I uncomment only line 2, the routine runs great. SC_OnCommsHold receives one record, and SC_Remainder_1 gets the other three.
When I uncomment only line 3, the result fails. SC_OnCommsHold receives no records. All records get piled into SC_Remainder_1
It seems to me like it's got to be some sort of logical error in line 3, but I an absolutely baffled.
-=-=-=-=-
Edit #2
The following sequence produces the result I'm looking for:
ClearCollect(SC_OnCommsHold, ScanDataCollection_SmartComm_MasterList);
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
Clear(SC_OnCommsHold);
Clear(SC_Remainder_1);
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
(LookUp(Master_Transaction_Log, Result in Scan_Code, Scan_Code)) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
but the following sequence fails out (produces an empty Collection for SC_OnCommsHold):
ClearCollect(SC_OnCommsHold, ScanDataCollection_SmartComm_MasterList);
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
Clear(SC_OnCommsHold);
Clear(SC_Remainder_1);
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
(LookUp(Master_Transaction_Log, Result in Scan_Code && "Comms Lockoff" in Transaction_Type, Scan_Code)) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
I have confirmed that there is in fact a data entry in Master_Transaction_Log that fits that criteria.
I assume my LookUp line needs to be better, but this is about as simple as I can imagine and it's still not working. Argh.
-=-=-=-=-
Edited for clarity #1 (I hope!):
The Master_Transactions_Log database is well over half a million entries, but my scans are typically under a hundred in count.
My four test entries (saved into ScanDataCollection_SmartComm_MasterList) are:
REQ2069120.RITM2374312.01
REQ2075966.RITM2382958.01
REQ2081369.RITM2389938.01
REQ2091095.RITM2402123.01
I have manually added a record into the SQL database (for testing purposes, and I know at this moment that there is only one single entry that matches this criteria) such that there is a single record where Scan_Code = "REQ2081369.RITM2389938.01" and Transaction_Type = "New Equipment Delivery - Comms Lockoff" (just trying to keep it simple right now)
The result should be two collections:
SC_OnCommsHold with a column called "Result_OnHold" which only contains a single entry:
REQ2081369.RITM2389938.01
SC_Remainder_1 with a column called "Result" which contains all entries from ScanDataCollection_SmartComm_MasterList minus entries that are now in SC_OnCommsHold:
REQ2069120.RITM2374312.01
REQ2075966.RITM2382958.01
REQ2091095.RITM2402123.01
My code at-the-moment is:
ClearCollect(SC_OnCommsHold, RenameColumns(ScanDataCollection_SmartComm_MasterList,"Result","Result_OnHold"));
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
//RemoveIf(SC_OnCommsHold, "REQ2081369.RITM2389938.01" in Result_OnHold);
RemoveIf(SC_OnCommsHold, IsEmpty(Filter(Master_Transaction_Log, Scan_Code = Result_OnHold, Transaction_Type="New Equipment Delivery - Comms Lockoff")));
It takes a few seconds to run, so I suspect it's actually looking through the database, but the result is that SC_OnCommsHold ends up with all four test codes, and that is a non-starter for the rest of everything. My little commented-out line (#3) above was my helper to make sure I was thinking right on the data structures, but the actual implementation above (line #4) fails.
I keep thinking that this line is the one kicking me, but I can't figure out what I need.
After I can get SC_OnCommsHold properly populated, I'm hoping to be able to use the same trick to depopulate those elements from SC_Remainder_1.
-=-=-=-=-
I'm really struggling with this...
I start with a collection: ScanDataCollection_SmartComm_MasterList
It looks like this:
Result
REQ1991799.RITM2280596.01
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
I have a database table:
Master_Transaction_Log
...which has three particular columns of interest:
Timestamp
Scan_Code
Transaction_Type
I would like to end up with TWO collections:
SC_ReturnToDepot
Result
REQ1991799.RITM2280596.01
SC_Remainder_1
Result
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
The criteria is as follows: for any given Result in ScanDataCollection_SmartComm_MasterList, if:
A database record has Scan_Code = Result AND Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" AND Timestamp > 72 hours ago, then that value of Result is added to SC_ReturnToDepot
SC_Remainder_1 are all remaining values that do not fit the above criteria.
I got as far as this so far, but it's killin' me after this:
ClearCollect(SC_ReturnToDepot,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
ClearCollect(SC_Remainder_1,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
I have a feeling if I can just nail that one single line of code, I am off to the races, but this is just... ugh, my brain is being a jerk.
-=-=-=-=-=-
Clarifications
This is part of a many-steps cascading process that starts with
ScanDataCollection_SmartComm_MasterList
I am not in a position where I'm able to change the data on the SQL table, nor can I switch
all my logic structure over to doing it all in Stored Procedures,
etc.
If there was a way in PowerApps to send a specific SQL query,
and that specific SQL query would produce the right results in a
Collection, then I'm open to that, but the result has to be the
Collection as described above, because I use these resultant Collections to
feed other processes.
There are a couple of ways in which you can go about this scenario. You can create each collection with two Filter expressions, where the condition of the second one is the negation of the first, something along the lines of
ClearCollect(
SC_ReturnToDepot,
Filter(
Master_Transaction_Log,
Scan_Code in ScanDataCollection_SmartComm_MasterList,
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days)));
ClearCollect(
SC_Remainder_1,
Filter(
Master_Transaction_Log,
Not(
Scan_Code in ScanDataCollection_SmartComm_MasterList,
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days))));
Another option would be to use a ForAll expression and for each record, use an If expression to insert into one of the two collections, similar to the formula below:
Clear(SC_ReturnToDepot);
Clear(SC_Remainder_1);
ForAll(
Master_Transaction_Log,
If(
Scan_Code in Master_Transaction_Log
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days),
Collect(SC_ReturnToDepot, ThisRecord),
Collect(SC_Remainder_1, ThisRecord)))
Related
Summary
I have a database of ~1500 entries. Some of them have more than one entry (up to 5) under the same identifier. I have a pre-set list of IDs I would like to check to see if they have multiple records, and if so, pull all of them, so I can compare them for differences.
Current solution
I currently have this working correctly, using a method to pull the 1st, 2nd, 3rd, 4th and 5th response respectively. The current formula is:
=INDEX('DATABASE'!$B:$B,SMALL(IF($A2='DATABASE'!$A:$A,ROW('DATABASE'!$A:$A)-ROW('DATABASE'!$A$1)+1),1)) for the 1st result, =INDEX('DATABASE'!$B:$B,SMALL(IF($A2='DATABASE'!$A:$A,ROW('DATABASE'!$A:$A)-ROW('DATABASE'!$A$1)+1),2)) for the second, and so on. In this case, the above code would check the value in A2 (ID2Check), and then do a match for it on the 'DATABASE' sheet in column A. If it matches, it will return the first value that matches from column B. The second piece of code above does the same, except it returns the second match from column B. ID2Check is populated by me, and the Results 1-5 are the code output. For example:
Database:
ID
Data Item
AAA
Lemons
111
Greenhouse
FOO
Computer
AAA
Monitor
CAT
Coffee
ORANGE
Pintglass
123
Birthday
FOO
Avengers
AAA
Plasters
FOO
NachoTaco
Code output:
ID2Check
Result 1
Result 2
Result 3
Result 4
Result 5
AAA
Lemons
Monitor
123
Birthday
FOO
Computer
Avengers
NachoTaco
Problem
This solution works correctly, and will pull the expected values as required. The problem is, as this code needs to be entered 5 times per identifier, and there are ~1500 records to check, I need to flash fill 7500 formulas. Unsurprisingly, this leads to Excel crashing and becoming unresponsive for the best part of 30 minutes. I am looking to find a solution that would be more efficient and stable when running en-mass. Any assistance would be appreciated.
Final Notes / Info
I'm using MSO 365, version 2202 (I cannot update beyond this). This will be run in the Desktop version of Excel. I would prefer this is done exclusively using formulas, but I am open to using Visual Basic if it would be otherwise impossible or incredibly inefficient.
=UNIQUE(C5:C15)
=TRANSPOSE(FILTER($D$5:$D$15;$C$5:$C$15=F5))
If you have access to HSTACK() function then could try-
=HSTACK(UNIQUE(A2:A11),TEXTSPLIT(TEXTJOIN("/",FALSE,BYROW(UNIQUE(A2:A11),LAMBDA(x,TEXTJOIN("|",TRUE,FILTER(B2:B11,A2:A11=x))))),"|","/",,,""))
Short form of my question: Is there a way to split a Collection into several sub-Collections, based on SQL database entries?
Update
Okay, a simpler version, perhaps. I'm really struggling with this...
I start with a collection: ScanDataCollection_SmartComm_MasterList
It looks like this:
Result
REQ1991799.RITM2280596.01
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
I have a database table:
Master_Transaction_Log
...which has three particular columns of interest:
Timestamp
Scan_Code
Transaction_Type
I would like to end up with TWO collections:
SC_ReturnToDepot
Result
REQ1991799.RITM2280596.01
SC_Remainder_1
Result
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
The criteria is as follows: for any given Result in ScanDataCollection_SmartComm_MasterList, if:
A database record has Scan_Code = Result AND Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" AND Timestamp > 72 hours ago, then that value of Result is added to SC_ReturnToDepot
SC_Remainder_1 are all remaining values that do not fit the above criteria.
I got as far as this so far, but it's killin' me after this:
ClearCollect(SC_ReturnToDepot,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
I have a feeling if I can just nail that one single line of code, I am off to the races, but this is just... ugh, my brain is being a jerk.
-=-=-=-=-=-=-
Longer more detailed explanation:
I'm trying to create a mechanism that takes a list of Scan Codes from a physical in-field process (scanning codes on boxes on a shelf), and for each Code, and reviews a SQL database table (Master_Transaction_Log), looking for that code. It creates another Collection that includes those associated Scan Codes, in an exclusive fashion.
At a high level:
Collection ScanDataCollection_SmartComm_MasterList, which contains one column Result, must be split out into the seven different lists, below. These lists are in a specific sequence (because we send communications in a specific sequence):
Collection Name: ScanDataCollection_SmartComm_ReturnToDepotImmediately - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" and where Timestamp is older than 48 hours. - Notes: The technician will be instructed to immediately place this item into an outgoing bin for pickup. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_AnnounceRemovalOfItem - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Final Warning" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that the items will be returned to the Depot. - Data Note: This Scan Code must not be included in any Collection below
this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFinalWarning - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Warning" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is their last chance to pick them up. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFirstWarning - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Reminder" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is a warning to pick them up. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendReminder - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: First Contact" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is a reminder to collect their order. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFirstContact - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code and where Transaction_Type = "New Equipment Delivery - Received at Stockroom/Tech Bar". (note: no time delay -- this should go out immediately upon arriving at a location) - Notes: The customer receives an email for these, announcing the orders are ready to pick up.
Collection Name: ScanDataCollection_SmartComm_NoActionTaken - Criteria: The given Scan Code (Result) has not been added to any of the items above, based on previous logical rules. For example, a Reminder might have been sent, but it was sent only ten hours ago, so we take no action on this item.
Each of these COLLECTIONS only needs a single column: Result.
It's important that no single Scan Code reside in more than one collection -- they are built to address an escalating notification sequence. For example, a Scan Code for an item that has been on a shelf for a week may already have a First Contact sent, a Reminder sent, and a Warning sent. It must ONLY go into the ScanDataCollection_SmartComm_SendFinalWarning Collection, because all items in that Collection will be sent Final Warnings.
I already have written the parts of the program that generates and sends the appropriate emails (including householding, which was a tough nut to crack). I think all I need is to be able to split my Master Collection into these sub-Collections and then I can simply attack each sub-Collection using its own ForAll loop.
I would dearly dearly appreciate advice on this! And of course, I'm happy to offer clarifications where needed.
-=-=-=-=-
Larger perspective: Right now, our techs organize incoming physical boxes across different physical shelves, and each day they move sections of boxes and perform different comms based on shelf. For example, Shelf #3 gets Reminders sent, and then all items are moved to Shelf #4 for the next day. But as our production system ramps up, more boxes arrive each day. So instead of having the technicians determine which comm to send based on which shelf, and moving dozens of boxes each day, I just want them to scan the entire room and then let the tool look into the database and decide what is the next most appropriate comm to send. This will save them about 20-40 minutes a day. Furthermore, in the near future, once this system is working well, I plan to centralize comms from a single technician, instead of each tech at each remote location performing this function. But later, later...
In my Anylogic model I succesfully create plots of datasets that count the number of trucks arriving from terminals each hour in my simulation. Now, I want to add the actual/"observed" number of trucks arriving at a terminal, to compare my simulation to these numbers. I added these numbers in a database table (see picture below). Is there a simple way of adding this data to the plot?
I tried it by creating a variable that reads the database table for every hour and adding that to a dataset (like can be seen in the pictures below), but this did not work unfortunately (the plot was empty).
Maybe simply delete the variable and fill the dataset at the start of the model by looping through the dbase table data. Use the dbase query wizard to create a for-loop. Something like this should work:
int numEntries = (int) selectFrom(observed_arrivals).count();
DataSet myDataSet = new DataSet(numEntries);
List<Tuple> rows = selectFrom(observed_arrivals).list();
for (Tuple
row : rows) {
myDataSet.add(row.get( observed_arrivals.hour ), row.get( observed_arrivals.terminal_a ));
}
myChart.addDataSet(myDataSet);
You don't explain why it "didn't work" (what errors/problems did you get?), nor where you defined these elements.
(1) Since you want both observed (empirical) and simulated arrivals per terminal, datasets for each should be in the Terminal agent. And then the replicated plot (in Main) can have two data entries referring to data sets terminals(index).observedArrivals and terminals(index).simulatedArrivals or whatever you name them.
(2) Using getHourOfDay to add to the observed dataset is wrong because that just returns 0-23 (i.e., the hour in the current day for the current model date). Your database table looks like it has hours since model start, so you just want time(HOUR) to get the model time in elapsed hours (irrespective of what the model time unit is). Or possibly time(HOUR) - 1 if you only want to update the empirical arrivals for the hour at the end of that hour (i.e., at the same time that you updated the simulated arrivals).
(3) Using a Variable to get the database value each hour doesn't work because a variable's initial value is only evaluated once at model initialisation. You want an hourly cyclic Event in Terminal instead which adds the relevant row's value. (You need to use the Insert Database Query wizard to generate the relevant Java code for the query you need in the event's action.)
(4) Because you have a database table with specifically-named columns for each terminal (columns terminal_a and presumably terminal_b etc.) that makes it slightly more awkward. (This isn't proper relational table design where, instead of 4 columns for the 4 terminals, you'd instead have two columns for terminal_id and observed_value with a row for each time period and terminal combination.)
So your database query expression (in your Terminal agents) will need to use the SQL format (not the QueryDSL format) so that you can 'stitch in' the correct column name into the SQL.
I'm trying to create a simple chart to show how many employees are on a given corrective action level within a specified date range. The issue I'm running into is this:
The log shows associate Test 1 received a verbal warning on 8/14/19 for their productivity, then a first written warning on 8/24/19, then a final written warning that was processed later but took place on 8/23/19.
The formula I wrote will show this as 1 person at each level of correction (verbal, first written, and final written). I want it to only count the highest-level warning for each person. So the chart would only count 1 entry at the final written warning level.
What am I missing to accomplish this?
Raw Data:
Summary Chart:
Summary Chart Formula (across from Verbal level):
={SUM(--(FREQUENCY(IF(('2019'!$C:$C<>"")*('2019'!$F:$F=$B$2)*('2019'!$D:$D>=$B$3)*('2019'!$D:$D<=$C$3)*('2019'!$E:$E=$B5)*('2019'!$E:$E<>$B6),MATCH('2019'!$C:$C,'2019'!$C:$C,0)),ROW('2019'!$C:$C)-ROW('2019'!$C$2)+1)>0))}''''
Cracked it! I added two helper columns to the raw data in between Step and Reason.
The first, Level, is a VLOOKUP that converts the Step to a numerical value (in order of severity, the lowest being a Verbal, highest being an Exit).
The second, Max, is a MAXIFS formula to flag which step is the highest severity by associate ID and Reason:
=IF(MAXIFS(F:F,C:C,C2,H:H,H2,D:D,">="&Summary!$B$3,D:D,"<="&Summary!$C$3)=F2,"X","")
The formula in the summary chart now reads as follows:
=SUM(--(FREQUENCY(IF(('2019'!$C:$C<>"")*('2019'!$I:$I=$B$2)*('2019'!$D:$D>=$B$3)*('2019'!$D:$D<=$C$3)*('2019'!$F:$F=$B5)*('2019'!$H:$H="X"),MATCH('2019'!$C:$C,'2019'!$C:$C,0)),ROW('2019'!$C:$C)-ROW('2019'!$C$1)+1)>0))
I have a table that contains test marks from different terms, ca1_percent, sa1_percent, ca2_percent and sa2_percent. These 4 fields reside in the Results table that contains results from the different terms.
I used a self-relationship linking using the matched field overall_percent_match which is calculated using year & " " & subject & " " & _kf_studentID. This relationship allows me to obtain the test results from past terms (of a year). For example, my term 3 results will contain results from term 1 and term 2 (of each subject). All works fine unless there is a new student who joins mid way of the year. If he joins in term 3, his ca2 results (done in term 3) will fall into his ca1_percent column (which is supposed to contain term 1 results) like other records before him.
Image shows what I mean.
I could not figure out the solution. Can anyone help me?
This StackOverflow link contains more details of my work that was done related to this problem.
The underlying problem, per your prior query, is that you're pulling the values through:
GetNthRecord(SA1_Results_Match::mark_percent,2)
This statement assumes the existence of an N=1, N=2 and N=3. To make this work properly you could do any of the following:
Ensure that your Results table always has records from the prior semester, even if the student joins later in the semester. You could keep using GetNthRecord this way, but you will always need to ensure that the records are in order.
Use an ExecuteSQL statement to gather only the correct semester's results for the correct summary field.
Make four separate relationships, with separate Table Occurrences, to define ca1, sa1, ca2 and sa2 each separately. This looks like what you started out trying to do in the prior question.