PowerApps - split Collection into smaller Collections based on database data - database

Short form of my question: Is there a way to split a Collection into several sub-Collections, based on SQL database entries?
Update
Okay, a simpler version, perhaps. I'm really struggling with this...
I start with a collection: ScanDataCollection_SmartComm_MasterList
It looks like this:
Result
REQ1991799.RITM2280596.01
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
I have a database table:
Master_Transaction_Log
...which has three particular columns of interest:
Timestamp
Scan_Code
Transaction_Type
I would like to end up with TWO collections:
SC_ReturnToDepot
Result
REQ1991799.RITM2280596.01
SC_Remainder_1
Result
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
The criteria is as follows: for any given Result in ScanDataCollection_SmartComm_MasterList, if:
A database record has Scan_Code = Result AND Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" AND Timestamp > 72 hours ago, then that value of Result is added to SC_ReturnToDepot
SC_Remainder_1 are all remaining values that do not fit the above criteria.
I got as far as this so far, but it's killin' me after this:
ClearCollect(SC_ReturnToDepot,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
I have a feeling if I can just nail that one single line of code, I am off to the races, but this is just... ugh, my brain is being a jerk.
-=-=-=-=-=-=-
Longer more detailed explanation:
I'm trying to create a mechanism that takes a list of Scan Codes from a physical in-field process (scanning codes on boxes on a shelf), and for each Code, and reviews a SQL database table (Master_Transaction_Log), looking for that code. It creates another Collection that includes those associated Scan Codes, in an exclusive fashion.
At a high level:
Collection ScanDataCollection_SmartComm_MasterList, which contains one column Result, must be split out into the seven different lists, below. These lists are in a specific sequence (because we send communications in a specific sequence):
Collection Name: ScanDataCollection_SmartComm_ReturnToDepotImmediately - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" and where Timestamp is older than 48 hours. - Notes: The technician will be instructed to immediately place this item into an outgoing bin for pickup. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_AnnounceRemovalOfItem - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Final Warning" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that the items will be returned to the Depot. - Data Note: This Scan Code must not be included in any Collection below
this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFinalWarning - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Warning" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is their last chance to pick them up. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFirstWarning - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: Reminder" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is a warning to pick them up. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendReminder - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code, where Transaction_Type = "New Equipment Delivery - Cust. Msg: First Contact" and where Timestamp is older than 48 hours. - Notes: The customer receives an email for these items, announcing that this is a reminder to collect their order. - Data Note: This Scan Code must not be included in any Collection below this one on this list.
Collection Name: ScanDataCollection_SmartComm_SendFirstContact - Criteria: If there is an entry in the database table where Scan_Code = the given Scan Code and where Transaction_Type = "New Equipment Delivery - Received at Stockroom/Tech Bar". (note: no time delay -- this should go out immediately upon arriving at a location) - Notes: The customer receives an email for these, announcing the orders are ready to pick up.
Collection Name: ScanDataCollection_SmartComm_NoActionTaken - Criteria: The given Scan Code (Result) has not been added to any of the items above, based on previous logical rules. For example, a Reminder might have been sent, but it was sent only ten hours ago, so we take no action on this item.
Each of these COLLECTIONS only needs a single column: Result.
It's important that no single Scan Code reside in more than one collection -- they are built to address an escalating notification sequence. For example, a Scan Code for an item that has been on a shelf for a week may already have a First Contact sent, a Reminder sent, and a Warning sent. It must ONLY go into the ScanDataCollection_SmartComm_SendFinalWarning Collection, because all items in that Collection will be sent Final Warnings.
I already have written the parts of the program that generates and sends the appropriate emails (including householding, which was a tough nut to crack). I think all I need is to be able to split my Master Collection into these sub-Collections and then I can simply attack each sub-Collection using its own ForAll loop.
I would dearly dearly appreciate advice on this! And of course, I'm happy to offer clarifications where needed.
-=-=-=-=-
Larger perspective: Right now, our techs organize incoming physical boxes across different physical shelves, and each day they move sections of boxes and perform different comms based on shelf. For example, Shelf #3 gets Reminders sent, and then all items are moved to Shelf #4 for the next day. But as our production system ramps up, more boxes arrive each day. So instead of having the technicians determine which comm to send based on which shelf, and moving dozens of boxes each day, I just want them to scan the entire room and then let the tool look into the database and decide what is the next most appropriate comm to send. This will save them about 20-40 minutes a day. Furthermore, in the near future, once this system is working well, I plan to centralize comms from a single technician, instead of each tech at each remote location performing this function. But later, later...

Related

Powerapps - Split Collection into smaller Collections based on SQL lookup data

-=-=-=-=-
Edit #3
Okay, this is just baffling as heck.
I've narrowed this down to one single line...
A sample of my code, with three examples:
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
// line 1 (LookUp(Master_Transaction_Log, Result in Scan_Code).Scan_Code) in Result,
// line 2 (LookUp(Master_Transaction_Log, "Comms Lockoff" in Transaction_Type).Scan_Code) in Result,
// line 3 (LookUp(Master_Transaction_Log, "Comms Lockoff" in Transaction_Type And Result in Scan_Code).Scan_Code) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
My database has one single entry that fits the above criteria.
When I uncomment only line 1, the routine runs great. SC_OnCommsHold receives one record, and SC_Remainder_1 gets the other three.
When I uncomment only line 2, the routine runs great. SC_OnCommsHold receives one record, and SC_Remainder_1 gets the other three.
When I uncomment only line 3, the result fails. SC_OnCommsHold receives no records. All records get piled into SC_Remainder_1
It seems to me like it's got to be some sort of logical error in line 3, but I an absolutely baffled.
-=-=-=-=-
Edit #2
The following sequence produces the result I'm looking for:
ClearCollect(SC_OnCommsHold, ScanDataCollection_SmartComm_MasterList);
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
Clear(SC_OnCommsHold);
Clear(SC_Remainder_1);
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
(LookUp(Master_Transaction_Log, Result in Scan_Code, Scan_Code)) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
but the following sequence fails out (produces an empty Collection for SC_OnCommsHold):
ClearCollect(SC_OnCommsHold, ScanDataCollection_SmartComm_MasterList);
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
Clear(SC_OnCommsHold);
Clear(SC_Remainder_1);
ForAll(ScanDataCollection_SmartComm_MasterList,
If(
(LookUp(Master_Transaction_Log, Result in Scan_Code && "Comms Lockoff" in Transaction_Type, Scan_Code)) in Result,
Collect(SC_OnCommsHold,ThisRecord),
Collect(SC_Remainder_1,ThisRecord));
);
I have confirmed that there is in fact a data entry in Master_Transaction_Log that fits that criteria.
I assume my LookUp line needs to be better, but this is about as simple as I can imagine and it's still not working. Argh.
-=-=-=-=-
Edited for clarity #1 (I hope!):
The Master_Transactions_Log database is well over half a million entries, but my scans are typically under a hundred in count.
My four test entries (saved into ScanDataCollection_SmartComm_MasterList) are:
REQ2069120.RITM2374312.01
REQ2075966.RITM2382958.01
REQ2081369.RITM2389938.01
REQ2091095.RITM2402123.01
I have manually added a record into the SQL database (for testing purposes, and I know at this moment that there is only one single entry that matches this criteria) such that there is a single record where Scan_Code = "REQ2081369.RITM2389938.01" and Transaction_Type = "New Equipment Delivery - Comms Lockoff" (just trying to keep it simple right now)
The result should be two collections:
SC_OnCommsHold with a column called "Result_OnHold" which only contains a single entry:
REQ2081369.RITM2389938.01
SC_Remainder_1 with a column called "Result" which contains all entries from ScanDataCollection_SmartComm_MasterList minus entries that are now in SC_OnCommsHold:
REQ2069120.RITM2374312.01
REQ2075966.RITM2382958.01
REQ2091095.RITM2402123.01
My code at-the-moment is:
ClearCollect(SC_OnCommsHold, RenameColumns(ScanDataCollection_SmartComm_MasterList,"Result","Result_OnHold"));
ClearCollect(SC_Remainder_1, ScanDataCollection_SmartComm_MasterList);
//RemoveIf(SC_OnCommsHold, "REQ2081369.RITM2389938.01" in Result_OnHold);
RemoveIf(SC_OnCommsHold, IsEmpty(Filter(Master_Transaction_Log, Scan_Code = Result_OnHold, Transaction_Type="New Equipment Delivery - Comms Lockoff")));
It takes a few seconds to run, so I suspect it's actually looking through the database, but the result is that SC_OnCommsHold ends up with all four test codes, and that is a non-starter for the rest of everything. My little commented-out line (#3) above was my helper to make sure I was thinking right on the data structures, but the actual implementation above (line #4) fails.
I keep thinking that this line is the one kicking me, but I can't figure out what I need.
After I can get SC_OnCommsHold properly populated, I'm hoping to be able to use the same trick to depopulate those elements from SC_Remainder_1.
-=-=-=-=-
I'm really struggling with this...
I start with a collection: ScanDataCollection_SmartComm_MasterList
It looks like this:
Result
REQ1991799.RITM2280596.01
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
I have a database table:
Master_Transaction_Log
...which has three particular columns of interest:
Timestamp
Scan_Code
Transaction_Type
I would like to end up with TWO collections:
SC_ReturnToDepot
Result
REQ1991799.RITM2280596.01
SC_Remainder_1
Result
REQ2048874.RITM2349401.01
REQ2037354.RITM2335400.01
The criteria is as follows: for any given Result in ScanDataCollection_SmartComm_MasterList, if:
A database record has Scan_Code = Result AND Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot" AND Timestamp > 72 hours ago, then that value of Result is added to SC_ReturnToDepot
SC_Remainder_1 are all remaining values that do not fit the above criteria.
I got as far as this so far, but it's killin' me after this:
ClearCollect(SC_ReturnToDepot,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
ClearCollect(SC_Remainder_1,
ForAll(ScanDataCollection_SmartComm_MasterList,
...?
);
);
I have a feeling if I can just nail that one single line of code, I am off to the races, but this is just... ugh, my brain is being a jerk.
-=-=-=-=-=-
Clarifications
This is part of a many-steps cascading process that starts with
ScanDataCollection_SmartComm_MasterList
I am not in a position where I'm able to change the data on the SQL table, nor can I switch
all my logic structure over to doing it all in Stored Procedures,
etc.
If there was a way in PowerApps to send a specific SQL query,
and that specific SQL query would produce the right results in a
Collection, then I'm open to that, but the result has to be the
Collection as described above, because I use these resultant Collections to
feed other processes.
There are a couple of ways in which you can go about this scenario. You can create each collection with two Filter expressions, where the condition of the second one is the negation of the first, something along the lines of
ClearCollect(
SC_ReturnToDepot,
Filter(
Master_Transaction_Log,
Scan_Code in ScanDataCollection_SmartComm_MasterList,
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days)));
ClearCollect(
SC_Remainder_1,
Filter(
Master_Transaction_Log,
Not(
Scan_Code in ScanDataCollection_SmartComm_MasterList,
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days))));
Another option would be to use a ForAll expression and for each record, use an If expression to insert into one of the two collections, similar to the formula below:
Clear(SC_ReturnToDepot);
Clear(SC_Remainder_1);
ForAll(
Master_Transaction_Log,
If(
Scan_Code in Master_Transaction_Log
And Transaction_Type = "New Equipment Delivery - Cust. Msg: Equipment Returning to Depot"
And Timestamp > DateAdd(Now(), -3, TimeUnit.Days),
Collect(SC_ReturnToDepot, ThisRecord),
Collect(SC_Remainder_1, ThisRecord)))

Anylogic: How to create plot from database table?

In my Anylogic model I succesfully create plots of datasets that count the number of trucks arriving from terminals each hour in my simulation. Now, I want to add the actual/"observed" number of trucks arriving at a terminal, to compare my simulation to these numbers. I added these numbers in a database table (see picture below). Is there a simple way of adding this data to the plot?
I tried it by creating a variable that reads the database table for every hour and adding that to a dataset (like can be seen in the pictures below), but this did not work unfortunately (the plot was empty).
Maybe simply delete the variable and fill the dataset at the start of the model by looping through the dbase table data. Use the dbase query wizard to create a for-loop. Something like this should work:
int numEntries = (int) selectFrom(observed_arrivals).count();
DataSet myDataSet = new DataSet(numEntries);
List<Tuple> rows = selectFrom(observed_arrivals).list();
for (Tuple
row : rows) {
myDataSet.add(row.get( observed_arrivals.hour ), row.get( observed_arrivals.terminal_a ));
}
myChart.addDataSet(myDataSet);
You don't explain why it "didn't work" (what errors/problems did you get?), nor where you defined these elements.
(1) Since you want both observed (empirical) and simulated arrivals per terminal, datasets for each should be in the Terminal agent. And then the replicated plot (in Main) can have two data entries referring to data sets terminals(index).observedArrivals and terminals(index).simulatedArrivals or whatever you name them.
(2) Using getHourOfDay to add to the observed dataset is wrong because that just returns 0-23 (i.e., the hour in the current day for the current model date). Your database table looks like it has hours since model start, so you just want time(HOUR) to get the model time in elapsed hours (irrespective of what the model time unit is). Or possibly time(HOUR) - 1 if you only want to update the empirical arrivals for the hour at the end of that hour (i.e., at the same time that you updated the simulated arrivals).
(3) Using a Variable to get the database value each hour doesn't work because a variable's initial value is only evaluated once at model initialisation. You want an hourly cyclic Event in Terminal instead which adds the relevant row's value. (You need to use the Insert Database Query wizard to generate the relevant Java code for the query you need in the event's action.)
(4) Because you have a database table with specifically-named columns for each terminal (columns terminal_a and presumably terminal_b etc.) that makes it slightly more awkward. (This isn't proper relational table design where, instead of 4 columns for the 4 terminals, you'd instead have two columns for terminal_id and observed_value with a row for each time period and terminal combination.)
So your database query expression (in your Terminal agents) will need to use the SQL format (not the QueryDSL format) so that you can 'stitch in' the correct column name into the SQL.

Access Query & Email assistance

I'm decent when it comes to working with Access, and can usually figure things out but am currently stumped on this query process.
I only have 2 shipment tables. One contains historical data (tblBookedLoads), while the second table (RMCData) contains current orders.
What I need to do, is for each row in RMCData, figure out who has driven the same route recently (Origin to Destination) and send them an email with the details of the matching shipments... and any others that match, IE, if I have 50 shipments from Atlanta to Memphis tomorrow, I don't want to send them 50 emails.
I'm just not sure about how to go about retrieving the information since it it stored in different formats, or how to create a query based on row by row data from fields in another table... Let me know what you think. I'm guessing it has something to do with looping in vba, but not sure.
Both tables are read only.
tblRMCData 'Contains details for new orders (approx. 1000 rows)
' has fields 'ReferenceNum','Ship Date','Origin City', 'Origin State','Origin Zip','destination', 'Weight','Comments","Comments2"
'Origin/Destination City, State and Zip are separated
tblBookedLoads 'contains history of orders (approx. 20000 rows). CONTAINS DUPLICATES
' has fields 'Carrier', 'Ship Date", 'Order','Origin', 'Destination','Weight','Bill Amount",'email address'
' Origin/Destination is populated as "LITTLE ROCK, AR 12345"
Any advise is appreciated.

Take UNIQUE rows value under conditions

I have a G-spreadsheet where I applied an API connector, that takes data (invoices data) from an external DB; every day the G-sheet receives the last 250 rows Data from the external DB, so some rows are news and others are repeated and added again below in the list.
Usually, I have 2-3 times repeated rows with the same information and sometimes I can have repeated value like: invoiceID - invoice number - ecc., but with a different amount (because the administration system sometimes has to correct the invoices).
My question is How can I create a formula in a new sheet collecting the UNIQUE newest values of the repeated rows?
As you can see the Fetch date is the API connector that for 3 days repeated the value, and on 29 it gots a different amount after correction of the invoice from the admin department.
try SORTN with the 3rd parameter set to 2:
=SORTN(SORT(B2:G; 1; 0); 99^99; 2; 5; 0)

Database Structure and Querying

Trying to figure out how to change a structure from what I currently have which is this:
tblHaulLogs
intLogID
intHaulType
intSerial
intOriginSource
intOrigin
intDestinationSource
intDestination
dtmHaulDate
ccyLogPay
intHauler
txtLogNotes
intInvoiceID
In this table, what I am doing is using the origin and destination source fields to determine which table the fk for the origin and destination comes from. This feels very wrong to me.
tblHaulTypes
intHaulTypeID
chrHaulType
intOriginSourceType
intDestinationSourceType
Data in the Haul Types Table:
LOT, 1, 1
DEL, 1, 2
RPO, 2, 1
Now let me explain:
The first type happens when an item goes from a sales lot to another sales lot.
The second type happens when an item goes from a sales lot to a customer(sale gets delivered).
The third type happens when an item returns from the customer back to the sales lot.
Then the Item can be resold/returned/resold/returned(rent-to-own system).
Now, here are the problems I have:
An Haul Log's origin will always be the destination of the last move. Therefore I thought that the origin field is redundant. However, it's the relation between the destination of the last move and the destination of the new move that defines what the shipper gets paid and what type of haul it is.
In other words, even though the first type and the third type technically have the same fields, the type of move is not the same because of the previous move type. What do I need to do here? Am I totally missing the boat on what the structure should be?
The questions I need to answer based on this data is:
How many Items do I have on my sales lots that are new inventory(have never been sold).
How many Items do I have that have been sold and returned(doesn't matter how many times).
I'm guessing at the relationship between the various fields and tables.
Your tblHaulTypes table looks fine.
intHaulTypeID
chrHaulType
intOriginSourceType
intDestinationSourceType
You're missing a haul type that accounts for deliveries from suppliers to your lots.
There has to be some table that lists your lots. I'd call it tblHaulLot.
intLotNumber
txtLotName
...
I'd make a tblHaulTransaction table that looks like this.
intTransactionID
intHaulTypeID
intHauler
intOriginOrganizationID
intDestinationOrganizationID
intOriginLot (null if origin is supplier)
intDestinationLot (null if destination is customer)
dtmHaulDate
txtLogNotes
Now, we need an tblOrganization.
intOrganizationID
txtOrganizationName
txtOrganizationAddress
...
The organization at ID 0 is your organization. Suppliers and customers would fill the rest of the table.
I'd make a tblHaulInvoice table that looks like this.
intInvoiceID
intTransactionID
ccyTransactionPay
dtmDateInvoiced
AmountInvoiced
The amount invoiced (and amount paid) have to be accounted for in some table. I don't know what ccy stands for, and I don't know your 3 letter code for a decimal (money) field.
How many Items do I have on my sales lots that are new inventory(have never been sold). How many Items do I have that have been sold and returned(doesn't matter how many times).
Nowhere in your data model is there any kind of inventory table. I'd need to know a lot more about your business to create one or more inventory tables.

Resources