CakePHP 3.0: How Can I Organize These Model Functions - cakephp

Question

I am working on a CakePHP3 application that will be used to display information about which products our suppliers are currently offering.
** Different Vendors provide their product lists in different ways, CSV, JSON, or by way of a web scrape **
I have 2 models that I have created:
Vendors - This references a specific Vendor that we use.
VendorProducts - This references all the products that all of our vendors offer.
I would like to be able to call something like:
$vendor->getAvailableProducts()
and have it either get the CSV and parse it, grab the JSON, or scrape the suppliers website and use this to populate the VendorProducts table in the database with products from this supplier.
I understand the idea behind Fat Models and Skinny Controllers, however I'm having a bit of difficulty implementing this feature.
I would like to provide the following functionality.
The Vendor's getAvailableProducts() function can be called via the web interface AND/OR a cakephp shell script that could be run in a cron job.
As some functionality (like scraping the website) takes a considerable amount of time, I would like to be able to see the progress of this function in the view,
eg: X/Y Products Updated from {Supplier}.
This can be broken down into the following questions:
1. Which file should my "getAvailableProducts()" function go in?
2. As each $vendor has a unique updateProducts() function, how would the correct function be called from $vendor->getAvailableProducts()
// something like this?
public function getAvailableProducts() {
if($vendor->name == "SupplierA") {
getProductsFromSupplierA();
}
if($vendor->name == "SupplierB") {
getProductsFromSupplierB();
}
..., etc.
}
3. How can the progress of this function be returned to a View?

Answer 1

Don't use table classes for that create a new namespace within the model layer or in the app itself:
src/Vendor
src/Model/Vendor
Have a factory that constructs and returns you the Vendor classes:
$vendorA = VendorFactory::get('SupplierA');
$vendorB = VendorFactory::get('SupplierB');
Each vendor class must implement a method fetchProducts(), use an interface or an abstract base class for that.
The method should return a normalized array that can be used to turn the products in entities:
$this->newEntities(VendorFactory::get('SupplierA')->fetchProducts());
You'll have a hard time determining the progress if there is no way to know the total amount of records. Which is likely when you scrape the website. Same issue applies when the API doesn't tell you the total amount of records per JSON data set. If you're able to get that total count somehow you can do this:
$this->newEntities(VendorFactory::get('SupplierA')->fetchProducts([
'limit' => 50,
'offset' => 0
]);
And implement pagination for the vendor which you can then use to run over all the records in a while() loop in chunks of X records. If you trigger that via shell you can create a "job" for that and update the progress after each chunk. There are multiple existing solutions for this kind of task already out there. Finally use Comet or Websockets to get the status updated on your website. Or the old way: Trigger an AJAX request every X seconds to check the status.
There is a lot more that could be said, but this is actually already a very broad question, there is very likely not enough detail to cover all cases. Also it might be possible (I'm pretty sure) there are different ways to solve this.