How do I migrate data from one model to another using South in Django? - database

I created a Django app that had its own internal voting system and a model called Vote to track it. I want to refactor the voting system into its own app so I can reuse it. However, the original app is in production and I need to create a data migration that will take all the Votes and transplant them into the separate app.
How can I get two apps to participate in a migration so that I have access to both their models? Unfortunately, the original and separate apps both have a model named Vote now, so I need to be aware of any conflicts.

Have you tried db.rename_table?
I would start by creating a migration in either the new or old app that looks something like this.
class Migration:
def forwards(self, orm):
db.rename_table('old_vote', 'new_vote')
def backwards(self, orm):
db.rename_table('new_vote', 'old_vote')
If that does not work you can migrate each item in a loop with something along these lines:
def forwards(self, orm):
for old in orm['old.vote'].objects.all():
# create a new.Vote with old's data
models = {
'old.vote' = { ... },
'new.vote' = { ... },
}
Note: You must use orm[...] to access any models outside the app currently being migrated. Otherwise, standard orm.Vote.objects.all() notation works.

Related

Cakephp 3 - How to integrate external sources in table?

I working on an application that has its own database and gets user information from another serivce (an LDAP is this case, through an API package).
Say I have a tables called Articles, with a column user_id. There is no Users table, instead a user or set of users is retrieved through the external API:
$user = LDAPConnector::getUser($user_id);
$users = LDAPConnector::getUsers([1, 2, 5, 6]);
Of course I want retrieving data from inside a controller to be as simple as possible, ideally still with something like:
$articles = $this->Articles->find()->contain('Users');
foreach ($articles as $article) {
echo $article->user->getFullname();
}
I'm not sure how to approach this.
Where should I place the code in the table object to allow integration with the external API?
And as a bonus question: How to minimise the number of LDAP queries when filling the Entities?
i.e. it seems to be a lot faster by first retrieving the relevant users with a single ->getUsers() and placing them later, even though iterating over the articles and using multiple ->getUser() might be simpler.
The most simple solution would be to use a result formatter to fetch and inject the external data.
The more sophisticated solution would a custom association, and a custom association loader, but given how database-centric associations are, you'd probably also have to come up with a table and possibly a query implementation that handles your LDAP datasource. While it would be rather simple to move this into a custom association, containing the association will look up a matching table, cause the schema to be inspected, etc.
So I'll stick with providing an example for the first option. A result formatter would be pretty simple, something like this:
$this->Articles
->find()
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$userIds = array_unique($results->extract('user_id')->toArray());
$users = LDAPConnector::getUsers($userIds);
$usersMap = collection($users)->indexBy('id')->toArray();
return $results
->map(function ($article) use ($usersMap) {
if (isset($usersMap[$article['user_id']])) {
$article['user'] = $usersMap[$article['user_id']];
}
return $article;
});
});
The example makes the assumption that the data returned from LDAPConnector::getUsers() is a collection of associative arrays, with an id key that matches the user id. You'd have to adapt this accordingly, depending on what exactly LDAPConnector::getUsers() returns.
That aside, the example should be rather self-explanatory, first obtain a unique list of users IDs found in the queried articles, obtain the LDAP users using those IDs, then inject the users into the articles.
If you wanted to have entities in your results, then create entities from the user data, for example like this:
$userData = $usersMap[$article['user_id']];
$article['user'] = new \App\Model\Entity\User($userData);
For better reusability, put the formatter in a custom finder. In your ArticlesTable class:
public function findWithUsers(\Cake\ORM\Query $query, array $options)
{
return $query->formatResults(/* ... */);
}
Then you can just do $this->Articles->find('withUsers'), just as simple as containing.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Custom Finder Methods

Organize Models in subdirectories CakePHP 3

we are using subdirectories in our projects no separete views and controllers but in models we didn’t learn yet. Recently I’ve found this https://github.com/cakephp/cakephp/issues/60451 and actually routes and plugins we are already using, we just want to separete our models like this:
Model
-Entity
–Financial
—Money.php
-Table
–Financial
—MoneyTable.php
I’ve tryed put like this then controller is not able to find his model. How can I do to organize it, and make it work?
Things that we've tried:
Use $this->setAlias('TableModel');
Call in controller:
$this->TableModel = $this->loadModel('Subfolder/TableModel');
didn't work for SQL build, and other classes.
CakePHP uses the TableRegister to load models. That class can be configured to use a class that implements the LocatorInterface, and CakePHP uses the TableLocator as the default.
The only thing you can do is configure your own LocatorInterface instance in your bootstrap.php. You would have to create your MyTableLocator and have it change the className for tables to point to subdirectories. What rules for this class name rewritting are used is purely up to you.
bootstrap.php:
TableRegister::setTableLocator(new MyTableLocator());
MyTableLocator.php:
class MyTableLocator extends TableLocator {
protected function _getClassName($alias, array $options = [])
{
if($alias === 'Subfolder/TableModel') {
return TableModel::class;
}
return parent::_getClassName($alias, $options);
}
}
The above isn't working code.
I'm just demonstrating what the function is you need to override, and that you need logic in place to return a different class name.
You can check if the $alias contains the / character, and if so. Return a class name by extracting the subfolder name from the $alias. Take a look at the TableLocator to see how it's using the App::className function.

Yii2 + Redis as Database

I want to use Yii2 and redis as database.
So far, I got Redis ActiveRecord Class for Yii2 from Here.
link1
link2
but, I got a problem. WHY THIS CLASS ADDS ANYTHING AS HASH IN REDIS????
Above that I cant Find in which pattern it Insert data. I add one user and it will add a user under user:xxx namespace and another record under s:user:xxx and so on but none of theme has any fields that i defined in attributes!! only contain IDs.
I know that a Key-value type database and RDBMS are different and also know how can implement relation like records in Redis, but I don't know why it will only save IDs.
I could not find any example of using redis ActiveRecords so far.
There is one in here and its not good enough.
So here is my main wuestion: how can add data to redis Using activeRecords and different data types In YII2?
And if its impossible with ActiveRecords what is the best solution? in this case
ANOTHER QUESTION: is it possible to use a Model instead and write my own model::save() method? and what is the best data validation solution at this rate?
Actually I want to make a telegram bot, so i should get messages and send them in RabitMQ and get data in a worker, do the process and save results to Redis, and finally send response to user through the RabitMQ.
So I need to do a lot of validations AND OF COURSE AUTHENTICATIONS and save and select and range and save to sets an lists and this and that ....
I want a good way to make Model or active record or the proper solution to validation, save and retrieve data to Redis and Yii2.
Redis DB can be declared as a cache component or as a database connection or both.
When it is declared as a cache component (using the yii/redis/cache) it is accessible within that component to store key/value pairs as shown here.
$cache = Yii::$app->cache;
// try retrieving $data from cache
$data = $cache->get($key);
// store $data in cache so that it can be retrieved next time
$cache->set($key, $data);
// one more example:
$access_token = Yii::$app->security->generateRandomString();
$cache->add(
// key
$access_token,
// data (can also be an array)
[
'id' => Yii::$app->user->identity->id
'name' => Yii::$app->user->identity->name
],
// expires
60*60*3
);
Also other components may start using it for caching proposes like session if configured to do so or like the yii\web\UrlManager which by default will try to cache the generated URL rules in whatever valid caching mechanism defined under the config file's cache component as explained here. So it is normal to find some stored data other than yours in that case.
When Redis is declared as a DB connection like in the links you provided which means using the yii\redis\Connection class you can make your model extending its \yii\redis\ActiveRecord class as any other ActiveRecord model in Yii. The only difference I know so far is that you need to define your attributes manually as there is no DB schema to parse for NoSQL databases. Then just define your rules, scenarios, relations, events, ... as any other ActiveRecord model:
class Customer extends \yii\redis\ActiveRecord
{
public function attributes()
{
return ['id', 'name', 'address', 'registration_date'];
}
public function rules()
{
return [
['name', 'required'],
['name', 'string', 'min' => 3, 'max' => 12, 'on' => 'register'],
...
];
}
public function attributeLabels() {...}
...
}
All available methods including save(), validate(), getErrors(), ... could be found here and should be used like any other ActiveRecord class as shown in the official guide.

ngResource resolving nested resources

What options are there there for resolving nested resources in ngResource responses?
There have been some related questions about resolving endpoints for nested resource in ngResource, but this question is about when a REST response contains a second resource nested in the collection that is being queried, especially 1-to-1 mappings where you wouldn't have e.g. pets/<id>/owner as its own resource.
Say there are two resources, Pets and Owners:
GET /pets:
[{
name: 'spark',
type: 'dog',
owner: '/owners/3/' # alternatively just '3' or the full object.
}]
As a developer, I sometimes want to query the Owner resource as a whole, sometimes I want to query the Pet resource and then I automatically want to resolve the owner attribute into a resource instance.
This is my current solution:
.factory('Pet', function ($resource, Owner) {
var Pet = $resource('/pets/:id', {id: '#id'});
Pet.prototype.getOwner = function () {
return new Owner(this.owner); // or Owner.get({id: this.owner})
}
return Pet;
})
Problems here are many. There's integrity – for one. This implementation, I believe, allows for multiple instances of the same resource. Then there's practicality. You also have additional attributes to keep track of (owner and getOwner(), instead of just owner; possibly setOwner if you want to be able to save the model).
An alternative solution could be built on transformResponse, but it would feel like a hack to include that in every resource that has a nested mapping.
I believe this is the exact reason why Martin Gontovnikas created Restangular. He didn't like having to deal with nested $resources in the main angular framework. I think his Restangular solution would fit nicely into your needs. His code is on GitHub here and he's got a nice intro video on youtube here.
Check it out. I think you'll find it does exactly what you want it to do.
Update: I ended up working on this for a bit and have started a new angular module, available on GitHub. The answer below is about the Gist I wrote originally.
There doesn't seem to be anything around there like what I have been looking for. I have started an implementation of a solution that only supports get and getList (query) operations. The remaining methods should be trivial to add since I've pretty much kept with the layout of the ngResource module. The Gist for my implementation is below.
https://gist.github.com/lyschoening/7102262
Resources can be embedded in JSON either as full objects that simply get wrapped in the correct Resource model, or as URIs, which get resolved automatically. In addition to embedded resources, the module also supports typical nested resources, either as true parent-child collections (where the resource is only accessible after selecting the parent) or as cross-referenced collection.
Yard = Resource('/yard') # resource model
Yard.$nested('trees') # embedded item or list of items
Chair = Resource('/chair')
Yard.$nested('/chair') # sub-collection without its own model
# (for many-to-many)
Tree = Resource('/tree')
# child-collection with its own model
TreeHouse = Tree.$childResource('/treehouse')
yard = Yard.get(1)
# GET /yard/1
# {
# "uri": "/yard/1",
# "trees": [
# "/tree/15", -- reference, looked-up automatically with GET
# {"uri": "/tree/16", "name": "Apple tree"}
# -- full object, resolved to Tree instance
# ]
# }
# GET /tree/16
# {"uri": "/tree/15", "name": "Pine tree"}
yard.chair.getList()
# GET /yard/1/chair
# [{"uri": "/chair/1", ...}, ..]
# -- model inferred from URI
yard.trees[0].treehouse.getList()
# GET /tree/15/treehouse
# [{"uri": "/tree/15/treehouse/1", ...}, ..]
# -- automatically resolved to TreeHouse instance

Adding a 1 to many file upload to CRUD

My app has sales listing functionality that will allow the user to add 1 or more photos for the product that they want to sell.
I'm attempting to use the upload/filestore_image of ATK with a Join table to create the relationship - my models:
class Model_Listing extends Model_Table {
public $entity_code='listing';
function init(){
parent::init();
$this->addField('name');
$this->addField('body')->type('text');
$this->addField('status');
$this->addField('showStatus')->calculated(true);
}
function calculate_showStatus(){
return ($this->status == 1) ? "Sold" : "For Sale" ;
}
}
class Model_listingimages extends Model_Table {
public $entity_code='listing_images';
function init(){
parent::init();
$this->addField('listing_id')->refModel('Model_Listing');
$this->addField('filestore_image_id')->refModel('Model_Filestore_Image');
}
}
In my page manager class I have added the file upload to the crud:
class page_manager extends Page {
function init(){
parent::init();
$tabs=$this->add('Tabs');
$s = $tabs->addTab('Sales')->add('CRUD');
$s->setModel('Listing',array('name','body','status'),array('name','status'));
if ($s->form) {
$f = $s->form;
$f->addField('upload','Add Photos')->setModel('Filestore_Image');
$f->add('FileGrid')->setModel('Filestore_Image');
}
}
}
My questions:
I am getting a "Unable to include FileGrid.php" error - I want the user to be able to see the images that they have uploaded and hoped that this would be the best way to do so - by adding the file grid to bottom of the form. - EDIT - ignore this question, I created a FileGrid class based on the code in the example link below - that fixed the issue.
How do I make the association between the CRUD form so that a submit will save the uploaded files and create entries in the join table?
I have installed the latest release of ATK4, added the 4 filestore tables to the db and referenced the following page in the documentation http://codepad.agiletoolkit.org/image
TIA
PG
By creating model based on Filestore_File
You need to specify a proper model. By proper I mean:
It must be extending Model_Filestore_File
It must have MasterField set to link it with your entry
In this case, however you must know the referenced ID when the images are being uploaded, so it won't work if you upload image before creating record. Just to give you idea the code would look
$mymodel=$this->add('Model_listingimages');
$mymodel->setMasterField('listing_id',$listing_id);
$upload_field->setModel($mymodel);
$upload_field->allowMultiple();
This way all the images uploaded through the field will automatically be associated with your listing. You will need to inherit model from Model_Filestore_File. The Model_Filestore_Image is a really great example which you can use. You should add related entity (join) and define fields in that table.
There is other way too:
By doing some extra work in linking images
When form is submitted, you can retrieve list of file IDs by simply getting them.
$form->get('add_photos')
Inside form submission handler you can perform some manual insertion into listingimages.
$form->onSubmit(function($form) uses($listing_id){
$photos = explode(',',$form->get('add_photos'));
$m=$form->add('Model_listingimages');
foreach($photos as $photo_id){
$m->unloadDdata()->set('listing_id',$listing_id)
->set('filestore_image_id',$photo_id)->update();
}
}); // I'm not sure if this will be called by CRUD, which has
// it's own form submit handler, but give it a try.
You must be careful, through, if you use global model inside the upload field without restrictions, then user can access or delete images uploaded by other users. If you use file model with MVCGrid you should see what files they can theoretically get access to. That's normal and that's why I recommend using the first method described above.
NOTE: you should not use spaces in file name, 2nd argument to addField, it breaks javascript.

Resources