I have this in my 2 models:
class Weather
include Mongoid::Document
store_in database: ->{ "myapp_shared_#{Rails.env}" }
...
class Event
include Mongoid::Document
belongs_to :weather
...
This is a multi-tenant app, so I have this in my application controller, to change the database based on subdomain:
class ApplicationController < ActionController::Base
before_filter :switch_database
def switch_database
Mongoid.override_database("myapp_#{subdomain_name}_#{Rails.env}")
end
...
Now when I run Event.first.weather, I see this in my debug log:
D, [2015-09-03T18:38:18] DEBUG -- : MONGODB | myapp_subdomain_development.find | STARTED | {"find"=>"events", "filter"=>{}}
D, [2015-09-03T18:38:18] DEBUG -- : MONGODB | myapp_subdomain_development.find | SUCCEEDED | 0.025892s
D, [2015-09-03T18:38:18] DEBUG -- : MONGODB | myapp_subdomain_development.find | STARTED | {"find"=>"weathers", "filter"=>{"_id"=>BSON::ObjectId('5522c1d9526f6d6b9f0c0000')}}
D, [2015-09-03T18:38:18] DEBUG -- : MONGODB | myapp_subdomain_development.find | SUCCEEDED | 0.00078s
So weather is searched in the override_database, not in the store_in database. If I do not run Mongoid.override_database, then the weather is taken from the app_shared_development database as specified in store_in.
The question is, how do I use override_database as this is a multi-tenant app, but still force certain models to always use the shared database?
It seems the best solution is to stop using override_database altogether. It has too many side effects and unintentional consequences.
I have simplified the following code, but in essence these were the steps:
1) I have abstracted all of my models to inherit from another class which looks a bit like this:
module XanModel
module Base
extend ActiveSupport::Concern
include Mongoid::Document
included do
store_in database: -> { DatabaseSwitcher.current_database_name }
end
2) Then in my model:
class Event
include XanModel::Base
3) The application controller has this:
class ApplicationController < ActionController::Base
before_filter :set_database
def set_database
DatabaseSwitcher.switch_to(subdomain)
end
4) The DatabaseSwitcher has:
class DatabaseSwitcher
def self.switch_to(database_name)
Thread.current[:database_name] = database_name
end
def self.current_database_name
Thread.current[:database_name] || \
Mongoid::Threaded.database_override || \
"xanview" # fallback
end
The end result is the application automatically switches databases based on subdomains but certain models can use a global shared database.
Related
My app links invoices, contracts and services with Many-to-One-Relationships:
class Invoice(models.Model):
contract = models.ForeignKey(Contract, on_delete=models.CASCADE)
class Contract(models.Model):
service = models.ForeignKey(Service, on_delete=models.CASCADE)
Whenever a new invoice is registered, it can be linked to a service and split/billed internally. Unfortunately, some contracts/invoices need to be linked to more than one service according to a fixed split (e.g. 30/70).
For this to work on the surface, I could to reverse the relationship between contracts and services –
class Service(models.Model):
contract = models.ForeignKey(Contract, on_delete=models.CASCADE)
– or change the ForeignKey field on the Contract class to a ManyToManyField.
But in both cases, I will not be able to get back from the invoice to the service easily anymore, as with the following statement:
invoices = Invoice.objects.filter(models.Q(contract__service__building=self.tenant.unit.building), models.Q(begin__lte=self.begin, end__gt=self.begin) | models.Q(begin__gt=self.begin, begin__lt=self.end))
Is it wise to insert an intermediate helper model (ContractService) with two ForeignKey fields to keep the current app logic and add the option to link a contract to more than one service?
Ok just to clarify one example:
Contract is "Cleaning of House"
Services are "Cleaning of first floor" and "Cleaning of second floor"
Invoices are "Invoice1", "Invoice2", ...
You want a relationship that "Invoice1" can be linked to "Cleaning of first floor" AND "Cleaning of second floor".
models.py
class Contract(models.Model):
"""Can hold multiple Services"""
pass
class Service(models.Model):
"""Is linked to one specific Contract"""
contract = models.ForeignKey(Contract, on_delete=models.CASCADE)
class Invoice(models.Model):
"""Can hold multiple Services and one Service can hold multiple Invoices"""
service = models.ManyToManyField(Service)
Now your question is:
"I can easily get the contract when I have the Service object, but how can I get the Service when I have the Contract object?
con = Contract.objects.all().first()
queryset = con.service_set.all() # gives you all related Services for that specific Contract
Read more about ManytoOne
And:
"How can I get the Invoice when I have the Service object?"
ser = Service.objects.all().first()
queryset = ser.invoice_set.all() # gives you all related Invoices for that specific Service
Read more about ManyToMany
Let me know how it goes
Thanks to #Tarquinius for your help – I found a solution based on his suggestion. The three models in question are connected as follows:
class Service(models.Model):
pass
class Contract(models.Model):
services = models.ManyToManyField(Service)
class Invoice(models.Model):
contract = models.ForeignKey(Contract, on_delete=models.CASCADE)
The query quoted above did not even need to be modified (apart from the different fieldname) to reflect the possibility of more than one service per contract, I just had to add the distinct() function:
invoices = Invoice.objects.filter(models.Q(contract__services__building=self.tenancy_agreement.unit.building), models.Q(begin__lte=self.begin, end__gt=self.begin) | models.Q(begin__gt=self.begin, begin__lt=self.end)).distinct()
In hindsight, this is quite obvious (and simple).
I have a class A
class A
include Mongoid::Document
has_many :bs
accepts_nested_attributes_for :bs
and a class B
class B
include Mongoid::Document
belongs_to :a
and a class C that inherits from B
class C < B
field :new_field, type: String
This worked fine with Mongoid 6. With Mongoid 7, on a form with fields_for, upon submit, I now get:
Attempted to set a value for 'new_field' which is not allowed on the model B
Note, this is NOT the mongoid polymorphism supported in 7.0 (I believe) b/c that is not talking about single table inheritance (STI), rather, it supports multiple tables belonging to a single class / table as the same symbol. This is not that. And I've tried using as and polymorphic:true.
Any ideas how to fix this?
Thanks,
Kevin
Turns you for Mongoid 7.0 or Rails 5.2 (not sure which change broke this) you have to set the type in the form for STI, I did this by a hidden field:
<%f.hidden_field :_type, value: "C"%>
This allows you to set descendant-only attributes.
I faced a similar sort of issue. I was upgrading mongoid from 3.0 to 7.0 in a project and STI was implemented in the old app. _type was explicitly declared in the parent table which was causing unexpected behavior. I removed it and the project started working.
When separating the API classes into multiple files, the API explorer shows the same request definition for all resources.
So based on the structure shown below (my apologies if it's too long), in the API explorer, both my_api.api_a.test and my_api.api_b.test show the same attribute, attr_b, which is the last in the api_server list definition. If I change it and put ApiA last, then both methods show attr_a.
Any idea what am I doing wrong
# model/model_a.py
class A(EndpointsModel):
attr_a = ndb.StringProperty()
# model/model_b.py
class B(EndpointsModel):
attr_b = ndb.StringProperty()
# api/__init__.py
my_api = endpoints.api(name='my_api', version='v1')
# api/api_a.py
#my_api.api_class(resource_name='api_a')
class ApiA(remote.Service):
#A.method(name='test', ...)
...
# api/api_b.py
#my_api.api_class(resource_name='api_b')
class ApiB(remote.Service):
#B.method(name='test', ...)
...
# services.py
from api import my_api
application = endpoints.api_server([ApiA, ApiB])
Also tried to define the api_server as shown below, but didn't work at all.
application = endpoints.api_server([my_api])
I've noticed similar issues (which might be a bug in the endpoints-proto-datastore libary) when the actual method names (not the name in the decorator) are the same in different api classes.
Doesn't work:
class ApiA(remote.Service):
#A.method(...)
def test(self, model):
...
class ApiB(remote.Service):
#B.method(...)
def test(self, model):
...
Works:
class ApiA(remote.Service):
#A.method(...)
def test_a(self, model):
...
class ApiB(remote.Service):
#B.method(...)
def test_b(self, model):
...
You skipped those lines in your sample, but the behaviour you state matches what I encountered in this scenario.
I am trying to create a small django project of a football application to show the stats & stuff.. to start with I created two classes in my models.py.. with a many to many relation.. but for some reason it's throwing a strange Database Error: no such table: football_league_team
please any help is appreciated, thanks in advance.
from django.db import models
# Create your models here.
class Team(models.Model):
team_name = models.CharField(max_length=30, unique=True)
team_code = models.CharField(max_length=4, unique=True)
team_home = models.CharField(max_length=30, unique=True)
team_registry_date = models.DateTimeField('Date of Registry')
def __unicode__(self):
return self.team_name
class League(models.Model):
league_name = models.CharField(max_length=30)
league_code = models.CharField(max_length=4)
league_division = models.IntegerField()
team = models.ManyToManyField(Team)
def __unicode__(self):
return self.league_name
You removed the field football_league_team in one of your model. Django doesn't know it and is still expecting said field. Depending your Django version, there are several ways to reset the corresponding model.
Django 1.4 and lower
> ./manage.py reset <appname>
I believe it works for earlier versions of Django, not sure though. Keep in mind that this option will reset each models of your application as opposed to the below method which allow single table drops.
Django 1.5 and higher
> ./manage.py sqlclear <appname>
will print out the commands to clear the database from the application's models.
> ./manage.py dbshell
Will allow you to use the sqlclear commands in order to drop the tables yopu want to be reseted.
Database migrations are a popular pattern, particularly with Ruby on Rails. Since migrations specify how to mold old data to fit a new schema, they can be helpful when you have production data that must be converted quickly and reliably.
But migrating models in App Engine is difficult since processing all entities sequentially is difficult, and there is no offline operation to migrate everything effectively in one big transaction.
What are your techniques for modifying a db.Model "schema" and migrating the data to fit the new schema?
Here is what I do.
I have a MigratingModel class, which all of my models inherit from. Here is migrating_model.py:
"""Models which know how to migrate themselves"""
import logging
from google.appengine.ext import db
from google.appengine.api import memcache
class MigrationError(Exception):
"""Error migrating"""
class MigratingModel(db.Model):
"""A model which knows how to migrate itself.
Subclasses must define a class-level migration_version integer attribute.
"""
current_migration_version = db.IntegerProperty(required=True, default=0)
def __init__(self, *args, **kw):
if not kw.get('_from_entity'):
# Assume newly-created entities needn't migrate.
try:
kw.setdefault('current_migration_version',
self.__class__.migration_version)
except AttributeError:
msg = ('migration_version required for %s'
% self.__class__.__name__)
logging.critical(msg)
raise MigrationError, msg
super(MigratingModel, self).__init__(*args, **kw)
#classmethod
def from_entity(cls, *args, **kw):
# From_entity() calls __init__() with _from_entity=True
obj = super(MigratingModel, cls).from_entity(*args, **kw)
return obj.migrate()
def migrate(self):
target_version = self.__class__.migration_version
if self.current_migration_version < target_version:
migrations = range(self.current_migration_version+1, target_version+1)
for self.current_migration_version in migrations:
method_name = 'migrate_%d' % self.current_migration_version
logging.debug('%s migrating to %d: %s'
% (self.__class__.__name__,
self.current_migration_version, method_name))
getattr(self, method_name)()
db.put(self)
return self
MigratingModel intercepts the conversion from the raw datastore entity to the full db.Model instance. If current_migration_version has fallen behind the class's latest migration_version, then it runs a series of migrate_N() methods which do the heavy lifting.
For example:
"""Migrating model example"""
# ...imports...
class User(MigratingModel):
migration_version = 3
name = db.StringProperty() # deprecated: use first_name and last_name
first_name = db.StringProperty()
last_name = db.StringProperty()
age = db.IntegerProperty()
invalid = db.BooleanProperty() # to search for bad users
def migrate_1(self):
"""Convert the unified name to dedicated first/last properties."""
self.first_name, self.last_name = self.name.split()
def migrate_2(self):
"""Ensure the users' names are capitalized."""
self.first_name = self.first_name.capitalize()
self.last_name = self.last_name.capitalize()
def migrate_3(self):
"""Detect invalid accounts"""
if self.age < 0 or self.age > 85:
self.invalid = True
On a busy site, the migrate() method should retry if db.put() fails, and possibly log a critical error if the migration didn't work.
I haven't gotten there yet, but at some point I would probably mix-in my migrations from a separate file.
Final thoughts
It is hard to test on App Engine. It's hard to get access to your production data in a test environment, and at this time it is difficult-to-impossible to make a coherent snapshot backup. Therefore, for major changes, consider making a new version that uses a completely different model name which imports from the old model and migrates as it needs. (For example, User2 instead of User). That way, if you need to fall back to the previous version, you have an effective backup of the data.