I have a case that I am curious about for a long, long time. Say I have 3 models:
class Product(models.Model):
manufacturer = models.CharField(max_length=100)
description = models.TextField(blank=True)
class PurchasedProduct(models.Model):
product = models.ForeignKey(Product)
purchase = models.ForeignKey('Purchase')
quantity = models.PositiveIntegerField()
class Purchase(models.Model):
customer = models.ForeignKey('customers.Customer')
products = models.ManyToManyField(Product, through=PurchasedProduct)
comment = models.CharField(max_length=200)
I have an API and client application written in some JavaScript framework. So now I need to communicate between them! I am not sure how should I handle this situation in DRF, naturally I would expect to get something like this when accessing /purchase/1/
{
"id": 1,
"customer": 1,
"comment": "Foobar",
"products": [
{
"id": 1,
"product": {
"id": 1,
....
},
....
},
....
]
}
So I created proper serializer specifying that products field should use PurchasedProductSerializer which in turn uses nested ProductSerializer. It is fine cause I get all necessary info to, say, display what specific products where purchased and in what quantity during shopping using appropriate components in say React.
The problem for me is however when I need to POST new PurchasedProduct. I would expect the most convenient form to be:
{
"quantity": 10,
"purchase": 1,
"product": 1
}
As it carries all necessary info and has the smallest footprint. However I can't be accomplished using PurchasedProductSerializer as it requires product to be object instead of id.
So my question here, is this a good approach (it seems very natural to me), should I use two separate serializers for GET and POST? Should I perform this differently? Could you point me to some best practices/books how to write APIs and client apps?
I had the exact same problem a few months back and would've been more than happy if someone would've told me. I ended up with the exact solution that you proposed to add products to a purchase. I do agree that your proposed POST request it is the most natural way with the minimal required footprint.
In order to correctly process the POST request's data correctly, though, I ended up using two separate Serializers, just as you described. If you're using DRF viewsets, one way to select the correct serializer on GET and POST is to override the get_serializer_class method as described here.
The deserializer for POST requests could look like this:
class PurchasedProductDeserializer(serializers.ModelSerializer):
product = serializers.PrimaryKeyRelatedField(queryset=Product.objects.all())
purchase = serializers.PrimaryKeyRelatedField(queryset=Purchase.objects.all())
class Meta:
model = PurchasedProduct
fields = ('id', 'product', 'purchase', 'quantity')
write_only_fields = ('product', 'purchase', 'quantity')
That deserializer can then be used for input validation and finally to add a product to a purchase (or increase its quantity).
E.g., inside your viewset:
def create(self, request, *args, **kwargs):
# ...
# init your serializer here
serializer = self.get_serializer(data=request.data)
if serializer.is_valid(raise_exception=True):
# now check if the same item is already in the cart
try:
# try to find the product in the list of purchased products
purchased_product = serializer.validated_data['purchase'].purchasedproduct_set.get(product=serializer.validated_data['product'])
# if so, simply increase its quantity, else add the product as a new item to the cart (see except case)
purchased_product.quantity += serializer.validated_data['quantity']
purchased_product.save()
# update the serializer so it knows the id of the existing instance
serializer.instance = purchased_product
except PurchasedProduct.DoesNotExist:
# product is not yet part of the purchase cart, add it now
self.perform_create(serializer)
# ...
# do other stuff here
As for best practices, there's a ton of documentation available on the internet, but if you're looking for books you might wanna look at some of the ones posted here. When you get bored of REST you might even wanna look into GraphQL.
Related
I have a DailyReflectionPage Model with a reflection_date field that forms the basis for the Page's slug, which is in the form YYYY-MM-DD. Here's an extract of my Page model:
class DailyReflectionPage(Page):
"""
The Daily Reflection Model
"""
...
...
reflection_date = models.DateField("Reflection Date", max_length=254)
...
...
#cached_property
def date(self):
"""
Returns the Reflection's date as a string in %Y-%m-%d format
"""
fmt = "%Y-%m-%d"
date_as_string = (self.reflection_date).strftime(fmt)
return date_as_string
...
...
def full_clean(self, *args, **kwargs):
# first call the built-in cleanups (including default slug generation)
super(DailyReflectionPage, self).full_clean(*args, **kwargs)
# now make your additional modifications
if self.slug is not self.date:
self.slug = self.date
...
...
These daily reflections are written by different authors, as part of a booklet that is published towards the end of the year, for use in the coming year. I would like to have a workflow where, for instance, the daily reflections from January to June are reviewed by one group, and those from July to December are reviewed by another group, as illustrated in the diagram below:
How can this be achieved?
This should be able to be achieved by creating ONE new Workflow Task type that has a relationship to two sets of User Groups (e.g. a/b or before/after, it is probably best to keep this generic in the model definition).
This new Task can be created as part of a new Workflow within the Wagtail admin, and each of the groups linked to the Moderator Group 1 / 2.
Wagtail's methods on the Task allow you to return approval options based on the Page model for any created workflow, from here you can look for a method that would be on the class and assign the groups from there.
The benefits of having a bit more of a generic approach is that you could leverage this for any splitting of moderator assignments as part of future Workflow tasks.
Implementation Overview
1 - read the Wagatail Docs on how to add a new Task Type and the Task model reference to understand this process.
2 - Read through the full implementation in the code of the built in GroupApprovalTask.
3 - In the GroupApprovalTask you can see that the methods with overrides all rely on the checking of self.groups but they all get the page passed in as a arg to those methods.
4 - Create a new Task that extends the Wagtail Task class and on this model create two ManyToManyField that allow for two sets of user groups being linked (note: you do not have do to this as two fields, you could put a model in the middle but the example below is just the simplest way to get to the gaol).
5 - On the DailyReflectionPage model create a method get_approval_group_key which will return maybe a simple Boolean or a 'A' or 'B' based on the business requirements you described above (check the model's date etc)
6 - In your custom Task create a method that abstracts the checking of the Page for this method and returns the Tasks' user group. You may want to add some error handling and default values. E.g. get_approval_groups
7 - Add a custom method for each of the 'start', 'user_can_access_editor', page_locked_for_user, user_can_lock, user_can_unlock, get_task_states_user_can_moderate methods that calls get_approval_group with the page and returns the values (see the code GroupApprovalTask for what these should do.
Example Code Snippets
models.py
class DailyReflectionPage(Page):
"""
The Daily Reflection Model
"""
def get_approval_group_key(self):
# custom logic here that checks all the date stuff
if date_is_after_foo:
return 'A'
return 'B'
class SplitGroupApprovalTask(Task):
## note: this is the simplest approach, two fields of linked groups, you could further refine this approach as needed.
groups_a = models.ManyToManyField(
Group,
help_text="Pages at this step in a workflow will be moderated or approved by these groups of users",
related_name="split_task_group_a",
)
groups_b = models.ManyToManyField(
Group,
help_text="Pages at this step in a workflow will be moderated or approved by these groups of users",
related_name="split_task_group_b",
)
admin_form_fields = Task.admin_form_fields + ["groups_a", "groups_b"]
admin_form_widgets = {
"groups_a": forms.CheckboxSelectMultiple,
"groups_b": forms.CheckboxSelectMultiple,
}
def get_approval_groups(self, page):
"""This method gets used by all checks when determining what group to allow/assign this Task to"""
# recommend some checks here, what if `get_approval_group` is not on the Page?
approval_group = page.specific.get_approval_group_key()
if (approval_group == 'A'):
return self.group_a
return self.group_b
# each of the following methods will need to be implemented, all checking for the correct groups for the Page when called
# def start(self, ...etc)
# def user_can_access_editor(self, ...etc)
# def page_locked_for_user(self, ...etc)
# def user_can_lock(self, ...etc)
# def user_can_unlock(self, ...etc)
def get_task_states_user_can_moderate(self, user, **kwargs):
# Note: this has not been tested, however as this method does not get `page` we must find all the tasks allowed indirectly via their TaskState pages
tasks = TaskState.objects.filter(status=TaskState.STATUS_IN_PROGRESS, task=self.task_ptr)
filtered_tasks = []
for task in tasks:
page = task.select_related('page_revision', 'task', 'page_revision__page')
groups = self.get_approval_groups(page)
if groups.filter(id__in=user.groups.all()).exists() or user.is_superuser:
filtered_tasks.append(task)
return TaskState.objects.filter(pk__in=[task.pk for task in filtered_tasks])
def get_actions(self, page, user):
# essentially a copy of this method on `GroupApprovalTask` but with the ability to have a dynamic 'group' returned.
approval_groups = self.get_approval_groups(page)
if approval_groups.filter(id__in=user.groups.all()).exists() or user.is_superuser:
return [
('reject', "Request changes", True),
('approve', "Approve", False),
('approve', "Approve with comment", True),
]
return super().get_actions(page, user)
What options are there there for resolving nested resources in ngResource responses?
There have been some related questions about resolving endpoints for nested resource in ngResource, but this question is about when a REST response contains a second resource nested in the collection that is being queried, especially 1-to-1 mappings where you wouldn't have e.g. pets/<id>/owner as its own resource.
Say there are two resources, Pets and Owners:
GET /pets:
[{
name: 'spark',
type: 'dog',
owner: '/owners/3/' # alternatively just '3' or the full object.
}]
As a developer, I sometimes want to query the Owner resource as a whole, sometimes I want to query the Pet resource and then I automatically want to resolve the owner attribute into a resource instance.
This is my current solution:
.factory('Pet', function ($resource, Owner) {
var Pet = $resource('/pets/:id', {id: '#id'});
Pet.prototype.getOwner = function () {
return new Owner(this.owner); // or Owner.get({id: this.owner})
}
return Pet;
})
Problems here are many. There's integrity – for one. This implementation, I believe, allows for multiple instances of the same resource. Then there's practicality. You also have additional attributes to keep track of (owner and getOwner(), instead of just owner; possibly setOwner if you want to be able to save the model).
An alternative solution could be built on transformResponse, but it would feel like a hack to include that in every resource that has a nested mapping.
I believe this is the exact reason why Martin Gontovnikas created Restangular. He didn't like having to deal with nested $resources in the main angular framework. I think his Restangular solution would fit nicely into your needs. His code is on GitHub here and he's got a nice intro video on youtube here.
Check it out. I think you'll find it does exactly what you want it to do.
Update: I ended up working on this for a bit and have started a new angular module, available on GitHub. The answer below is about the Gist I wrote originally.
There doesn't seem to be anything around there like what I have been looking for. I have started an implementation of a solution that only supports get and getList (query) operations. The remaining methods should be trivial to add since I've pretty much kept with the layout of the ngResource module. The Gist for my implementation is below.
https://gist.github.com/lyschoening/7102262
Resources can be embedded in JSON either as full objects that simply get wrapped in the correct Resource model, or as URIs, which get resolved automatically. In addition to embedded resources, the module also supports typical nested resources, either as true parent-child collections (where the resource is only accessible after selecting the parent) or as cross-referenced collection.
Yard = Resource('/yard') # resource model
Yard.$nested('trees') # embedded item or list of items
Chair = Resource('/chair')
Yard.$nested('/chair') # sub-collection without its own model
# (for many-to-many)
Tree = Resource('/tree')
# child-collection with its own model
TreeHouse = Tree.$childResource('/treehouse')
yard = Yard.get(1)
# GET /yard/1
# {
# "uri": "/yard/1",
# "trees": [
# "/tree/15", -- reference, looked-up automatically with GET
# {"uri": "/tree/16", "name": "Apple tree"}
# -- full object, resolved to Tree instance
# ]
# }
# GET /tree/16
# {"uri": "/tree/15", "name": "Pine tree"}
yard.chair.getList()
# GET /yard/1/chair
# [{"uri": "/chair/1", ...}, ..]
# -- model inferred from URI
yard.trees[0].treehouse.getList()
# GET /tree/15/treehouse
# [{"uri": "/tree/15/treehouse/1", ...}, ..]
# -- automatically resolved to TreeHouse instance
I have a Django-tastypie resource that represents a banner and has a field called impression that I increment whenever the banner appears on the site.
class BannerResource(ModelResource):
owner = fields.ForeignKey('advertisment.api.AdvertiserResource', 'owner', full=True)
class Meta:
queryset = Banner.objects.all()
resource_name = 'banner'
authorization = Authorization()
I would like to get the banner that has the minimum impression, in the official documentation there is nothing like
filtering = {'impressions': ('min',)}
I'm using BackboneJS in the front end and I could get all the banners with Backbone collection and do the filtering with JavaScript but I'm looking for a quicker way to do it.
Any ideas?
Thanks
If you'd like to retrieve banners with number of impressions greater than X you need to things. For one you need to define possible filtering operations on your resource like so (given your model has impressions field):
class BannerResource(ModelResource):
owner = fields.ForeignKey('advertisment.api.AdvertiserResource', 'owner', full=True)
class Meta:
queryset = Banner.objects.all()
resource_name = 'banner'
authorization = Authorization()
filtering = { 'impressions' : ALL }
for available options take a look at Tastypie's documentation on filtering.
Then if you made the following request:
GET http://<your_host>/v1/banners?impressions__gte=X
you should get what you need.
I want to create two objects and link them via a parent child relationship in C# using the Metadata API.
I can create objects and 'custom' fields for the objects via the metadata, but the service just ignores the field def for the relationship.
By snipet for the fields are as follows:
CustomField[] fields = new CustomField[] { new CustomField()
{
type = FieldType.Text,
label = "FirstName",
length = 50,
lengthSpecified = true,
fullName = "LJUTestObject__c.FirstName__c"
},
new CustomField()
{
type = FieldType.Text,
label = "LastName",
length = 50,
lengthSpecified = true,
fullName = "LJUTestObject__c.Lastname__c"
},
new CustomField()
{
type = FieldType.Text,
label = "Postcode",
length = 50,
lengthSpecified = true,
fullName = "LJUTestChildObject__c.Postcode__c"
},
new CustomField()
{
type = FieldType.MasterDetail,
relationshipLabel = "PostcodeLookup",
relationshipName = "LJUTestObject__c.LJUTestObject_Id__c",
relationshipOrder = 0,
relationshipOrderSpecified = true,
fullName = "LJUTestChildObject__c.Lookup__r"
}
};
The parent object looks like:
LJUTestObject
ID,
FirstName, Text(50)
LastName, Text(50)
The child objext looks like:
LJUTestChildObject
ID,
Postcode, Text(50)
I want to link the parent to the child so one "LJUTestObject", can have many "LJUTestChildObjects".
What values do I need for FieldType, RelationshipName, and RelationshipOrder to make this happen?
TL;DR:
Use this as a template for accomplishing what you want:
var cf = new CustomField();
cf.fullName = "ChildCustomObject__c.ParentCustomField__c";
cf.type = FieldType.MasterDetail;
cf.typeSpecified = true;
cf.label = "Parent Or Whatever You Want This To Be Called In The UI";
cf.referenceTo = "ParentCustomObject__c";
cf.relationshipName = "ParentOrWhateverYouWantThisToBeCalledInternally";
cf.relationshipLabel = "This is an optional label";
var aUpsertResponse = smc.upsertMetadata(metadataSession, null, null, new Metadata[] { cf });
The key difference:
The natural temptation is to put the CustomField instances into the fields array of a CustomObject, and pass that CustomObject to the Salesforce Metadata API. And this does work for most data fields, but it seems that it does not work for relationship fields.
Instead, pass the CustomField directly to the Salesforce Metadata API, not wrapped in a CustomObject.
Those muted errors:
Turns out that errors are occurring, and the Salesforce Metadata API knows about them, but doesn't bother telling you about them when they occur for CustomFields nested inside a CustomObject.
By passing the CustomField directly to the Metadata API (not wrapped in a CustomObject), the call to upsertMetadata will still return without an exception being thrown (as it was already doing for you), but this time, if something goes wrong, upsertResponse[0].success will be false instead of true, and upsertResponse[0].errors will give you more information.
Other gotchas
Must specify referenceTo, and if it doesn't match the name of an existing built-in or custom object, the error message will be the same as if you had not specified referenceTo at all.
fullName should end in __c not __r. __r is for relationship names, but remember that fullName is specifying the field name, not the relationship name.
relationshipName - I got it working by not including __r on the end, and not including the custom object name at the start. I haven't tested to be sure other ways don't work, but be aware that at the very least, you don't need to have those extra components in the relationshipName.
Remember generally that anything with label in its name is probably for display to users in the UI, and thus can have spaces in it to be nicely formatted the way users expect.
Salesforce... really???
(mini rant warning)
The Salesforce Metadata API is unintuitive and poorly documented. That's why you got stuck on such a simple thing. That's why no-one knew the answer to your question. That's why, four years later, I got stuck on the same thing. Creating relationships is one of the main things you would want to do with the Salesforce Metadata API, and yet it has been this difficult to figure out, for this long. C'mon Salesforce, we know you're a sales company more than a tech company, but you earn trazillions of dollars and are happy to show it off - invest a little more in a better API experience for the developers who invest in learning your platform.
I've not created these through the meta data API like this myself, but I'd suggest that:
relationshipName = "LJUTestObject__c.LJUTestObject_Id__c
Should be:
relationshipName = "LJUTestObject__c.Id
as Id is a standard field, the __c suffix is only used for custom fields (not standard fields on custom objects). Also, it may be that the relationship full name should end in __c not __r, but try the change above first and see how you go.
SELECT
Id,
OwnerId,
WhatId,
Reminder_Date_Time__c,
WhoId,
Record_Type_Name__c,
Task_Type__c,
Assigned_Date__c,
Task_Status__c,
ActivityDate,
Subject,
Attended_By__c,
Is_Assigned__c
FROM Task
WHERE
(NOT Task_Status__c LIKE 'Open') AND
ActivityDate >= 2017-12-13 AND
(NOT Service__r.Service_State__c LIKE 'Karnataka')
django nonrel's documentation states: "you have to manually write code for merging the results of multiple queries (JOINs, select_related(), etc.)".
Can someone point me to any snippets that manually add the related data? #nickjohnson has an excellent post showing how to do this with the straight AppEngine models, but I'm using django-nonrel.
For my particular use I'm trying to get the UserProfiles with their related User models. This should be just two simple queries, then match the data.
However, using django-nonrel, a new query gets fired off for each result in the queryset. How can I get access to the related items in a 'select_related' sort of way?
I've tried this, but it doesn't seem to work as I'd expect. Looking at the rpc stats, it still seems to be firing a query for each item displayed.
all_profiles = UserProfile.objects.all()
user_pks = set()
for profile in all_profiles:
user_pks.add(profile.user_id) # a way to access the pk without triggering the query
users = User.objects.filter(pk__in=user_pks)
for profile in all_profiles:
profile.user = get_matching_model(profile.user_id, users)
def get_matching_model(key, queryset):
"""Generator expression to get the next match for a given key"""
try:
return (model for model in queryset if model.pk == key).next()
except StopIteration:
return None
UPDATE:
Ick... I figured out what my issue was.
I was trying to improve the efficiency of the changelist_view in the django admin. It seemed that the select_related logic above was still producing additional queries for each row in the results set when a foreign key was in my 'display_list'. However, I traced it down to something different. The above logic does not produce multiple queries (but if you more closely mimic Nick Johnson's way it will look a lot prettier).
The issue is that in django.contrib.admin.views.main on line 117 inside the ChangeList method there is the following code: result_list = self.query_set._clone(). So, even though I was properly overriding the queryset in the admin and selecting the related stuff, this method was triggering a clone of the queryset which does NOT keep the attributes on the model that I had added for my 'select related', resulting in an even more inefficient page load than when I started.
Not sure what to do about it yet, but the code that selects related stuff is just fine.
I don't like answering my own question, but the answer might help others.
Here is my solution that will get related items on a queryset based entirely on Nick Johnson's solution linked above.
from collections import defaultdict
def get_with_related(queryset, *attrs):
"""
Adds related attributes to a queryset in a more efficient way
than simply triggering the new query on access at runtime.
attrs must be valid either foreign keys or one to one fields on the queryset model
"""
# Makes a list of the entity and related attribute to grab for all possibilities
fields = [(model, attr) for model in queryset for attr in attrs]
# we'll need to make one query for each related attribute because
# I don't know how to get everything at once. So, we make a list
# of the attribute to fetch and pks to fetch.
ref_keys = defaultdict(list)
for model, attr in fields:
ref_keys[attr].append(get_value_for_datastore(model, attr))
# now make the actual queries for each attribute and store the results
# in a dict of {pk: model} for easy matching later
ref_models = {}
for attr, pk_vals in ref_keys.items():
related_queryset = queryset.model._meta.get_field(attr).rel.to.objects.filter(pk__in=set(pk_vals))
ref_models[attr] = dict((x.pk, x) for x in related_queryset)
# Finally put related items on their models
for model, attr in fields:
setattr(model, attr, ref_models[attr].get(get_value_for_datastore(model, attr)))
return queryset
def get_value_for_datastore(model, attr):
"""
Django's foreign key fields all have attributes 'field_id' where
you can access the pk of the related field without grabbing the
actual value.
"""
return getattr(model, attr + '_id')
To be able to modify the queryset on the admin to make use of the select related we have to jump through a couple hoops. Here is what I've done. The only thing changed on the 'get_results' method of the 'AppEngineRelatedChangeList' is that I removed the self.query_set._clone() and just used self.query_set instead.
class UserProfileAdmin(admin.ModelAdmin):
list_display = ('username', 'user', 'paid')
select_related_fields = ['user']
def get_changelist(self, request, **kwargs):
return AppEngineRelatedChangeList
class AppEngineRelatedChangeList(ChangeList):
def get_query_set(self):
qs = super(AppEngineRelatedChangeList, self).get_query_set()
related_fields = getattr(self.model_admin, 'select_related_fields', [])
return get_with_related(qs, *related_fields)
def get_results(self, request):
paginator = self.model_admin.get_paginator(request, self.query_set, self.list_per_page)
# Get the number of objects, with admin filters applied.
result_count = paginator.count
# Get the total number of objects, with no admin filters applied.
# Perform a slight optimization: Check to see whether any filters were
# given. If not, use paginator.hits to calculate the number of objects,
# because we've already done paginator.hits and the value is cached.
if not self.query_set.query.where:
full_result_count = result_count
else:
full_result_count = self.root_query_set.count()
can_show_all = result_count self.list_per_page
# Get the list of objects to display on this page.
if (self.show_all and can_show_all) or not multi_page:
result_list = self.query_set
else:
try:
result_list = paginator.page(self.page_num+1).object_list
except InvalidPage:
raise IncorrectLookupParameters
self.result_count = result_count
self.full_result_count = full_result_count
self.result_list = result_list
self.can_show_all = can_show_all
self.multi_page = multi_page
self.paginator = paginator