IMPORTANT: import path changed, it's now: "from polymorphic import PolymorphicModel, ..."

- added python2.4 compatibility. Contributed by Charles Leifer. Thanks!
- general reorganization of the code - there is no single polymorphic.py module anymore,
  so d-p now needs to be installed as a regular Django app
- polymorphic.VERSION/get_version added
- version numbering started: V0.5 beta
fix_request_path_info
Bert Constantin 2010-02-19 17:12:21 +01:00
parent e6c1e7ec6e
commit 116e2af08b
17 changed files with 1010 additions and 761 deletions

View File

@ -1,4 +1,4 @@
django_polymorphic was created by Bert Constantin in 2009/2010. django_polymorphic was created by Bert Constantin in 2009/2010.
setup.py contributed by Andrew Ingram Andrew Ingram contributed setup.py
Charles Leifer contributed Python 2.4 compatibility

View File

@ -14,7 +14,7 @@ Testing
------- -------
The repository (or tar file) contains a complete Django project The repository (or tar file) contains a complete Django project
that may be used for tests or experiments (without any installation needed). that may be used for tests or experiments, without any installation needed.
To run the included test suite, execute:: To run the included test suite, execute::
@ -36,13 +36,13 @@ Alternatively you can simply copy the ``polymorphic`` directory
(under "django_polymorphic") into your Django project dir. (under "django_polymorphic") into your Django project dir.
If you want to use the management command ``polymorphic_dumpdata``, then If you want to use the management command ``polymorphic_dumpdata``, then
you need to add ``polymorphic`` to your INSTALLED_APPS setting. you need to add ``polymorphic`` to your INSTALLED_APPS setting. This is also
needed if you want to run the test cases in `polymorphic/tests.py`.
In any case, Django's ContentType framework (``django.contrib.contenttypes``) In any case, Django's ContentType framework (``django.contrib.contenttypes``)
needs to be listed in INSTALLED_APPS (usually it already is). needs to be listed in INSTALLED_APPS (usually it already is).
Defining Polymorphic Models Defining Polymorphic Models
=========================== ===========================
@ -50,7 +50,7 @@ To make models polymorphic, use ``PolymorphicModel`` instead of Django's
``models.Model`` as the superclass of your base model. All models ``models.Model`` as the superclass of your base model. All models
inheriting from your base class will be polymorphic as well:: inheriting from your base class will be polymorphic as well::
from polymorphic.models import PolymorphicModel from polymorphic import PolymorphicModel
class ModelA(PolymorphicModel): class ModelA(PolymorphicModel):
field1 = models.CharField(max_length=10) field1 = models.CharField(max_length=10)
@ -244,6 +244,8 @@ from ``PolymorphicManager`` instead of ``models.Manager``. In your model
class, explicitly add the default manager first, and then your class, explicitly add the default manager first, and then your
custom manager:: custom manager::
from polymorphic import PolymorphicModel, PolymorphicManager
class MyOrderedManager(PolymorphicManager): class MyOrderedManager(PolymorphicManager):
def get_query_set(self): def get_query_set(self):
return super(MyOrderedManager,self).get_query_set().order_by('some_field') return super(MyOrderedManager,self).get_query_set().order_by('some_field')
@ -282,6 +284,8 @@ The ``PolymorphicManager`` class accepts one initialization argument,
which is the queryset class the manager should use. A custom which is the queryset class the manager should use. A custom
custom queryset class can be defined and used like this:: custom queryset class can be defined and used like this::
from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet
class MyQuerySet(PolymorphicQuerySet): class MyQuerySet(PolymorphicQuerySet):
def my_queryset_method(...): def my_queryset_method(...):
... ...
@ -409,6 +413,10 @@ Restrictions & Caveats
by subclassing it instead of modifying Django core (as we do here by subclassing it instead of modifying Django core (as we do here
with PolymorphicModel). with PolymorphicModel).
* Django Admin Integration: There currently is no admin integration,
but it surely would be nice to have one. There is a discussion about it here:
http://groups.google.de/group/django-polymorphic/browse_thread/thread/84290fe76c40c12d
* It must be possible to instantiate the base model objects, even if your * It must be possible to instantiate the base model objects, even if your
application never does this itself. This is needed by the current application never does this itself. This is needed by the current
implementation of polymorphic querysets but (likely) also by Django internals. implementation of polymorphic querysets but (likely) also by Django internals.

View File

@ -4,10 +4,10 @@ Release Notes, Usage, Code
* Please see `here for release notes, news and discussion`_ (Google Group) * Please see `here for release notes, news and discussion`_ (Google Group)
* `Many Examples`_, or full `Installation and Usage Docs`_ (or the short `Overview`_) * `Many Examples`_, or full `Installation and Usage Docs`_ (or the short `Overview`_)
* Download from GitHub_ or Bitbucket_, or as TGZ_ or ZIP_ * Download from GitHub_ or Bitbucket_, or as TGZ_ or ZIP_
* Improve django_polymorphic: Report issues, discuss, post patch, or fork the code (GitHub_, Bitbucket_, Newsgroup_, Mail_) * Improve django_polymorphic: Report issues, discuss, post patch, or fork (GitHub_, Bitbucket_, Group_, Mail_)
.. _here for release notes, news and discussion: http://groups.google.de/group/django-polymorphic/topics .. _here for release notes, news and discussion: http://groups.google.de/group/django-polymorphic/topics
.. _Newsgroup: http://groups.google.de/group/django-polymorphic/topics .. _Group: http://groups.google.de/group/django-polymorphic/topics
.. _Mail: http://github.com/bconstantin/django_polymorphic/tree/master/setup.py .. _Mail: http://github.com/bconstantin/django_polymorphic/tree/master/setup.py
.. _Installation and Usage Docs: http://bserve.webhop.org/wiki/django_polymorphic/doc .. _Installation and Usage Docs: http://bserve.webhop.org/wiki/django_polymorphic/doc
.. _Many Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc#defining-polymorphic-models .. _Many Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc#defining-polymorphic-models
@ -72,6 +72,30 @@ License
django_polymorphic uses the same license as Django (BSD-like). django_polymorphic uses the same license as Django (BSD-like).
API Change on February 22, plus Installation Note
-------------------------------------------------
The django_polymorphic source code has been restructured
and as a result needs to be installed like a normal Django App
- either via copying the "polymorphic" directory into your
Django project or by running setup.py. Adding 'polymorphic'
to INSTALLED_APPS in settings.py is still optional, however.
The file `polymorphic.py` cannot be used as a standalone
extension module anymore (as is has been split into a number
of smaller files).
Importing works slightly different now: All relevant symbols are
imported directly from 'polymorphic' instead from
'polymorphic.models'::
# new way
from polymorphic import PolymorphicModel, ...
# old way, doesn't work anymore
from polymorphic.models import PolymorphicModel, ...
Database Schema Change on January 26 Database Schema Change on January 26
------------------------------------ ------------------------------------

View File

@ -7,6 +7,9 @@ import sys, os
project_path = os.path.dirname(os.path.abspath(__file__)) project_path = os.path.dirname(os.path.abspath(__file__))
libs_local_path = os.path.join(project_path, 'libraries-local') libs_local_path = os.path.join(project_path, 'libraries-local')
if libs_local_path not in sys.path: sys.path.insert(1, libs_local_path) if libs_local_path not in sys.path: sys.path.insert(1, libs_local_path)
sys.stderr.write( 'using Python version: %s\n' % sys.version[:5])
import django import django
sys.stderr.write( 'using Django version: %s, from %s\n' % ( sys.stderr.write( 'using Django version: %s, from %s\n' % (
django.get_version(), django.get_version(),

View File

@ -1,6 +1,8 @@
# -*- coding: utf-8 -*-
from django.db import models from django.db import models
from polymorphic.models import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes
class Project(ShowFields, PolymorphicModel): class Project(ShowFields, PolymorphicModel):

View File

@ -0,0 +1,24 @@
# -*- coding: utf-8 -*-
"""
Seamless Polymorphic Inheritance for Django Models
Copyright:
This code and affiliated files are (C) by Bert Constantin and individual contributors.
Please see LICENSE and AUTHORS for more information.
"""
from polymorphic_model import PolymorphicModel
from manager import PolymorphicManager
from query import PolymorphicQuerySet
from showfields import ShowFields, ShowFieldsAndTypes
VERSION = (0, 5, 0, 'beta')
def get_version():
version = '%s.%s' % VERSION[0:2]
if VERSION[2]:
version += '.%s' % VERSION[2]
if VERSION[3]:
version += ' %s' % VERSION[3]
return version

152
polymorphic/base.py 100644
View File

@ -0,0 +1,152 @@
# -*- coding: utf-8 -*-
""" PolymorphicModel Meta Class
Please see README.rst or DOCS.rst or http://bserve.webhop.org/wiki/django_polymorphic
"""
from django.db import models
from django.db.models.base import ModelBase
from manager import PolymorphicManager
from query import PolymorphicQuerySet
# PolymorphicQuerySet Q objects (and filter()) support these additional key words.
# These are forbidden as field names (a descriptive exception is raised)
POLYMORPHIC_SPECIAL_Q_KWORDS = [ 'instance_of', 'not_instance_of']
###################################################################################
### PolymorphicModel meta class
class PolymorphicModelBase(ModelBase):
"""
Manager inheritance is a pretty complex topic which may need
more thought regarding how this should be handled for polymorphic
models.
In any case, we probably should propagate 'objects' and 'base_objects'
from PolymorphicModel to every subclass. We also want to somehow
inherit/propagate _default_manager as well, as it needs to be polymorphic.
The current implementation below is an experiment to solve this
problem with a very simplistic approach: We unconditionally
inherit/propagate any and all managers (using _copy_to_model),
as long as they are defined on polymorphic models
(the others are left alone).
Like Django ModelBase, we special-case _default_manager:
if there are any user-defined managers, it is set to the first of these.
We also require that _default_manager as well as any user defined
polymorphic managers produce querysets that are derived from
PolymorphicQuerySet.
"""
def __new__(self, model_name, bases, attrs):
#print; print '###', model_name, '- bases:', bases
# create new model
new_class = self.call_superclass_new_method(model_name, bases, attrs)
# check if the model fields are all allowed
self.validate_model_fields(new_class)
# create list of all managers to be inherited from the base classes
inherited_managers = new_class.get_inherited_managers(attrs)
# add the managers to the new model
for source_name, mgr_name, manager in inherited_managers:
#print '** add inherited manager from model %s, manager %s, %s' % (source_name, mgr_name, manager.__class__.__name__)
new_manager = manager._copy_to_model(new_class)
new_class.add_to_class(mgr_name, new_manager)
# get first user defined manager; if there is one, make it the _default_manager
user_manager = self.get_first_user_defined_manager(attrs)
if user_manager:
def_mgr = user_manager._copy_to_model(new_class)
#print '## add default manager', type(def_mgr)
new_class.add_to_class('_default_manager', def_mgr)
new_class._default_manager._inherited = False # the default mgr was defined by the user, not inherited
# validate resulting default manager
self.validate_model_manager(new_class._default_manager, model_name, '_default_manager')
return new_class
def get_inherited_managers(self, attrs):
"""
Return list of all managers to be inherited/propagated from the base classes;
use correct mro, only use managers with _inherited==False,
skip managers that are overwritten by the user with same-named class attributes (in attrs)
"""
add_managers = []; add_managers_keys = set()
for base in self.__mro__[1:]:
if not issubclass(base, models.Model): continue
if not getattr(base, 'polymorphic_model_marker', None): continue # leave managers of non-polym. models alone
for key, manager in base.__dict__.items():
if type(manager) == models.manager.ManagerDescriptor: manager = manager.manager
if not isinstance(manager, models.Manager): continue
if key in attrs: continue
if key in add_managers_keys: continue # manager with that name already added, skip
if manager._inherited: continue # inherited managers have no significance, they are just copies
if isinstance(manager, PolymorphicManager): # validate any inherited polymorphic managers
self.validate_model_manager(manager, self.__name__, key)
add_managers.append((base.__name__, key, manager))
add_managers_keys.add(key)
return add_managers
@classmethod
def get_first_user_defined_manager(self, attrs):
mgr_list = []
for key, val in attrs.items():
if not isinstance(val, models.Manager): continue
mgr_list.append((val.creation_counter, val))
# if there are user defined managers, use first one as _default_manager
if mgr_list: #
_, manager = sorted(mgr_list)[0]
return manager
return None
@classmethod
def call_superclass_new_method(self, model_name, bases, attrs):
"""call __new__ method of super class and return the newly created class.
Also work around a limitation in Django's ModelBase."""
# There seems to be a general limitation in Django's app_label handling
# regarding abstract models (in ModelBase). See issue 1 on github - TODO: propose patch for Django
# We run into this problem if polymorphic.py is located in a top-level directory
# which is directly in the python path. To work around this we temporarily set
# app_label here for PolymorphicModel.
meta = attrs.get('Meta', None)
model_module_name = attrs['__module__']
do_app_label_workaround = (meta
and model_module_name == 'polymorphic'
and model_name == 'PolymorphicModel'
and getattr(meta, 'app_label', None) is None )
if do_app_label_workaround: meta.app_label = 'poly_dummy_app_label'
new_class = super(PolymorphicModelBase, self).__new__(self, model_name, bases, attrs)
if do_app_label_workaround: del(meta.app_label)
return new_class
def validate_model_fields(self):
"check if all fields names are allowed (i.e. not in POLYMORPHIC_SPECIAL_Q_KWORDS)"
for f in self._meta.fields:
if f.name in POLYMORPHIC_SPECIAL_Q_KWORDS:
e = 'PolymorphicModel: "%s" - field name "%s" is not allowed in polymorphic models'
raise AssertionError(e % (self.__name__, f.name) )
@classmethod
def validate_model_manager(self, manager, model_name, manager_name):
"""check if the manager is derived from PolymorphicManager
and its querysets from PolymorphicQuerySet - throw AssertionError if not"""
if not issubclass(type(manager), PolymorphicManager):
e = 'PolymorphicModel: "' + model_name + '.' + manager_name + '" manager is of type "' + type(manager).__name__
e += '", but must be a subclass of PolymorphicManager'
raise AssertionError(e)
if not getattr(manager, 'queryset_class', None) or not issubclass(manager.queryset_class, PolymorphicQuerySet):
e = 'PolymorphicModel: "' + model_name + '.' + manager_name + '" (PolymorphicManager) has been instantiated with a queryset class which is'
e += ' not a subclass of PolymorphicQuerySet (which is required)'
raise AssertionError(e)
return manager

View File

@ -0,0 +1,85 @@
# -*- coding: utf-8 -*-
"""
Compatibility layer for Python 2.4
==================================
Currently implements:
+ collections.defaultdict
+ compat_partition (compatibility replacement for str.partition)
"""
try:
assert False
from collections import defaultdict
except:
class defaultdict(dict):
def __init__(self, default_factory=None, *a, **kw):
if (default_factory is not None and
not hasattr(default_factory, '__call__')):
raise TypeError('first argument must be callable')
dict.__init__(self, *a, **kw)
self.default_factory = default_factory
def __getitem__(self, key):
try:
return dict.__getitem__(self, key)
except KeyError:
return self.__missing__(key)
def __missing__(self, key):
if self.default_factory is None:
raise KeyError(key)
self[key] = value = self.default_factory()
return value
def __reduce__(self):
if self.default_factory is None:
args = tuple()
else:
args = self.default_factory,
return type(self), args, None, None, self.items()
def copy(self):
return self.__copy__()
def __copy__(self):
return type(self)(self.default_factory, self)
def __deepcopy__(self, memo):
import copy
return type(self)(self.default_factory,
copy.deepcopy(self.items()))
def __repr__(self):
return 'defaultdict(%s, %s)' % (self.default_factory, dict.__repr__(self))
if getattr(str,'partition',None):
def compat_partition(s,sep): return s.partition(sep)
else:
""" from:
http://mail.python.org/pipermail/python-dev/2005-September/055962.html
"""
def compat_partition(s, sep, at_sep=1):
""" Returns a three element tuple, (head, sep, tail) where:
head + sep + tail == s
sep == '' or sep is t
bool(sep) == (t in s) # sep indicates if the string was found
"""
if not isinstance(sep, basestring) or not sep:
raise ValueError('partititon argument must be a non-empty string')
if at_sep == 0:
result = ('', '', s)
else:
if at_sep > 0:
parts = s.split(sep, at_sep)
if len(parts) <= at_sep:
result = (s, '', '')
else:
result = (sep.join(parts[:at_sep]), sep, parts[at_sep])
else:
parts = s.rsplit(sep, at_sep)
if len(parts) <= at_sep:
result = ('', '', s)
else:
result = (parts[0], sep, sep.join(parts[1:]))
assert len(result) == 3
assert ''.join(result) == s
assert result[1] == '' or result[1] is sep
return result

View File

@ -0,0 +1,36 @@
# -*- coding: utf-8 -*-
""" PolymorphicManager
Please see README.rst or DOCS.rst or http://bserve.webhop.org/wiki/django_polymorphic
"""
from django.db import models
from query import PolymorphicQuerySet
class PolymorphicManager(models.Manager):
"""
Manager for PolymorphicModel
Usually not explicitly needed, except if a custom manager or
a custom queryset class is to be used.
"""
use_for_related_fields = True
def __init__(self, queryset_class=None, *args, **kwrags):
if not queryset_class: self.queryset_class = PolymorphicQuerySet
else: self.queryset_class = queryset_class
super(PolymorphicManager, self).__init__(*args, **kwrags)
def get_query_set(self):
return self.queryset_class(self.model)
# Proxy all unknown method calls to the queryset, so that its members are
# directly accessible as PolymorphicModel.objects.*
# The advantage of this method is that not yet known member functions of derived querysets will be proxied as well.
# We exclude any special functions (__) from this automatic proxying.
def __getattr__(self, name):
if name.startswith('__'): return super(PolymorphicManager, self).__getattr__(self, name)
return getattr(self.get_query_set(), name)
def __unicode__(self):
return self.__class__.__name__ + ' (PolymorphicManager) using ' + self.queryset_class.__name__

View File

@ -1,3 +1,10 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
"""
IMPORTANT:
from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes The models.py module is not used anymore.
Please use the following import method in your apps:
from polymorphic import PolymorphicModel, ...
"""

View File

@ -1,750 +0,0 @@
# -*- coding: utf-8 -*-
"""
Fully Polymorphic Django Models
===============================
For an overview, examples, documentation and updates please see here:
http://bserve.webhop.org/wiki/django_polymorphic
or in the included README.rst and DOCS.rst files.
Copyright:
This code and affiliated files are (C) by Bert Constantin and individual contributors.
Please see LICENSE and AUTHORS for more information.
"""
from django.db import models
from django.db.models.base import ModelBase
from django.db.models.query import QuerySet
from django.contrib.contenttypes.models import ContentType
from django import VERSION as django_VERSION
from collections import defaultdict
from pprint import pprint
import sys
# chunk-size: maximum number of objects requested per db-request
# by the polymorphic queryset.iterator() implementation
Polymorphic_QuerySet_objects_per_request = 100
###################################################################################
### PolymorphicManager
class PolymorphicManager(models.Manager):
"""
Manager for PolymorphicModel
Usually not explicitly needed, except if a custom manager or
a custom queryset class is to be used.
"""
use_for_related_fields = True
def __init__(self, queryset_class=None, *args, **kwrags):
if not queryset_class: self.queryset_class = PolymorphicQuerySet
else: self.queryset_class = queryset_class
super(PolymorphicManager, self).__init__(*args, **kwrags)
def get_query_set(self):
return self.queryset_class(self.model)
# Proxy all unknown method calls to the queryset, so that its members are
# directly accessible as PolymorphicModel.objects.*
# The advantage is that not yet known member functions of derived querysets will be proxied as well.
# We exclude any special functions (__) from this automatic proxying.
def __getattr__(self, name):
if name.startswith('__'): return super(PolymorphicManager, self).__getattr__(self, name)
return getattr(self.get_query_set(), name)
def __unicode__(self):
return self.__class__.__name__ + ' (PolymorphicManager) using ' + self.queryset_class.__name__
###################################################################################
### PolymorphicQuerySet
# PolymorphicQuerySet Q objects (and filter()) support these additional key words.
# These are forbidden as field names (a descriptive exception is raised)
POLYMORPHIC_SPECIAL_Q_KWORDS = [ 'instance_of', 'not_instance_of']
class PolymorphicQuerySet(QuerySet):
"""
QuerySet for PolymorphicModel
Contains the core functionality for PolymorphicModel
Usually not explicitly needed, except if a custom queryset class
is to be used.
"""
def __init__(self, *args, **kwargs):
"init our queryset object member variables"
self.polymorphic_disabled = False
super(PolymorphicQuerySet, self).__init__(*args, **kwargs)
def _clone(self, *args, **kwargs):
"Django's _clone only copies its own variables, so we need to copy ours here"
new = super(PolymorphicQuerySet, self)._clone(*args, **kwargs)
new.polymorphic_disabled = self.polymorphic_disabled
return new
def instance_of(self, *args):
"""Filter the queryset to only include the classes in args (and their subclasses).
Implementation in _translate_polymorphic_filter_defnition."""
return self.filter(instance_of=args)
def not_instance_of(self, *args):
"""Filter the queryset to exclude the classes in args (and their subclasses).
Implementation in _translate_polymorphic_filter_defnition."""
return self.filter(not_instance_of=args)
def _filter_or_exclude(self, negate, *args, **kwargs):
"We override this internal Django functon as it is used for all filter member functions."
_translate_polymorphic_filter_definitions_in_args(self.model, args) # the Q objects
additional_args = _translate_polymorphic_filter_definitions_in_kwargs(self.model, kwargs) # filter_field='data'
return super(PolymorphicQuerySet, self)._filter_or_exclude(negate, *(list(args) + additional_args), **kwargs)
def order_by(self, *args, **kwargs):
"""translate the field paths in the args, then call vanilla order_by."""
new_args = [ _translate_polymorphic_field_path(self.model, a) for a in args ]
return super(PolymorphicQuerySet, self).order_by(*new_args, **kwargs)
def _process_aggregate_args(self, args, kwargs):
"""for aggregate and annotate kwargs: allow ModelX___field syntax for kwargs, forbid it for args.
Modifies kwargs if needed (these are Aggregate objects, we translate the lookup member variable)"""
for a in args:
assert not '___' in a.lookup, 'PolymorphicModel: annotate()/aggregate(): ___ model lookup supported for keyword arguments only'
for a in kwargs.values():
a.lookup = _translate_polymorphic_field_path(self.model, a.lookup)
def annotate(self, *args, **kwargs):
"""translate the field paths in the kwargs, then call vanilla annotate.
_get_real_instances will do the rest of the job after executing the query."""
self._process_aggregate_args(args, kwargs)
return super(PolymorphicQuerySet, self).annotate(*args, **kwargs)
def aggregate(self, *args, **kwargs):
"""translate the field paths in the kwargs, then call vanilla aggregate.
We need no polymorphic object retrieval for aggregate => switch it off."""
self._process_aggregate_args(args, kwargs)
self.polymorphic_disabled = True
return super(PolymorphicQuerySet, self).aggregate(*args, **kwargs)
def extra(self, *args, **kwargs):
self.polymorphic_disabled = not kwargs.get('polymorphic',False)
if 'polymorphic' in kwargs: kwargs.pop('polymorphic')
return super(PolymorphicQuerySet, self).extra(*args, **kwargs)
def _get_real_instances(self, base_result_objects):
"""
Polymorphic object loader
Does the same as:
return [ o.get_real_instance() for o in base_result_objects ]
The list base_result_objects contains the objects from the executed
base class query. The class of all of them is self.model (our base model).
Some, many or all of these objects were not created and stored as
class self.model, but as a class derived from self.model. We want to re-fetch
these objects from the db as their original class so we can return them
just as they were created/saved.
We identify these objects by looking at o.polymorphic_ctype, which specifies
the real class of these objects (the class at the time they were saved).
First, we sort the result objects in base_result_objects for their
subclass (from o.polymorphic_ctype), and then we execute one db query per
subclass of objects. Here, we handle any annotations from annotate().
Finally we re-sort the resulting objects into the correct order and
return them as a list.
"""
ordered_id_list = [] # list of ids of result-objects in correct order
results = {} # polymorphic dict of result-objects, keyed with their id (no order)
# dict contains one entry per unique model type occurring in result,
# in the format idlist_per_model[modelclass]=[list-of-object-ids]
idlist_per_model = defaultdict(list)
# - sort base_result_object ids into idlist_per_model lists, depending on their real class;
# - also record the correct result order in "ordered_id_list"
# - store objects that already have the correct class into "results"
base_result_objects_by_id = {}
self_model_content_type_id = ContentType.objects.get_for_model(self.model).pk
for base_object in base_result_objects:
ordered_id_list.append(base_object.pk)
base_result_objects_by_id[base_object.pk] = base_object
# this object is not a derived object and already the real instance => store it right away
if (base_object.polymorphic_ctype_id == self_model_content_type_id):
results[base_object.pk] = base_object
# this object is derived and its real instance needs to be retrieved
# => store it's id into the bin for this model type
else:
idlist_per_model[base_object.get_real_instance_class()].append(base_object.pk)
# For each model in "idlist_per_model" request its objects (the real model)
# from the db and store them in results[].
# Then we copy the annotate fields from the base objects to the real objects.
# TODO: defer(), only(): support for these would be around here
for modelclass, idlist in idlist_per_model.items():
qs = modelclass.base_objects.filter(id__in=idlist)
qs.dup_select_related(self) # copy select related configuration to new qs
for o in qs:
if self.query.aggregates:
for anno in self.query.aggregates.keys():
attr = getattr(base_result_objects_by_id[o.pk], anno)
setattr(o, anno, attr)
results[o.pk] = o
# re-create correct order and return result list
resultlist = [ results[ordered_id] for ordered_id in ordered_id_list if ordered_id in results ]
return resultlist
def iterator(self):
"""
This function is used by Django for all object retrieval.
By overriding it, we modify the objects that this queryset returns
when it is evaluated (or its get method or other object-returning methods are called).
Here we do the same as:
base_result_objects=list(super(PolymorphicQuerySet, self).iterator())
real_results=self._get_real_instances(base_result_objects)
for o in real_results: yield o
but it requests the objects in chunks from the database,
with Polymorphic_QuerySet_objects_per_request per chunk
"""
base_iter = super(PolymorphicQuerySet, self).iterator()
# disabled => work just like a normal queryset
if self.polymorphic_disabled:
for o in base_iter: yield o
raise StopIteration
while True:
base_result_objects = []
reached_end = False
for i in range(Polymorphic_QuerySet_objects_per_request):
try: base_result_objects.append(base_iter.next())
except StopIteration:
reached_end = True
break
real_results = self._get_real_instances(base_result_objects)
for o in real_results:
yield o
if reached_end: raise StopIteration
def __repr__(self):
result = [ repr(o) for o in self.all() ]
return '[ ' + ',\n '.join(result) + ' ]'
###################################################################################
### PolymorphicQuerySet support functions
# These functions implement the additional filter- and Q-object functionality.
# They form a kind of small framework for easily adding more
# functionality to filters and Q objects.
# Probably a more general queryset enhancement class could be made out of them.
def _translate_polymorphic_filter_definitions_in_kwargs(queryset_model, kwargs):
"""
Translate the keyword argument list for PolymorphicQuerySet.filter()
Any kwargs with special polymorphic functionality are replaced in the kwargs
dict with their vanilla django equivalents.
For some kwargs a direct replacement is not possible, as a Q object is needed
instead to implement the required functionality. In these cases the kwarg is
deleted from the kwargs dict and a Q object is added to the return list.
Modifies: kwargs dict
Returns: a list of non-keyword-arguments (Q objects) to be added to the filter() query.
"""
additional_args = []
for field_path, val in kwargs.items():
new_expr = _translate_polymorphic_filter_defnition(queryset_model, field_path, val)
if type(new_expr) == tuple:
# replace kwargs element
del(kwargs[field_path])
kwargs[new_expr[0]] = new_expr[1]
elif isinstance(new_expr, models.Q):
del(kwargs[field_path])
additional_args.append(new_expr)
return additional_args
def _translate_polymorphic_filter_definitions_in_args(queryset_model, args):
"""
Translate the non-keyword argument list for PolymorphicQuerySet.filter()
In the args list, we replace all kwargs to Q-objects that contain special
polymorphic functionality with their vanilla django equivalents.
We traverse the Q object tree for this (which is simple).
TODO: investigate: we modify the Q-objects ina args in-place. Is this OK?
Modifies: args list
"""
def tree_node_correct_field_specs(node):
" process all children of this Q node "
for i in range(len(node.children)):
child = node.children[i]
if type(child) == tuple:
# this Q object child is a tuple => a kwarg like Q( instance_of=ModelB )
key, val = child
new_expr = _translate_polymorphic_filter_defnition(queryset_model, key, val)
if new_expr:
node.children[i] = new_expr
else:
# this Q object child is another Q object, recursively process this as well
tree_node_correct_field_specs(child)
for q in args:
if isinstance(q, models.Q):
tree_node_correct_field_specs(q)
def _translate_polymorphic_filter_defnition(queryset_model, field_path, field_val):
"""
Translate a keyword argument (field_path=field_val), as used for
PolymorphicQuerySet.filter()-like functions (and Q objects).
A kwarg with special polymorphic functionality is translated into
its vanilla django equivalent, which is returned, either as tuple
(field_path, field_val) or as Q object.
Returns: kwarg tuple or Q object or None (if no change is required)
"""
# handle instance_of expressions or alternatively,
# if this is a normal Django filter expression, return None
if field_path == 'instance_of':
return _create_model_filter_Q(field_val)
elif field_path == 'not_instance_of':
return _create_model_filter_Q(field_val, not_instance_of=True)
elif not '___' in field_path:
return None #no change
# filter expression contains '___' (i.e. filter for polymorphic field)
# => get the model class specified in the filter expression
newpath = _translate_polymorphic_field_path(queryset_model, field_path)
return (newpath, field_val)
def _translate_polymorphic_field_path(queryset_model, field_path):
"""
Translate a field path from a keyword argument, as used for
PolymorphicQuerySet.filter()-like functions (and Q objects).
Supports leading '-' (for order_by args).
E.g.: ModelC___field3 is translated into modela__modelb__modelc__field3
Returns: translated path (unchanged, if no translation needed)
"""
classname, sep, pure_field_path = field_path.partition('___')
if not sep: return field_path
assert classname, 'PolymorphicModel: %s: bad field specification' % field_path
negated = False
if classname[0] == '-':
negated = True
classname = classname.lstrip('-')
if '__' in classname:
# the user has app label prepended to class name via __ => use Django's get_model function
appname, sep, classname = classname.partition('__')
model = models.get_model(appname, classname)
assert model, 'PolymorphicModel: model %s (in app %s) not found!' % (model.__name__, appname)
if not issubclass(model, queryset_model):
e = 'PolymorphicModel: queryset filter error: "' + model.__name__ + '" is not derived from "' + queryset_model.__name__ + '"'
raise AssertionError(e)
else:
# the user has only given us the class name via __
# => select the model from the sub models of the queryset base model
# function to collect all sub-models, this should be optimized (cached)
def add_all_sub_models(model, result):
if issubclass(model, models.Model) and model != models.Model:
# model name is occurring twice in submodel inheritance tree => Error
if model.__name__ in result and model != result[model.__name__]:
e = 'PolymorphicModel: model name alone is ambiguous: %s.%s and %s.%s!\n'
e += 'In this case, please use the syntax: applabel__ModelName___field'
assert model, e % (
model._meta.app_label, model.__name__,
result[model.__name__]._meta.app_label, result[model.__name__].__name__)
result[model.__name__] = model
for b in model.__subclasses__():
add_all_sub_models(b, result)
submodels = {}
add_all_sub_models(queryset_model, submodels)
model = submodels.get(classname, None)
assert model, 'PolymorphicModel: model %s not found (not a subclass of %s)!' % (classname, queryset_model.__name__)
# create new field path for expressions, e.g. for baseclass=ModelA, myclass=ModelC
# 'modelb__modelc" is returned
def _create_base_path(baseclass, myclass):
bases = myclass.__bases__
for b in bases:
if b == baseclass:
return myclass.__name__.lower()
path = _create_base_path(baseclass, b)
if path: return path + '__' + myclass.__name__.lower()
return ''
basepath = _create_base_path(queryset_model, model)
newpath = ('-' if negated else '') + basepath + ('__' if basepath else '')
newpath += pure_field_path
return newpath
def _create_model_filter_Q(modellist, not_instance_of=False):
"""
Helper function for instance_of / not_instance_of
Creates and returns a Q object that filters for the models in modellist,
including all subclasses of these models (as we want to do the same
as pythons isinstance() ).
.
We recursively collect all __subclasses__(), create a Q filter for each,
and or-combine these Q objects. This could be done much more
efficiently however (regarding the resulting sql), should an optimization
be needed.
"""
if not modellist: return None
from django.db.models import Q
if type(modellist) != list and type(modellist) != tuple:
if issubclass(modellist, PolymorphicModel):
modellist = [modellist]
else:
assert False, 'PolymorphicModel: instance_of expects a list of models or a single model'
def q_class_with_subclasses(model):
q = Q(polymorphic_ctype=ContentType.objects.get_for_model(model))
for subclass in model.__subclasses__():
q = q | q_class_with_subclasses(subclass)
return q
qlist = [ q_class_with_subclasses(m) for m in modellist ]
q_ored = reduce(lambda a, b: a | b, qlist)
if not_instance_of: q_ored = ~q_ored
return q_ored
###################################################################################
### PolymorphicModel meta class
class PolymorphicModelBase(ModelBase):
"""
Manager inheritance is a pretty complex topic which may need
more thought regarding how this should be handled for polymorphic
models.
In any case, we probably should propagate 'objects' and 'base_objects'
from PolymorphicModel to every subclass. We also want to somehow
inherit/propagate _default_manager as well, as it needs to be polymorphic.
The current implementation below is an experiment to solve this
problem with a very simplistic approach: We unconditionally
inherit/propagate any and all managers (using _copy_to_model),
as long as they are defined on polymorphic models
(the others are left alone).
Like Django ModelBase, we special-case _default_manager:
if there are any user-defined managers, it is set to the first of these.
We also require that _default_manager as well as any user defined
polymorphic managers produce querysets that are derived from
PolymorphicQuerySet.
"""
def __new__(self, model_name, bases, attrs):
#print; print '###', model_name, '- bases:', bases
# create new model
new_class = self.call_superclass_new_method(model_name, bases, attrs)
# check if the model fields are all allowed
self.validate_model_fields(new_class)
# create list of all managers to be inherited from the base classes
inherited_managers = new_class.get_inherited_managers(attrs)
# add the managers to the new model
for source_name, mgr_name, manager in inherited_managers:
#print '** add inherited manager from model %s, manager %s, %s' % (source_name, mgr_name, manager.__class__.__name__)
new_manager = manager._copy_to_model(new_class)
new_class.add_to_class(mgr_name, new_manager)
# get first user defined manager; if there is one, make it the _default_manager
user_manager = self.get_first_user_defined_manager(attrs)
if user_manager:
def_mgr = user_manager._copy_to_model(new_class)
#print '## add default manager', type(def_mgr)
new_class.add_to_class('_default_manager', def_mgr)
new_class._default_manager._inherited = False # the default mgr was defined by the user, not inherited
# validate resulting default manager
self.validate_model_manager(new_class._default_manager, model_name, '_default_manager')
return new_class
def get_inherited_managers(self, attrs):
"""
Return list of all managers to be inherited/propagated from the base classes;
use correct mro, only use managers with _inherited==False,
skip managers that are overwritten by the user with same-named class attributes (in attrs)
"""
add_managers = []; add_managers_keys = set()
for base in self.__mro__[1:]:
if not issubclass(base, models.Model): continue
if not getattr(base, 'polymorphic_model_marker', None): continue # leave managers of non-polym. models alone
for key, manager in base.__dict__.items():
if type(manager) == models.manager.ManagerDescriptor: manager = manager.manager
if not isinstance(manager, models.Manager): continue
if key in attrs: continue
if key in add_managers_keys: continue # manager with that name already added, skip
if manager._inherited: continue # inherited managers have no significance, they are just copies
if isinstance(manager, PolymorphicManager): # validate any inherited polymorphic managers
self.validate_model_manager(manager, self.__name__, key)
add_managers.append((base.__name__, key, manager))
add_managers_keys.add(key)
return add_managers
@classmethod
def get_first_user_defined_manager(self, attrs):
mgr_list = []
for key, val in attrs.items():
if not isinstance(val, models.Manager): continue
mgr_list.append((val.creation_counter, val))
# if there are user defined managers, use first one as _default_manager
if mgr_list: #
_, manager = sorted(mgr_list)[0]
return manager
return None
@classmethod
def call_superclass_new_method(self, model_name, bases, attrs):
"""call __new__ method of super class and return the newly created class.
Also work around a limitation in Django's ModelBase."""
# There seems to be a general limitation in Django's app_label handling
# regarding abstract models (in ModelBase). See issue 1 on github - TODO: propose patch for Django
# We run into this problem if polymorphic.py is located in a top-level directory
# which is directly in the python path. To work around this we temporarily set
# app_label here for PolymorphicModel.
meta = attrs.get('Meta', None)
model_module_name = attrs['__module__']
do_app_label_workaround = (meta
and model_module_name == 'polymorphic'
and model_name == 'PolymorphicModel'
and getattr(meta, 'app_label', None) is None )
if do_app_label_workaround: meta.app_label = 'poly_dummy_app_label'
new_class = super(PolymorphicModelBase, self).__new__(self, model_name, bases, attrs)
if do_app_label_workaround: del(meta.app_label)
return new_class
def validate_model_fields(self):
"check if all fields names are allowed (i.e. not in POLYMORPHIC_SPECIAL_Q_KWORDS)"
for f in self._meta.fields:
if f.name in POLYMORPHIC_SPECIAL_Q_KWORDS:
e = 'PolymorphicModel: "%s" - field name "%s" is not allowed in polymorphic models'
raise AssertionError(e % (self.__name__, f.name) )
@classmethod
def validate_model_manager(self, manager, model_name, manager_name):
"""check if the manager is derived from PolymorphicManager
and its querysets from PolymorphicQuerySet - throw AssertionError if not"""
if not issubclass(type(manager), PolymorphicManager):
e = 'PolymorphicModel: "' + model_name + '.' + manager_name + '" manager is of type "' + type(manager).__name__
e += '", but must be a subclass of PolymorphicManager'
raise AssertionError(e)
if not getattr(manager, 'queryset_class', None) or not issubclass(manager.queryset_class, PolymorphicQuerySet):
e = 'PolymorphicModel: "' + model_name + '.' + manager_name + '" (PolymorphicManager) has been instantiated with a queryset class which is'
e += ' not a subclass of PolymorphicQuerySet (which is required)'
raise AssertionError(e)
return manager
###################################################################################
### PolymorphicModel
class PolymorphicModel(models.Model):
"""
Abstract base class that provides polymorphic behaviour
for any model directly or indirectly derived from it.
For usage instructions & examples please see documentation.
PolymorphicModel declares one field for internal use (polymorphic_ctype)
and provides a polymorphic manager as the default manager
(and as 'objects').
PolymorphicModel overrides the save() method.
If your derived class overrides save() as well, then you need
to take care that you correctly call the save() method of
the superclass, like:
super(YourClass,self).save(*args,**kwargs)
"""
__metaclass__ = PolymorphicModelBase
polymorphic_model_marker = True # for PolymorphicModelBase
class Meta:
abstract = True
# avoid ContentType related field accessor clash (an error emitted by model validation)
# we really should use both app_label and model name, but this is only possible since Django 1.2
if django_VERSION[0] <= 1 and django_VERSION[1] <= 1:
p_related_name_template = 'polymorphic_%(class)s_set'
else:
p_related_name_template = 'polymorphic_%(app_label)s.%(class)s_set'
polymorphic_ctype = models.ForeignKey(ContentType, null=True, editable=False,
related_name=p_related_name_template)
# some applications want to know the name of the fields that are added to its models
polymorphic_internal_model_fields = [ 'polymorphic_ctype' ]
objects = PolymorphicManager()
base_objects = models.Manager()
def pre_save_polymorphic(self):
"""Normally not needed.
This function may be called manually in special use-cases. When the object
is saved for the first time, we store its real class in polymorphic_ctype.
When the object later is retrieved by PolymorphicQuerySet, it uses this
field to figure out the real class of this object
(used by PolymorphicQuerySet._get_real_instances)
"""
if not self.polymorphic_ctype:
self.polymorphic_ctype = ContentType.objects.get_for_model(self)
def save(self, *args, **kwargs):
"""Overridden model save function which supports the polymorphism
functionality (through pre_save_polymorphic)."""
self.pre_save_polymorphic()
return super(PolymorphicModel, self).save(*args, **kwargs)
def get_real_instance_class(self):
"""Normally not needed.
If a non-polymorphic manager (like base_objects) has been used to
retrieve objects, then the real class/type of these objects may be
determined using this method."""
# the following line would be the easiest way to do this, but it produces sql queries
#return self.polymorphic_ctype.model_class()
# so we use the following version, which uses the CopntentType manager cache
return ContentType.objects.get_for_id(self.polymorphic_ctype_id).model_class()
def get_real_instance(self):
"""Normally not needed.
If a non-polymorphic manager (like base_objects) has been used to
retrieve objects, then the complete object with it's real class/type
and all fields may be retrieved with this method.
Each method call executes one db query (if necessary)."""
real_model = self.get_real_instance_class()
if real_model == self.__class__: return self
return real_model.objects.get(pk=self.pk)
# Hack:
# For base model back reference fields (like basemodel_ptr),
# Django definitely must =not= use our polymorphic manager/queryset.
# For now, we catch objects attribute access here and handle back reference fields manually.
# This problem is triggered by delete(), like here:
# django.db.models.base._collect_sub_objects: parent_obj = getattr(self, link.name)
# TODO: investigate Django how this can be avoided
def __getattribute__(self, name):
if not name.startswith('__'): # do not intercept __class__ etc.
# for efficiency: create a dict containing all model attribute names we need to intercept
# (do this only once and store the result into self.__class__.inheritance_relation_fields_dict)
if not self.__class__.__dict__.get('inheritance_relation_fields_dict', None):
def add_if_regular_sub_or_super_class(model, as_ptr, result):
if ( issubclass(model, models.Model) and model != models.Model
and model != self.__class__ and model != PolymorphicModel):
name = model.__name__.lower()
if as_ptr: name+='_ptr'
result[name] = model
def add_all_base_models(model, result):
add_if_regular_sub_or_super_class(model, True, result)
for b in model.__bases__:
add_all_base_models(b, result)
def add_sub_models(model, result):
for b in model.__subclasses__():
add_if_regular_sub_or_super_class(b, False, result)
result = {}
add_all_base_models(self.__class__,result)
add_sub_models(self.__class__,result)
#print '##',self.__class__.__name__,' - ',result
self.__class__.inheritance_relation_fields_dict = result
model = self.__class__.inheritance_relation_fields_dict.get(name, None)
if model:
id = super(PolymorphicModel, self).__getattribute__('id')
attr = model.base_objects.get(id=id)
#print '---',self.__class__.__name__,name
return attr
return super(PolymorphicModel, self).__getattribute__(name)
def __repr__(self):
out = self.__class__.__name__ + ': id %d' % (self.pk or - 1)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name + ' (' + type(f).__name__ + ')'
return '<' + out + '>'
class ShowFields(object):
""" model mixin that shows the object's class, it's fields and field contents """
def __repr__(self):
out = 'id %d, ' % (self.pk)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name
if isinstance(f, (models.ForeignKey)):
o = getattr(self, f.name)
out += ': "' + ('None' if o == None else o.__class__.__name__) + '"'
else:
out += ': "' + getattr(self, f.name) + '"'
return '<' + (self.__class__.__name__ + ': ') + out + '>'
class ShowFieldsAndTypes(object):
""" model mixin, like ShowFields, but also show field types """
def __repr__(self):
out = 'id %d' % (self.pk)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name + ' (' + type(f).__name__ + ')'
if isinstance(f, (models.ForeignKey)):
o = getattr(self, f.name)
out += ': "' + ('None' if o == None else o.__class__.__name__) + '"'
else:
out += ': "' + getattr(self, f.name) + '"'
return '<' + self.__class__.__name__ + ': ' + out + '>'

View File

@ -0,0 +1,155 @@
# -*- coding: utf-8 -*-
"""
Seamless Polymorphic Inheritance for Django Models
==================================================
Please see README.rst and DOCS.rst for further information.
Or on the Web:
http://bserve.webhop.org/wiki/django_polymorphic
http://github.com/bconstantin/django_polymorphic
http://bitbucket.org/bconstantin/django_polymorphic
Copyright:
This code and affiliated files are (C) by Bert Constantin and individual contributors.
Please see LICENSE and AUTHORS for more information.
"""
from pprint import pprint
import sys
from compatibility_tools import defaultdict
from django.db import models
from django.contrib.contenttypes.models import ContentType
from django import VERSION as django_VERSION
from base import PolymorphicModelBase
from manager import PolymorphicManager
from query import PolymorphicQuerySet
from showfields import ShowFieldTypes
###################################################################################
### PolymorphicModel
class PolymorphicModel(ShowFieldTypes, models.Model):
"""
Abstract base class that provides polymorphic behaviour
for any model directly or indirectly derived from it.
For usage instructions & examples please see documentation.
PolymorphicModel declares one field for internal use (polymorphic_ctype)
and provides a polymorphic manager as the default manager
(and as 'objects').
PolymorphicModel overrides the save() method.
If your derived class overrides save() as well, then you need
to take care that you correctly call the save() method of
the superclass, like:
super(YourClass,self).save(*args,**kwargs)
"""
__metaclass__ = PolymorphicModelBase
polymorphic_model_marker = True # for PolymorphicModelBase
class Meta:
abstract = True
# avoid ContentType related field accessor clash (an error emitted by model validation)
# we really should use both app_label and model name, but this is only possible since Django 1.2
if django_VERSION[0] <= 1 and django_VERSION[1] <= 1:
p_related_name_template = 'polymorphic_%(class)s_set'
else:
p_related_name_template = 'polymorphic_%(app_label)s.%(class)s_set'
polymorphic_ctype = models.ForeignKey(ContentType, null=True, editable=False,
related_name=p_related_name_template)
# some applications want to know the name of the fields that are added to its models
polymorphic_internal_model_fields = [ 'polymorphic_ctype' ]
objects = PolymorphicManager()
base_objects = models.Manager()
def pre_save_polymorphic(self):
"""Normally not needed.
This function may be called manually in special use-cases. When the object
is saved for the first time, we store its real class in polymorphic_ctype.
When the object later is retrieved by PolymorphicQuerySet, it uses this
field to figure out the real class of this object
(used by PolymorphicQuerySet._get_real_instances)
"""
if not self.polymorphic_ctype:
self.polymorphic_ctype = ContentType.objects.get_for_model(self)
def save(self, *args, **kwargs):
"""Overridden model save function which supports the polymorphism
functionality (through pre_save_polymorphic)."""
self.pre_save_polymorphic()
return super(PolymorphicModel, self).save(*args, **kwargs)
def get_real_instance_class(self):
"""Normally not needed.
If a non-polymorphic manager (like base_objects) has been used to
retrieve objects, then the real class/type of these objects may be
determined using this method."""
# the following line would be the easiest way to do this, but it produces sql queries
#return self.polymorphic_ctype.model_class()
# so we use the following version, which uses the CopntentType manager cache
return ContentType.objects.get_for_id(self.polymorphic_ctype_id).model_class()
def get_real_instance(self):
"""Normally not needed.
If a non-polymorphic manager (like base_objects) has been used to
retrieve objects, then the complete object with it's real class/type
and all fields may be retrieved with this method.
Each method call executes one db query (if necessary)."""
real_model = self.get_real_instance_class()
if real_model == self.__class__: return self
return real_model.objects.get(pk=self.pk)
# hack: a small patch to Django would be a better solution.
# For base model back reference fields (like basemodel_ptr),
# Django definitely must =not= use our polymorphic manager/queryset.
# For now, we catch objects attribute access here and handle back reference fields manually.
# This problem is triggered by delete(), like here:
# django.db.models.base._collect_sub_objects: parent_obj = getattr(self, link.name)
# TODO: investigate Django how this can be avoided
def __getattribute__(self, name):
if not name.startswith('__'): # do not intercept __class__ etc.
# for efficiency: create a dict containing all model attribute names we need to intercept
# (do this only once and store the result into self.__class__.inheritance_relation_fields_dict)
if not self.__class__.__dict__.get('inheritance_relation_fields_dict', None):
def add_if_regular_sub_or_super_class(model, as_ptr, result):
if ( issubclass(model, models.Model) and model != models.Model
and model != self.__class__ and model != PolymorphicModel):
name = model.__name__.lower()
if as_ptr: name+='_ptr'
result[name] = model
def add_all_base_models(model, result):
add_if_regular_sub_or_super_class(model, True, result)
for b in model.__bases__:
add_all_base_models(b, result)
def add_sub_models(model, result):
for b in model.__subclasses__():
add_if_regular_sub_or_super_class(b, False, result)
result = {}
add_all_base_models(self.__class__,result)
add_sub_models(self.__class__,result)
#print '##',self.__class__.__name__,' - ',result
self.__class__.inheritance_relation_fields_dict = result
model = self.__class__.inheritance_relation_fields_dict.get(name, None)
if model:
id = super(PolymorphicModel, self).__getattribute__('id')
attr = model.base_objects.get(id=id)
#print '---',self.__class__.__name__,name
return attr
return super(PolymorphicModel, self).__getattribute__(name)

View File

@ -0,0 +1,201 @@
# -*- coding: utf-8 -*-
""" QuerySet for PolymorphicModel
Please see README.rst or DOCS.rst or http://bserve.webhop.org/wiki/django_polymorphic
"""
from compatibility_tools import defaultdict
from django.db.models.query import QuerySet
from django.contrib.contenttypes.models import ContentType
from query_translate import translate_polymorphic_filter_definitions_in_kwargs, translate_polymorphic_filter_definitions_in_args, translate_polymorphic_field_path
# chunk-size: maximum number of objects requested per db-request
# by the polymorphic queryset.iterator() implementation; we use the same chunk size as Django
from django.db.models.query import CHUNK_SIZE # this is 100 for Django 1.1/1.2
Polymorphic_QuerySet_objects_per_request = CHUNK_SIZE
###################################################################################
### PolymorphicQuerySet
class PolymorphicQuerySet(QuerySet):
"""
QuerySet for PolymorphicModel
Contains the core functionality for PolymorphicModel
Usually not explicitly needed, except if a custom queryset class
is to be used.
"""
def __init__(self, *args, **kwargs):
"init our queryset object member variables"
self.polymorphic_disabled = False
super(PolymorphicQuerySet, self).__init__(*args, **kwargs)
def _clone(self, *args, **kwargs):
"Django's _clone only copies its own variables, so we need to copy ours here"
new = super(PolymorphicQuerySet, self)._clone(*args, **kwargs)
new.polymorphic_disabled = self.polymorphic_disabled
return new
def instance_of(self, *args):
"""Filter the queryset to only include the classes in args (and their subclasses).
Implementation in _translate_polymorphic_filter_defnition."""
return self.filter(instance_of=args)
def not_instance_of(self, *args):
"""Filter the queryset to exclude the classes in args (and their subclasses).
Implementation in _translate_polymorphic_filter_defnition."""
return self.filter(not_instance_of=args)
def _filter_or_exclude(self, negate, *args, **kwargs):
"We override this internal Django functon as it is used for all filter member functions."
translate_polymorphic_filter_definitions_in_args(self.model, args) # the Q objects
additional_args = translate_polymorphic_filter_definitions_in_kwargs(self.model, kwargs) # filter_field='data'
return super(PolymorphicQuerySet, self)._filter_or_exclude(negate, *(list(args) + additional_args), **kwargs)
def order_by(self, *args, **kwargs):
"""translate the field paths in the args, then call vanilla order_by."""
new_args = [ translate_polymorphic_field_path(self.model, a) for a in args ]
return super(PolymorphicQuerySet, self).order_by(*new_args, **kwargs)
def _process_aggregate_args(self, args, kwargs):
"""for aggregate and annotate kwargs: allow ModelX___field syntax for kwargs, forbid it for args.
Modifies kwargs if needed (these are Aggregate objects, we translate the lookup member variable)"""
for a in args:
assert not '___' in a.lookup, 'PolymorphicModel: annotate()/aggregate(): ___ model lookup supported for keyword arguments only'
for a in kwargs.values():
a.lookup = translate_polymorphic_field_path(self.model, a.lookup)
def annotate(self, *args, **kwargs):
"""translate the field paths in the kwargs, then call vanilla annotate.
_get_real_instances will do the rest of the job after executing the query."""
self._process_aggregate_args(args, kwargs)
return super(PolymorphicQuerySet, self).annotate(*args, **kwargs)
def aggregate(self, *args, **kwargs):
"""translate the field paths in the kwargs, then call vanilla aggregate.
We need no polymorphic object retrieval for aggregate => switch it off."""
self._process_aggregate_args(args, kwargs)
self.polymorphic_disabled = True
return super(PolymorphicQuerySet, self).aggregate(*args, **kwargs)
def extra(self, *args, **kwargs):
self.polymorphic_disabled = not bool(kwargs.pop('polymorphic', False))
return super(PolymorphicQuerySet, self).extra(*args, **kwargs)
def _get_real_instances(self, base_result_objects):
"""
Polymorphic object loader
Does the same as:
return [ o.get_real_instance() for o in base_result_objects ]
The list base_result_objects contains the objects from the executed
base class query. The class of all of them is self.model (our base model).
Some, many or all of these objects were not created and stored as
class self.model, but as a class derived from self.model. We want to re-fetch
these objects from the db as their original class so we can return them
just as they were created/saved.
We identify these objects by looking at o.polymorphic_ctype, which specifies
the real class of these objects (the class at the time they were saved).
First, we sort the result objects in base_result_objects for their
subclass (from o.polymorphic_ctype), and then we execute one db query per
subclass of objects. Here, we handle any annotations from annotate().
Finally we re-sort the resulting objects into the correct order and
return them as a list.
"""
ordered_id_list = [] # list of ids of result-objects in correct order
results = {} # polymorphic dict of result-objects, keyed with their id (no order)
# dict contains one entry per unique model type occurring in result,
# in the format idlist_per_model[modelclass]=[list-of-object-ids]
idlist_per_model = defaultdict(list)
# - sort base_result_object ids into idlist_per_model lists, depending on their real class;
# - also record the correct result order in "ordered_id_list"
# - store objects that already have the correct class into "results"
base_result_objects_by_id = {}
self_model_content_type_id = ContentType.objects.get_for_model(self.model).pk
for base_object in base_result_objects:
ordered_id_list.append(base_object.pk)
base_result_objects_by_id[base_object.pk] = base_object
# this object is not a derived object and already the real instance => store it right away
if (base_object.polymorphic_ctype_id == self_model_content_type_id):
results[base_object.pk] = base_object
# this object is derived and its real instance needs to be retrieved
# => store it's id into the bin for this model type
else:
idlist_per_model[base_object.get_real_instance_class()].append(base_object.pk)
# For each model in "idlist_per_model" request its objects (the real model)
# from the db and store them in results[].
# Then we copy the annotate fields from the base objects to the real objects.
# TODO: defer(), only(): support for these would be around here
for modelclass, idlist in idlist_per_model.items():
qs = modelclass.base_objects.filter(id__in=idlist)
qs.dup_select_related(self) # copy select related configuration to new qs
for o in qs:
if self.query.aggregates:
for anno in self.query.aggregates.keys():
attr = getattr(base_result_objects_by_id[o.pk], anno)
setattr(o, anno, attr)
results[o.pk] = o
# re-create correct order and return result list
resultlist = [ results[ordered_id] for ordered_id in ordered_id_list if ordered_id in results ]
return resultlist
def iterator(self):
"""
This function is used by Django for all object retrieval.
By overriding it, we modify the objects that this queryset returns
when it is evaluated (or its get method or other object-returning methods are called).
Here we do the same as:
base_result_objects=list(super(PolymorphicQuerySet, self).iterator())
real_results=self._get_real_instances(base_result_objects)
for o in real_results: yield o
but it requests the objects in chunks from the database,
with Polymorphic_QuerySet_objects_per_request per chunk
"""
base_iter = super(PolymorphicQuerySet, self).iterator()
# disabled => work just like a normal queryset
if self.polymorphic_disabled:
for o in base_iter: yield o
raise StopIteration
while True:
base_result_objects = []
reached_end = False
for i in range(Polymorphic_QuerySet_objects_per_request):
try: base_result_objects.append(base_iter.next())
except StopIteration:
reached_end = True
break
real_results = self._get_real_instances(base_result_objects)
for o in real_results:
yield o
if reached_end: raise StopIteration
def __repr__(self):
result = [ repr(o) for o in self.all() ]
return '[ ' + ',\n '.join(result) + ' ]'

View File

@ -0,0 +1,220 @@
# -*- coding: utf-8 -*-
""" PolymorphicQuerySet support functions
Please see README.rst or DOCS.rst or http://bserve.webhop.org/wiki/django_polymorphic
"""
from django.db import models
from django.contrib.contenttypes.models import ContentType
from django.db.models import Q
from compatibility_tools import compat_partition
###################################################################################
### PolymorphicQuerySet support functions
# These functions implement the additional filter- and Q-object functionality.
# They form a kind of small framework for easily adding more
# functionality to filters and Q objects.
# Probably a more general queryset enhancement class could be made out of them.
def translate_polymorphic_filter_definitions_in_kwargs(queryset_model, kwargs):
"""
Translate the keyword argument list for PolymorphicQuerySet.filter()
Any kwargs with special polymorphic functionality are replaced in the kwargs
dict with their vanilla django equivalents.
For some kwargs a direct replacement is not possible, as a Q object is needed
instead to implement the required functionality. In these cases the kwarg is
deleted from the kwargs dict and a Q object is added to the return list.
Modifies: kwargs dict
Returns: a list of non-keyword-arguments (Q objects) to be added to the filter() query.
"""
additional_args = []
for field_path, val in kwargs.items():
new_expr = _translate_polymorphic_filter_definition(queryset_model, field_path, val)
if type(new_expr) == tuple:
# replace kwargs element
del(kwargs[field_path])
kwargs[new_expr[0]] = new_expr[1]
elif isinstance(new_expr, models.Q):
del(kwargs[field_path])
additional_args.append(new_expr)
return additional_args
def translate_polymorphic_filter_definitions_in_args(queryset_model, args):
"""
Translate the non-keyword argument list for PolymorphicQuerySet.filter()
In the args list, we replace all kwargs to Q-objects that contain special
polymorphic functionality with their vanilla django equivalents.
We traverse the Q object tree for this (which is simple).
TODO: investigate: we modify the Q-objects ina args in-place. Is this OK?
Modifies: args list
"""
def tree_node_correct_field_specs(node):
" process all children of this Q node "
for i in range(len(node.children)):
child = node.children[i]
if type(child) == tuple:
# this Q object child is a tuple => a kwarg like Q( instance_of=ModelB )
key, val = child
new_expr = _translate_polymorphic_filter_definition(queryset_model, key, val)
if new_expr:
node.children[i] = new_expr
else:
# this Q object child is another Q object, recursively process this as well
tree_node_correct_field_specs(child)
for q in args:
if isinstance(q, models.Q):
tree_node_correct_field_specs(q)
def _translate_polymorphic_filter_definition(queryset_model, field_path, field_val):
"""
Translate a keyword argument (field_path=field_val), as used for
PolymorphicQuerySet.filter()-like functions (and Q objects).
A kwarg with special polymorphic functionality is translated into
its vanilla django equivalent, which is returned, either as tuple
(field_path, field_val) or as Q object.
Returns: kwarg tuple or Q object or None (if no change is required)
"""
# handle instance_of expressions or alternatively,
# if this is a normal Django filter expression, return None
if field_path == 'instance_of':
return _create_model_filter_Q(field_val)
elif field_path == 'not_instance_of':
return _create_model_filter_Q(field_val, not_instance_of=True)
elif not '___' in field_path:
return None #no change
# filter expression contains '___' (i.e. filter for polymorphic field)
# => get the model class specified in the filter expression
newpath = translate_polymorphic_field_path(queryset_model, field_path)
return (newpath, field_val)
def translate_polymorphic_field_path(queryset_model, field_path):
"""
Translate a field path from a keyword argument, as used for
PolymorphicQuerySet.filter()-like functions (and Q objects).
Supports leading '-' (for order_by args).
E.g.: if queryset_model is ModelA, then "ModelC___field3" is translated
into modela__modelb__modelc__field3.
Returns: translated path (unchanged, if no translation needed)
"""
classname, sep, pure_field_path = compat_partition(field_path, '___')
if not sep: return field_path
assert classname, 'PolymorphicModel: %s: bad field specification' % field_path
negated = False
if classname[0] == '-':
negated = True
classname = classname.lstrip('-')
if '__' in classname:
# the user has app label prepended to class name via __ => use Django's get_model function
appname, sep, classname = compat_partition(classname, '__')
model = models.get_model(appname, classname)
assert model, 'PolymorphicModel: model %s (in app %s) not found!' % (model.__name__, appname)
if not issubclass(model, queryset_model):
e = 'PolymorphicModel: queryset filter error: "' + model.__name__ + '" is not derived from "' + queryset_model.__name__ + '"'
raise AssertionError(e)
else:
# the user has only given us the class name via __
# => select the model from the sub models of the queryset base model
# function to collect all sub-models, this should be optimized (cached)
def add_all_sub_models(model, result):
if issubclass(model, models.Model) and model != models.Model:
# model name is occurring twice in submodel inheritance tree => Error
if model.__name__ in result and model != result[model.__name__]:
e = 'PolymorphicModel: model name alone is ambiguous: %s.%s and %s.%s!\n'
e += 'In this case, please use the syntax: applabel__ModelName___field'
assert model, e % (
model._meta.app_label, model.__name__,
result[model.__name__]._meta.app_label, result[model.__name__].__name__)
result[model.__name__] = model
for b in model.__subclasses__():
add_all_sub_models(b, result)
submodels = {}
add_all_sub_models(queryset_model, submodels)
model = submodels.get(classname, None)
assert model, 'PolymorphicModel: model %s not found (not a subclass of %s)!' % (classname, queryset_model.__name__)
# create new field path for expressions, e.g. for baseclass=ModelA, myclass=ModelC
# 'modelb__modelc" is returned
def _create_base_path(baseclass, myclass):
bases = myclass.__bases__
for b in bases:
if b == baseclass:
return myclass.__name__.lower()
path = _create_base_path(baseclass, b)
if path: return path + '__' + myclass.__name__.lower()
return ''
basepath = _create_base_path(queryset_model, model)
if negated: newpath = '-'
else: newpath = ''
newpath += basepath
if basepath: newpath += '__'
newpath += pure_field_path
return newpath
def _create_model_filter_Q(modellist, not_instance_of=False):
"""
Helper function for instance_of / not_instance_of
Creates and returns a Q object that filters for the models in modellist,
including all subclasses of these models (as we want to do the same
as pythons isinstance() ).
.
We recursively collect all __subclasses__(), create a Q filter for each,
and or-combine these Q objects. This could be done much more
efficiently however (regarding the resulting sql), should an optimization
be needed.
"""
if not modellist: return None
from polymorphic_model import PolymorphicModel
if type(modellist) != list and type(modellist) != tuple:
if issubclass(modellist, PolymorphicModel):
modellist = [modellist]
else:
assert False, 'PolymorphicModel: instance_of expects a list of (polymorphic) models or a single (polymorphic) model'
def q_class_with_subclasses(model):
q = Q(polymorphic_ctype=ContentType.objects.get_for_model(model))
for subclass in model.__subclasses__():
q = q | q_class_with_subclasses(subclass)
return q
qlist = [ q_class_with_subclasses(m) for m in modellist ]
q_ored = reduce(lambda a, b: a | b, qlist)
if not_instance_of: q_ored = ~q_ored
return q_ored

View File

@ -0,0 +1,50 @@
# -*- coding: utf-8 -*-
from django.db import models
def _represent_foreign_key(o):
if o is None:
out = '"None"'
else:
out = '"' + o.__class__.__name__ + '"'
return out
class ShowFieldsAndTypes(object):
""" model mixin, like ShowFields, but also show field types """
def __repr__(self):
out = 'id %d' % (self.pk)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name + ' (' + type(f).__name__ + ')'
if isinstance(f, (models.ForeignKey)):
o = getattr(self, f.name)
out += ': ' + _represent_foreign_key(o)
else:
out += ': "' + getattr(self, f.name) + '"'
return '<' + self.__class__.__name__ + ': ' + out + '>'
class ShowFields(object):
""" model mixin that shows the object's class, it's fields and field contents """
def __repr__(self):
out = 'id %d, ' % (self.pk)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name
if isinstance(f, (models.ForeignKey)):
o = getattr(self, f.name)
out += ': ' + _represent_foreign_key(o)
else:
out += ': "' + getattr(self, f.name) + '"'
return '<' + (self.__class__.__name__ + ': ') + out + '>'
class ShowFieldTypes(object):
""" INTERNAL; don't use this!
This mixin is already used by default by PolymorphicModel.
(model mixin that shows the object's class and it's field types) """
def __repr__(self):
out = self.__class__.__name__ + ': id %d' % (self.pk or - 1)
for f in self._meta.fields:
if f.name in [ 'id' ] + self.polymorphic_internal_model_fields or 'ptr' in f.name: continue
out += ', ' + f.name + ' (' + type(f).__name__ + ')'
return '<' + out + '>'

View File

@ -1,4 +1,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
""" Test Cases
Please see README.rst or DOCS.rst or http://bserve.webhop.org/wiki/django_polymorphic
"""
import settings import settings
@ -8,7 +11,7 @@ from django.db.models import Q
from django.db import models from django.db import models
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from models import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes, get_version
class PlainA(models.Model): class PlainA(models.Model):
field1 = models.CharField(max_length=10) field1 = models.CharField(max_length=10)
@ -181,12 +184,18 @@ class testclass(TestCase):
x = '\n' + repr(BlogBase.objects.order_by('-BlogA___info')) x = '\n' + repr(BlogBase.objects.order_by('-BlogA___info'))
assert x == expected assert x == expected
#assert False
__test__ = {"doctest": """ __test__ = {"doctest": """
####################################################### #######################################################
### Tests ### Tests
>>> settings.DEBUG=True >>> settings.DEBUG=True
>>> get_version()
'0.5 beta'
### simple inheritance ### simple inheritance
>>> o=Model2A.objects.create(field1='A1') >>> o=Model2A.objects.create(field1='A1')
@ -219,9 +228,18 @@ __test__ = {"doctest": """
[ <Model2B: id 2, field1 (CharField), field2 (CharField)>, [ <Model2B: id 2, field1 (CharField), field2 (CharField)>,
<Model2C: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ] <Model2C: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
>>> Model2A.objects.filter(instance_of=Model2B)
[ <Model2B: id 2, field1 (CharField), field2 (CharField)>,
<Model2C: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
>>> Model2A.objects.filter(Q(instance_of=Model2B))
[ <Model2B: id 2, field1 (CharField), field2 (CharField)>,
<Model2C: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
>>> Model2A.objects.not_instance_of(Model2B) >>> Model2A.objects.not_instance_of(Model2B)
[ <Model2A: id 1, field1 (CharField)> ] [ <Model2A: id 1, field1 (CharField)> ]
### polymorphic filtering ### polymorphic filtering
>>> Model2A.objects.filter( Q( Model2B___field2 = 'B2' ) | Q( Model2C___field3 = 'C3' ) ) >>> Model2A.objects.filter( Q( Model2B___field2 = 'B2' ) | Q( Model2C___field3 = 'C3' ) )

View File

@ -0,0 +1,14 @@
#!/bin/bash
function testit {
if which $1 ; then
if ! $1 manage.py test ; then echo ERROR ; exit 10 ; fi
else
echo "### $1 is not installed!"
fi
}
testit python2.4
testit python2.5
testit python2.6