diff --git a/DOCS.rst b/DOCS.rst index d57e4f4..58036d7 100644 --- a/DOCS.rst +++ b/DOCS.rst @@ -190,7 +190,7 @@ Non-Polymorphic Queries manage.py dumpdata ------------------ - Django's standard ``dumpdata`` requires non-polymorphic + Django's standard ``dumpdata`` command requires non-polymorphic behaviour from the querysets it uses and produces incomplete results with polymorphic models. Django_polymorphic includes a slightly modified version, named ``polymorphic_dumpdata``. @@ -224,7 +224,7 @@ the plain ``PolymorphicManager`` here. Manager Inheritance / Propagation --------------------------------- -Polymorphic models unconditionally propagate (or inherit) all managers from +Polymorphic models unconditionally propagate/inherit all managers from their base models, as long as these are polymorphic. This means that all managers inherited from polymorphic base models work just the same as if they were defined in the new model. @@ -267,21 +267,21 @@ query :: performs one SQL query to retrieve ``ModelA`` objects and one additional query for each unique derived class occurring in result_objects. The best case for retrieving 100 objects is 1 db query if all are -class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ModelB, then two -queries are executed. If result_objects contains only the base model +class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ``ModelB``, then +two queries are executed. If result_objects contains only the base model type (``ModelA``), the polymorphic models are just as efficient as plain Django models (in terms of executed queries). The pathological worst case is 101 db queries if result_objects contains 100 different object types (with all of them subclasses of ``ModelA``). Performance ist relative: when Django users create their own -polymorphic ad-hoc solution (without a module like ``polymorphic.py``), +polymorphic ad-hoc solution (without a tool like ``django_polymorphic``), this usually results in a variation of :: result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ] -which of has really bad performance. Relative to this, the -performance of the current ``polymorphic.py`` is pretty good. +which has really bad performance. Relative to this, the +performance of the current ``django_polymorphic`` is pretty good. It may well be efficient enough for the majority of use cases. Chunking: The implementation always requests objects in chunks of @@ -380,7 +380,7 @@ Currently Unsupported Queryset Methods Restrictions & Caveats ---------------------- -+ Diamond shaped inheritance: There seems to be a general problem +* Diamond shaped inheritance: There seems to be a general problem with diamond shaped multiple model inheritance with Django models (tested with V1.1). An example is here: http://code.djangoproject.com/ticket/10808. @@ -388,22 +388,25 @@ Restrictions & Caveats by subclassing it instead of modifying Django core (as we do here with PolymorphicModel). -+ A reference (``ContentType``) to the real/leaf model is stored +* A reference (``ContentType``) to the real/leaf model is stored in the base model (the base model directly inheriting from PolymorphicModel). If a model or an app is renamed, then Django's ContentType table needs to be corrected too, if the db content should stay usable after the rename. + +* The use of ContentType together with Django's seralisation or + fixtures seems to pose problems up to Django 1.1. This issue + seems to be resolved for Django 1.2 (changeset 11863: Fixed #7052, + Added support for natural keys in serialization). + http://code.djangoproject.com/ticket/7052 + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django -+ The stability of the ``ContentType`` ids when combined with Django's - serialisation / fixtures has not yet been sufficiently - investigated (please see issue 4 on github). - -+ For all objects that are not instances of the base class type, but +* For all objects that are not instances of the base class, but instances of a subclass, the base class fields are currently transferred twice from the database (an artefact of the current implementation's simplicity). -+ __getattribute__ hack: For base model inheritance back relation +* __getattribute__ hack: For base model inheritance back relation fields (like basemodel_ptr), as well as implicit model inheritance forward relation fields, Django internally tries to use our polymorphic manager/queryset in some places, which of course it diff --git a/README.rst b/README.rst index 69da5d5..a2667f7 100644 --- a/README.rst +++ b/README.rst @@ -9,11 +9,11 @@ Usage, Examples, Installation & Documentation, Links ---------------------------------------------------- * Please see the `Documentation and Examples`_ (or the short `Overview`_) -* If you have comments, questions or suggestions: `Comments & Discussion`_ +* News & comments: `News, Comments, Questions & Discussion`_ * The code can be found on GitHub_ and Bitbucket_, or downloaded as TGZ_ or ZIP_ .. _Documentation and Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc -.. _Comments & Discussion: http://django-polymorphic.blogspot.com/2010/01/messages.html +.. _News, Comments, Questions & Discussion: http://groups.google.de/group/django-polymorphic/topics .. _GitHub: http://github.com/bconstantin/django_polymorphic .. _Bitbucket: http://bitbucket.org/bconstantin/django_polymorphic .. _TGZ: http://github.com/bconstantin/django_polymorphic/tarball/master @@ -28,8 +28,8 @@ It causes objects being retrieved from the database to always be returned back with the same type/class and fields they were created and saved with. Example: -If ``ArtProject`` and ``ResearchProject`` inherit from the model ``Project``, -and we have saved one of each into the database:: +If the models ``ArtProject`` and ``ResearchProject`` inherit from the model ``Project``, +and we have saved one of each of them into the database, then we can do:: >>> Project.objects.all() . @@ -41,7 +41,8 @@ It doesn't matter how these objects are retrieved: be it through the model's own managers/querysets, ForeignKeys, ManyToManyFields or OneToOneFields. -``django_polymorphic`` does this only for models that explicitly request this behaviour. +``django_polymorphic`` does this only for models that explicitly request +this behaviour (and implicitely for their submodels). The resulting querysets are polymorphic, i.e they may deliver objects of several different types in a single query result. @@ -54,30 +55,9 @@ It's important to consider that this code is still very new and experimental. Please see the docs for current restrictions, caveats, and performance implications. -Right now it's suitable only for the more enterprising early adopters. - It does seem to work well for a number of people (including me), but it's still very early and API changes, code reorganisations or further schema changes are still a possibility. +Right now it's suitable only for the more enterprising early adopters. -News ----- - -**2010-1-29:** - - Restructured django_polymorphic into a regular Django add-on - application. This is needed for the management commands, and - also seems to be a generally good idea for future enhancements - as well (and it makes sure the tests are always included). - - The ``poly`` app - until now being used for test purposes only - - has been renamed to ``polymorphic``. See DOCS.rst - ("installation/testing") for more info. - -**2010-1-26:** - - IMPORTANT - database schema change (more info in change log). - I hope I got this change in early enough before anyone started to use - polymorphic.py in earnest. Sorry for any inconvenience. - This should be the final DB schema now. diff --git a/polymorphic/polymorphic.py b/polymorphic/polymorphic.py index 66c5816..2491067 100644 --- a/polymorphic/polymorphic.py +++ b/polymorphic/polymorphic.py @@ -79,10 +79,17 @@ class PolymorphicQuerySet(QuerySet): return self.filter(not_instance_of=args) def _filter_or_exclude(self, negate, *args, **kwargs): - _translate_polymorphic_filter_specs_in_args(self.model, args) - additional_args = _translate_polymorphic_filter_specs_in_kwargs(self.model, kwargs) + """ we override this internal Django functon since it is used for all filtering """ + _translate_polymorphic_filter_defnitions_in_args(self.model, args) # the Q objects + additional_args = _translate_polymorphic_filter_defnitions_in_kwargs(self.model, kwargs) # filter_field='data' return super(PolymorphicQuerySet, self)._filter_or_exclude(negate, *(list(args) + additional_args), **kwargs) + # these queryset functions are not yet supported + def defer(self, *args, **kwargs): raise NotImplementedError + def only(self, *args, **kwargs): raise NotImplementedError + def aggregate(self, *args, **kwargs): raise NotImplementedError + def annotate(self, *args, **kwargs): raise NotImplementedError + def _get_real_instances(self, base_result_objects): """ Polymorphic object loader @@ -95,8 +102,9 @@ class PolymorphicQuerySet(QuerySet): base class query. The class of all of them is self.model (our base model). Some, many or all of these objects were not created and stored as - class self.model, but as a class derived from self.model. We want to fetch - these objects from the db so we can return them just as they were saved. + class self.model, but as a class derived from self.model. We want to re-fetch + these objects from the db as their original class so we can return them + just as they were created/saved. We identify these objects by looking at o.polymorphic_ctype, which specifies the real class of these objects (the class at the time they were saved). @@ -145,7 +153,11 @@ class PolymorphicQuerySet(QuerySet): def iterator(self): """ - This function does the same as: + This function is used by Django for all object retrieval. + By overriding it, we modify the objects that ths queryset returns + when it is evaluated (or it's get method or other object-returning methods are called). + + Here we do the same as: base_result_objects=list(super(PolymorphicQuerySet, self).iterator()) real_results=self._get_get_real_instances(base_result_objects) @@ -173,12 +185,6 @@ class PolymorphicQuerySet(QuerySet): if reached_end: raise StopIteration - # these queryset functions are not yet supported - def defer(self, *args, **kwargs): raise NotImplementedError - def only(self, *args, **kwargs): raise NotImplementedError - def aggregate(self, *args, **kwargs): raise NotImplementedError - def annotate(self, *args, **kwargs): raise NotImplementedError - def __repr__(self): result = [ repr(o) for o in self.all() ] return '[ ' + ',\n '.join(result) + ' ]' @@ -192,7 +198,7 @@ class PolymorphicQuerySet(QuerySet): # functionality to filters and Q objects. # Probably a more general queryset enhancement class could be made out them. -def _translate_polymorphic_filter_specs_in_kwargs(queryset_model, kwargs): +def _translate_polymorphic_filter_defnitions_in_kwargs(queryset_model, kwargs): """ Translate the keyword argument list for PolymorphicQuerySet.filter() @@ -209,7 +215,7 @@ def _translate_polymorphic_filter_specs_in_kwargs(queryset_model, kwargs): additional_args = [] for field_path, val in kwargs.items(): # normal filter expression => ignore - new_expr = _translate_polymorphic_filter_spec(queryset_model, field_path, val) + new_expr = _translate_polymorphic_filter_defnition(queryset_model, field_path, val) if type(new_expr) == tuple: # replace kwargs element del(kwargs[field_path]) @@ -221,7 +227,7 @@ def _translate_polymorphic_filter_specs_in_kwargs(queryset_model, kwargs): return additional_args -def _translate_polymorphic_filter_specs_in_args(queryset_model, args): +def _translate_polymorphic_filter_defnitions_in_args(queryset_model, args): """ Translate the non-keyword argument list for PolymorphicQuerySet.filter() @@ -229,6 +235,8 @@ def _translate_polymorphic_filter_specs_in_args(queryset_model, args): polymorphic functionality with their vanilla django equivalents. We traverse the Q object tree for this (which is simple). + TODO: investigate: we modify the Q-objects ina args in-place. Is this OK? + Modifies: args list """ @@ -240,7 +248,7 @@ def _translate_polymorphic_filter_specs_in_args(queryset_model, args): if type(child) == tuple: # this Q object child is a tuple => a kwarg like Q( instance_of=ModelB ) key, val = child - new_expr = _translate_polymorphic_filter_spec(queryset_model, key, val) + new_expr = _translate_polymorphic_filter_defnition(queryset_model, key, val) if new_expr: node.children[i] = new_expr else: @@ -251,7 +259,7 @@ def _translate_polymorphic_filter_specs_in_args(queryset_model, args): if isinstance(q, models.Q): tree_node_correct_field_specs(q) -def _translate_polymorphic_filter_spec(queryset_model, field_path, field_val): +def _translate_polymorphic_filter_defnition(queryset_model, field_path, field_val): """ Translate a keyword argument (field_path=field_val), as used for PolymorphicQuerySet.filter()-like functions (and Q objects). @@ -531,7 +539,10 @@ class PolymorphicModel(models.Model): class Meta: abstract = True - polymorphic_ctype = models.ForeignKey(ContentType, null=True, editable=False) + # TODO: %(class)s alone is not really enough, we also need to include app_label - patch for Django needed? + # see: django/db/models/fields/related.py/RelatedField + polymorphic_ctype = models.ForeignKey(ContentType, + null=True, editable=False, related_name='polymorphic_%(class)s_set') # some applications want to know the name of the fields that are added to its models polymorphic_internal_model_fields = [ 'polymorphic_ctype' ] @@ -540,8 +551,7 @@ class PolymorphicModel(models.Model): base_objects = models.Manager() def pre_save_polymorphic(self): - """ - Normally not needed. + """Normally not needed. This function may be called manually in special use-cases. When the object is saved for the first time, we store its real class in polymorphic_ctype. When the object later is retrieved by PolymorphicQuerySet, it uses this diff --git a/polymorphic/tests.py b/polymorphic/tests.py index a3c2457..7bab067 100644 --- a/polymorphic/tests.py +++ b/polymorphic/tests.py @@ -6,6 +6,7 @@ from django.test import TestCase from django.db.models.query import QuerySet from django.db.models import Q from django.db import models +from django.contrib.contenttypes.models import ContentType from models import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet, ShowFields, ShowFieldsAndTypes @@ -88,6 +89,12 @@ class MgrInheritB(MgrInheritA): class MgrInheritC(ShowFieldsAndTypes, MgrInheritB): pass +# validation error: "polymorphic.relatednameclash: Accessor for field 'polymorphic_ctype' clashes +# with related field 'ContentType.relatednameclash_set'." (reported by Andrew Ingram) +# fixed with related_name +class RelatedNameClash(PolymorphicModel): + ctype = models.ForeignKey(ContentType, null=True, editable=False) + class testclass(TestCase): def test_diamond_inheritance(self):