From b1905026bca9932648da485ba10184531a28025d Mon Sep 17 00:00:00 2001 From: Bert Constantin Date: Wed, 20 Oct 2010 09:31:36 +0200 Subject: [PATCH] Documentation updated. Test_all_versions script added. Some minor misc changes. Added polybench. --- .gitignore | 2 - CHANGES.html | 438 ++++++++++ CHANGES.rst | 266 +++++++ DOCS.html | 753 ++++++++++++++++++ DOCS.rst | 503 +++++++----- README.html | 366 +++++++++ README.rst | 183 +++-- diffmanagement | 8 + pexp/management/commands/pcmd.py | 4 +- pexp/management/commands/polybench.py | 106 +++ .../commands/polymorphic_create_test_data.py | 34 + polymorphic/__init__.py | 4 +- polymorphic/query.py | 8 +- polymorphic/tests.py | 19 +- rst-to-html.py | 19 + rst.css | 197 +++++ settings.py | 27 +- test_all_python_versions | 14 - test_all_versions | 62 ++ test_dumpdata | 26 + 20 files changed, 2740 insertions(+), 299 deletions(-) create mode 100644 CHANGES.html create mode 100644 CHANGES.rst create mode 100644 DOCS.html create mode 100644 README.html create mode 100755 diffmanagement create mode 100644 pexp/management/commands/polybench.py create mode 100644 pexp/management/commands/polymorphic_create_test_data.py create mode 100755 rst-to-html.py create mode 100644 rst.css delete mode 100755 test_all_python_versions create mode 100755 test_all_versions create mode 100755 test_dumpdata diff --git a/.gitignore b/.gitignore index 680811d..eef3882 100644 --- a/.gitignore +++ b/.gitignore @@ -23,6 +23,4 @@ ppreadme.py ppdocs.py common.css screen.css -README.html -DOCS.html diff --git a/CHANGES.html b/CHANGES.html new file mode 100644 index 0000000..36f2d4d --- /dev/null +++ b/CHANGES.html @@ -0,0 +1,438 @@ + + + + + + + + + + +
+ + +
+

django_polymorphic

+
+
+

Changelog

+

.

+
+
+

2010-10-18

+
+

This is a V1.0 Beta/Testing Release

+

This release is mostly a cleanup and maintenance release that also +improves a number of minor things and fixes one (non-critical) bug.

+

Some pending API changes and corrections have been folded into this release +in order to make the upcoming V1.0 API as stable as possible.

+

This release is also about getting feedback from you in case you don't +approve of any of these changes or would like to get additional +API fixes into V1.0.

+

The release contains a considerable amount of changes in some of the more +critical parts of the software. It's intended for testing and development +environments and not for production environments. For these, it's best to +wait a few weeks for the proper V1.0 release, to allow some time for any +potential problems to turn up (if they exist).

+

If you encounter any such problems, please post them in the discussion group +or open an issue on GitHub or BitBucket (or send me an email).

+

There also have been a number of minor API changes. +Please see the README for more information.

+
+
+

New Features

+
    +

  • official Django 1.3 alpha compatibility

    +
    • +

  • PolymorphicModel.__getattribute__ hack removed. +The python __getattribute__ hack generally causes a considerable +overhead and to have this in the performance-sensitive PolymorphicModel +class was somewhat problematic. It's gone for good now.

    +
    • +

  • polymorphic_dumpdata management command functionality removed: +The regular Django dumpdata command now automatically works correctly +for polymorphic models with all Django versions.

    +
    • +

  • .get_real_instances() has been elevated to an official part of the API:

    +
    +real_objects = ModelA.objects.get_real_instances(base_objects_list_or_queryset)
+
    +

    allows you to turn a queryset or list of base objects into a list of the real instances. +This is useful if e.g. you use ModelA.base_objects.extra(...) and then want to +transform the result to its polymorphic equivalent.

    +
    • +

  • translate_polymorphic_Q_object (see DOCS)

    +
    • +

  • improved testing

    +
    • +

  • Changelog added: CHANGES.rst/html

    +
    • +
+
+
+

Bugfixes

+
    +
  • removed requirement for primary key to be an IntegerField. +Thanks to Mathieu Steele and Malthe Borch.
    • +
+
+
+

API Changes

+
+

polymorphic_dumpdata

+

The polymorphic_dumpdata management command is not needed anymore +and has been removed, as the regular Django dumpdata command now automatically +works correctly with polymorphic models (for all supported versions of Django).

+
+
+

Output of Queryset or Object Printing

+

In order to improve compatibility with vanilla Django, printing quersets does not use +django_polymorphic's pretty printing by default anymore. +To get the old behaviour when printing querysets, you need to replace your model definition:

+
+>>> class Project(PolymorphicModel):
+
+

by:

+
+>>> class Project(PolymorphicModel, ShowFieldType):
+
+

The mixin classes for pretty output have been renamed:

+
+ShowFieldTypes, ShowFields, ShowFieldsAndTypes
+

are now:

+
+ShowFieldType, ShowFieldContent and ShowFieldTypeAndContent
+

(the old ones still exist for compatibility)

+
+
+

Running the Test suite with Django 1.3

+

Django 1.3 requires python manage.py test polymorphic instead of +just python manage.py test.

+
+
+
+
+
+

2010-2-22

+

IMPORTANT: API Changed (import path changed), and Installation Note

+

The django_polymorphic source code has been restructured +and as a result needs to be installed like a normal Django App +- either via copying the "polymorphic" directory into your +Django project or by running setup.py. Adding 'polymorphic' +to INSTALLED_APPS in settings.py is still optional, however.

+

The file polymorphic.py cannot be used as a standalone +extension module anymore, as is has been split into a number +of smaller files.

+

Importing works slightly different now: All relevant symbols are +imported directly from 'polymorphic' instead from +'polymorphic.models':

+
+# new way
+from polymorphic import PolymorphicModel, ...
+
+# old way, doesn't work anymore
+from polymorphic.models import PolymorphicModel, ...
+
+
    +
  • minor API addition: 'from polymorphic import VERSION, get_version'
    • +
+
+

New Features

+

Python 2.4 compatibility, contributed by Charles Leifer. Thanks!

+
+
+

Bugfixes

+

Fix: The exception "...has no attribute 'sub_and_superclass_dict'" +could be raised. (This occurred if a subclass defined __init__ +and accessed class members before calling the superclass __init__). +Thanks to Mattias Brändström.

+

Fix: There could be name conflicts if +field_name == model_name.lower() or similar. +Now it is possible to give a field the same name as the class +(like with normal Django models). +(Found through the example provided by Mattias Brändström)

+
+
+
+
+

2010-2-4

+
+

New features (and documentation)

+

queryset order_by method added

+

queryset aggregate() and extra() methods implemented

+

queryset annotate() method implemented

+

queryset values(), values_list(), distinct() documented; defer(), +only() allowed (but not yet supported)

+

setup.py added. Thanks to Andrew Ingram.

+

More about these additions in the docs: +http://bserve.webhop.org/wiki/django_polymorphic/doc

+
+
+

Bugfixes

+
    +
  • fix remaining potential accessor name clashes (but this only works +with Django 1.2+, for 1.1 no changes). Thanks to Andrew Ingram.
    • +
  • fix use of 'id' model field, replaced with 'pk'.
    • +
  • fix select_related bug for objects from derived classes (till now +sel.-r. was just ignored)
    • +
+
+
+

"Restrictions & Caveats" updated

+
    +
  • Django 1.1 only - the names of polymorphic models must be unique +in the whole project, even if they are in two different apps. +This results from a restriction in the Django 1.1 "related_name" +option (fixed in Django 1.2).
    • +
  • Django 1.1 only - when ContentType is used in models, Django's +seralisation or fixtures cannot be used. This issue seems to be +resolved for Django 1.2 (changeset 11863: Fixed #7052, Added +support for natural keys in serialization).
    • +
+
+
+
+
+

2010-1-30

+

Fixed ContentType related field accessor clash (an error emitted +by model validation) by adding related_name to the ContentType +ForeignKey. This happened if your polymorphc model used a ContentType +ForeignKey. Thanks to Andrew Ingram.

+
+
+
+

2010-1-29

+

Restructured django_polymorphic into a regular Django add-on +application. This is needed for the management commands, and +also seems to be a generally good idea for future enhancements +as well (and it makes sure the tests are always included).

+

The poly app - until now being used for test purposes only +- has been renamed to polymorphic. See DOCS.rst +("installation/testing") for more info.

+
+
+
+

2010-1-28

+

Added the polymorphic_dumpdata management command (github issue 4), +for creating fixtures, this should be used instead of +the normal Django dumpdata command. +Thanks to Charles Leifer.

+

Important: Using ContentType together with dumpdata generally +needs Django 1.2 (important as any polymorphic model uses +ContentType).

+
+
+
+

2010-1-26

+

IMPORTANT - database schema change (more info in change log). +I hope I got this change in early enough before anyone started +to use polymorphic.py in earnest. Sorry for any inconvenience. +This should be the final DB schema now.

+

Django's ContentType is now used instead of app-label and model-name +This is a cleaner and more efficient solution +Thanks to Ilya Semenov for the suggestion.

+
+
+
+ + diff --git a/CHANGES.rst b/CHANGES.rst new file mode 100644 index 0000000..584efe3 --- /dev/null +++ b/CHANGES.rst @@ -0,0 +1,266 @@ +*django_polymorphic* +++++++++++++++++++++ +Changelog +++++++++++ + +. + +------------------------------------------------------------------ + +2010-10-18 +========== + +This is a V1.0 Beta/Testing Release +----------------------------------- + +This release is mostly a cleanup and maintenance release that also +improves a number of minor things and fixes one (non-critical) bug. + +Some pending API changes and corrections have been folded into this release +in order to make the upcoming V1.0 API as stable as possible. + +This release is also about getting feedback from you in case you don't +approve of any of these changes or would like to get additional +API fixes into V1.0. + +The release contains a considerable amount of changes in some of the more +critical parts of the software. It's intended for testing and development +environments and not for production environments. For these, it's best to +wait a few weeks for the proper V1.0 release, to allow some time for any +potential problems to turn up (if they exist). + +If you encounter any such problems, please post them in the discussion group +or open an issue on GitHub or BitBucket (or send me an email). + +There also have been a number of minor API changes. +Please see the README for more information. + +New Features +------------------------ + +* official Django 1.3 alpha compatibility + +* ``PolymorphicModel.__getattribute__`` hack removed. + The python __getattribute__ hack generally causes a considerable + overhead and to have this in the performance-sensitive PolymorphicModel + class was somewhat problematic. It's gone for good now. + +* ``polymorphic_dumpdata`` management command functionality removed: + The regular Django dumpdata command now automatically works correctly + for polymorphic models with all Django versions. + +* .get_real_instances() has been elevated to an official part of the API:: + + real_objects = ModelA.objects.get_real_instances(base_objects_list_or_queryset) + + allows you to turn a queryset or list of base objects into a list of the real instances. + This is useful if e.g. you use ``ModelA.base_objects.extra(...)`` and then want to + transform the result to its polymorphic equivalent. + +* ``translate_polymorphic_Q_object`` (see DOCS) + +* improved testing + +* Changelog added: CHANGES.rst/html + +Bugfixes +------------------------ + +* removed requirement for primary key to be an IntegerField. + Thanks to Mathieu Steele and Malthe Borch. + +API Changes +----------- + +polymorphic_dumpdata +#################### + +The polymorphic_dumpdata management command is not needed anymore +and has been removed, as the regular Django dumpdata command now automatically +works correctly with polymorphic models (for all supported versions of Django). + +Output of Queryset or Object Printing +##################################### + +In order to improve compatibility with vanilla Django, printing quersets does not use +django_polymorphic's pretty printing by default anymore. +To get the old behaviour when printing querysets, you need to replace your model definition: + +>>> class Project(PolymorphicModel): + +by: + +>>> class Project(PolymorphicModel, ShowFieldType): + +The mixin classes for pretty output have been renamed: + + ``ShowFieldTypes, ShowFields, ShowFieldsAndTypes`` + +are now: + + ``ShowFieldType, ShowFieldContent and ShowFieldTypeAndContent`` + +(the old ones still exist for compatibility) + +Running the Test suite with Django 1.3 +###################################### + +Django 1.3 requires ``python manage.py test polymorphic`` instead of +just ``python manage.py test``. + + +------------------------------------------------------------------ + +2010-2-22 +========== + +IMPORTANT: API Changed (import path changed), and Installation Note + +The django_polymorphic source code has been restructured +and as a result needs to be installed like a normal Django App +- either via copying the "polymorphic" directory into your +Django project or by running setup.py. Adding 'polymorphic' +to INSTALLED_APPS in settings.py is still optional, however. + +The file `polymorphic.py` cannot be used as a standalone +extension module anymore, as is has been split into a number +of smaller files. + +Importing works slightly different now: All relevant symbols are +imported directly from 'polymorphic' instead from +'polymorphic.models':: + + # new way + from polymorphic import PolymorphicModel, ... + + # old way, doesn't work anymore + from polymorphic.models import PolymorphicModel, ... + ++ minor API addition: 'from polymorphic import VERSION, get_version' + +New Features +------------------------ + +Python 2.4 compatibility, contributed by Charles Leifer. Thanks! + +Bugfixes +------------------------ + +Fix: The exception "...has no attribute 'sub_and_superclass_dict'" +could be raised. (This occurred if a subclass defined __init__ +and accessed class members before calling the superclass __init__). +Thanks to Mattias Brändström. + +Fix: There could be name conflicts if +field_name == model_name.lower() or similar. +Now it is possible to give a field the same name as the class +(like with normal Django models). +(Found through the example provided by Mattias Brändström) + + + +------------------------------------------------------------------ + +2010-2-4 +========== + +New features (and documentation) +----------------------------------------- + +queryset order_by method added + +queryset aggregate() and extra() methods implemented + +queryset annotate() method implemented + +queryset values(), values_list(), distinct() documented; defer(), +only() allowed (but not yet supported) + +setup.py added. Thanks to Andrew Ingram. + +More about these additions in the docs: +http://bserve.webhop.org/wiki/django_polymorphic/doc + +Bugfixes +------------------------ + +* fix remaining potential accessor name clashes (but this only works + with Django 1.2+, for 1.1 no changes). Thanks to Andrew Ingram. + +* fix use of 'id' model field, replaced with 'pk'. + +* fix select_related bug for objects from derived classes (till now + sel.-r. was just ignored) + +"Restrictions & Caveats" updated +---------------------------------------- + +* Django 1.1 only - the names of polymorphic models must be unique + in the whole project, even if they are in two different apps. + This results from a restriction in the Django 1.1 "related_name" + option (fixed in Django 1.2). + +* Django 1.1 only - when ContentType is used in models, Django's + seralisation or fixtures cannot be used. This issue seems to be + resolved for Django 1.2 (changeset 11863: Fixed #7052, Added + support for natural keys in serialization). + + + +------------------------------------------------------------------ + +2010-1-30 +========== + +Fixed ContentType related field accessor clash (an error emitted +by model validation) by adding related_name to the ContentType +ForeignKey. This happened if your polymorphc model used a ContentType +ForeignKey. Thanks to Andrew Ingram. + + + +------------------------------------------------------------------ + +2010-1-29 +========== + +Restructured django_polymorphic into a regular Django add-on +application. This is needed for the management commands, and +also seems to be a generally good idea for future enhancements +as well (and it makes sure the tests are always included). + +The ``poly`` app - until now being used for test purposes only +- has been renamed to ``polymorphic``. See DOCS.rst +("installation/testing") for more info. + + + +------------------------------------------------------------------ + +2010-1-28 +========== + +Added the polymorphic_dumpdata management command (github issue 4), +for creating fixtures, this should be used instead of +the normal Django dumpdata command. +Thanks to Charles Leifer. + +Important: Using ContentType together with dumpdata generally +needs Django 1.2 (important as any polymorphic model uses +ContentType). + + + +------------------------------------------------------------------ + +2010-1-26 +========== + +IMPORTANT - database schema change (more info in change log). +I hope I got this change in early enough before anyone started +to use polymorphic.py in earnest. Sorry for any inconvenience. +This should be the final DB schema now. + +Django's ContentType is now used instead of app-label and model-name +This is a cleaner and more efficient solution +Thanks to Ilya Semenov for the suggestion. \ No newline at end of file diff --git a/DOCS.html b/DOCS.html new file mode 100644 index 0000000..29986c5 --- /dev/null +++ b/DOCS.html @@ -0,0 +1,753 @@ + + + + + + + + + + +
+ + + +
+

Quickstart

+
+

Install

+

After uncompressing (if necessary), in the directory "django_polymorphic", +execute (on Unix-like systems):

+
+sudo python setup.py install
+
+
+
+

Make Your Models Polymorphic

+

Use PolymorphicModel instead of Django's models.Model, like so:

+
+from polymorphic import PolymorphicModel
+
+class Project(PolymorphicModel):
+        topic = models.CharField(max_length=30)
+
+class ArtProject(Project):
+        artist = models.CharField(max_length=30)
+
+class ResearchProject(Project):
+        supervisor = models.CharField(max_length=30)
+
+

All models inheriting from your polymorphic models will be polymorphic as well.

+
+
+

Create some objects

+
+>>> Project.objects.create(topic="John's Gathering")
+>>> ArtProject.objects.create(topic="Sculpting with Tim", artist="T. Turner")
+>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")
+
+
+
+

Get polymorphic query results

+
+>>> Project.objects.all()
+[ <Project:         id 1, topic: "John's Gathering">,
+  <ArtProject:      id 2, topic: "Sculpting with Tim", artist: "T. Turner">,
+  <ResearchProject: id 3, topic: "Swallow Aerodynamics", supervisor: "Dr. Winter"> ]
+
+

using instance_of and not_instance_of for narrowing the result to specific subtypes:

+
+>>> Project.objects.instance_of(ArtProject)
+[ <ArtProject:      id 2, topic: "Sculpting with Tim", artist: "T. Turner"> ]
+
+
+>>> Project.objects.instance_of(ArtProject) | Project.objects.instance_of(ResearchProject)
+[ <ArtProject:      id 2, topic: "Sculpting with Tim", artist: "T. Turner">,
+  <ResearchProject: id 3, topic: "Swallow Aerodynamics", supervisor: "Dr. Winter"> ]
+
+

Polymorphic filtering: Let's get all projects where Mr. Turner is involved as an artist +or supervisor (note the three underscores):

+
+>>> Project.objects.filter(  Q(ArtProject___artist = 'T. Turner') | Q(ResearchProject___supervisor = 'T. Turner')  )
+[ <ArtProject:      id 2, topic: "Sculpting with Tim", artist: "T. Turner">,
+  <ResearchProject: id 3, topic: "History of Sculpting", supervisor: "T. Turner"> ]
+
+
+
+

What's More?

+

Most of Django's standard ORM functionality is available and works as expected. +ForeignKeys, ManyToManyFields and OneToToneFields to your polymorphic +models work as shey should (polymorphic).

+

In short, with django_polymorphic the Django models are much more "pythonic", i.e. +they just work as you expect them to work: very similar to ordinary python classes +(which is not the case with vanilla Django model inheritance).

+

Note: In all example output, above and below, for a nicer and more informative +output the ShowFieldType mixin has been used (documented below).

+
+
+
+

More about Installation / Testing

+
+

Requirements

+

Django 1.1 (or later) and Python 2.4 / 2.5 / 2.6. This code has been tested +on Django 1.1.1 / 1.2 beta and Python 2.4.6 / 2.5.4 / 2.6.4 on Linux.

+
+
+

Included Test Suite

+

The repository (or tar file) contains a complete Django project +that may be used for tests or experiments, without any installation needed.

+

To run the included test suite, execute:

+
+./manage test polymorphic
+
+

The management command pcmd.py in the app pexp can be used for +experiments - modify this file (pexp/management/commands/pcmd.py) +to your liking, then run:

+
+./manage syncdb      # db is created in /var/tmp/... (settings.py)
+./manage pcmd
+
+
+
+

Installation

+

In the directory "django_polymorphic", execute sudo python setup.py install.

+

Alternatively you can simply copy the polymorphic directory +(under "django_polymorphic") into your Django project dir +(e.g. if you want to distribute your project with more 'batteries included').

+

If you want to use the management command polymorphic_dumpdata, then +you need to add polymorphic to your INSTALLED_APPS setting. This is also +needed if you want to run the test cases in polymorphic/tests.py.

+

In any case, Django's ContentType framework (django.contrib.contenttypes) +needs to be listed in INSTALLED_APPS (usually it already is).

+
+
+
+

More Polymorphic Functionality

+

In the examples below, these models are being used:

+
+from polymorphic import PolymorphicModel
+
+class ModelA(PolymorphicModel):
+    field1 = models.CharField(max_length=10)
+
+class ModelB(ModelA):
+    field2 = models.CharField(max_length=10)
+
+class ModelC(ModelB):
+    field3 = models.CharField(max_length=10)
+
+
+

Filtering for classes (equivalent to python's isinstance() ):

+
+>>> ModelA.objects.instance_of(ModelB)
+.
+[ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
+  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
+
+

In general, including or excluding parts of the inheritance tree:

+
+ModelA.objects.instance_of(ModelB [, ModelC ...])
+ModelA.objects.not_instance_of(ModelB [, ModelC ...])
+
+

You can also use this feature in Q-objects (with the same result as above):

+
+>>> ModelA.objects.filter( Q(instance_of=ModelB) )
+
+
+
+

Polymorphic filtering (for fields in derived classes)

+

For example, cherrypicking objects from multiple derived classes +anywhere in the inheritance tree, using Q objects (with the +syntax: exact model name + three _ + field name):

+
+>>> ModelA.objects.filter(  Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3')  )
+.
+[ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
+  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
+
+
+
+

Combining Querysets of different types/models

+

Querysets may now be regarded as object containers that allow the +aggregation of different object types - very similar to python +lists (as long as the objects are accessed through the manager of +a common base class):

+
+>>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY)
+.
+[ <ModelX: id 1, field_x (CharField)>,
+  <ModelY: id 2, field_y (CharField)> ]
+
+
+
+

ManyToManyField, ForeignKey, OneToOneField

+

Relationship fields referring to polymorphic models work as +expected: like polymorphic querysets they now always return the +referred objects with the same type/class these were created and +saved as.

+

E.g., if in your model you define:

+
+field1 = OneToOneField(ModelA)
+
+

then field1 may now also refer to objects of type ModelB or ModelC.

+

A ManyToManyField example:

+
+# The model holding the relation may be any kind of model, polymorphic or not
+class RelatingModel(models.Model):
+    many2many = models.ManyToManyField('ModelA')  # ManyToMany relation to a polymorphic model
+
+>>> o=RelatingModel.objects.create()
+>>> o.many2many.add(ModelA.objects.get(id=1))
+>>> o.many2many.add(ModelB.objects.get(id=2))
+>>> o.many2many.add(ModelC.objects.get(id=3))
+
+>>> o.many2many.all()
+[ <ModelA: id 1, field1 (CharField)>,
+  <ModelB: id 2, field1 (CharField), field2 (CharField)>,
+  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
+
+
+
+

Using Third Party Models (without modifying them)

+

Third party models can be used as polymorphic models without +restrictions by subclassing them. E.g. using a third party +model as the root of a polymorphic inheritance tree:

+
+from thirdparty import ThirdPartyModel
+
+class MyThirdPartyModel(PolymorhpicModel, ThirdPartyModel):
+    pass    # or add fields
+
+

Or instead integrating the third party model anywhere into an +existing polymorphic inheritance tree:

+
+class MyModel(SomePolymorphicModel):
+    my_field = models.CharField(max_length=10)
+
+class MyModelWithThirdParty(MyModel, ThirdPartyModel):
+    pass    # or add fields
+
+
+
+

Non-Polymorphic Queries

+
+>>> ModelA.base_objects.all()
+.
+[ <ModelA: id 1, field1 (CharField)>,
+  <ModelA: id 2, field1 (CharField)>,
+  <ModelA: id 3, field1 (CharField)> ]
+
+

Each polymorphic model has 'base_objects' defined as a normal +Django manager. Of course, arbitrary custom managers may be +added to the models as well.

+
+
+

About Queryset Methods

+
    +
  • annotate() and aggregate() work just as usual, with the +addition that the ModelX___field syntax can be used for the +keyword arguments (but not for the non-keyword arguments).
    • +
+
    +
  • order_by() now similarly supports the ModelX___field syntax +for specifying ordering through a field in a submodel.
    • +
+
    +
  • distinct() works as expected. It only regards the fields of +the base class, but this should never make a difference.
    • +
+
    +
  • select_related() works just as usual, but it can not (yet) be used +to select relations in derived models +(like ModelA.objects.select_related('ModelC___fieldxy') )
    • +
+
    +
  • extra() by default works exactly like the original version, +with the resulting queryset not being polymorphic. There is +experimental support for a polymorphic extra() via the keyword +argument polymorphic=True (only the where and +order_by and params arguments of extra() should be used then). +The behaviour of extra() may change in the future, so it's best if you use +base_objects=ModelA.base_objects.extra(...) instead if you want to +sure to get non-polymorphic behaviour.
    • +
+
    +
  • get_real_instances(base_objects_list_or_queryset) allows you to turn a +queryset or list of base model objects efficiently into the real objects. +For example, you could do base_objects=ModelA.base_objects.extra(...) and +then call real_objects=ModelA.objects.get_real_instances(base_objects).
    • +
+
    +
  • values() & values_list() currently do not return polymorphic +results. This may change in the future however. If you want to use these +methods now, it's best if you use Model.base_objects.values... as +this is guaranteed to not change.
    • +
+
    +
  • defer() and only() are not yet supported (support will be added +in the future).
    • +
+
+
+

Using enhanced Q-objects in any Places

+

Sometimes it would be nice to be able to use the enhanced filter-definitions/Q-objects +outside of polymorphic models/querysets. Example (using limit_choices_to +to filter the selection of objects in the admin):

+
+class MyModel(models.Model):
+    somekey = model.ForeignKey(Model2A,
+        limit_choices_to = Q(instance_of=Model2B) )
+
+

instance_of is a django_polymorphic-specific enhancement of Q objects, which the +vanilla django function ForeignKey cannot process. In such cases you can do:

+
+from polymorphic import translate_polymorphic_Q_object
+
+class MyModel(models.Model):
+    somekey = model.ForeignKey(Model2A,
+        limit_choices_to = translate_polymorphic_Q_object( Model2A, Q(instance_of=Model2B) ) )
+
+
+
+

Nicely Displaying Polymorphic Querysets

+

In order to get the output as seen in all examples here, you need to use the +ShowFieldType class mixin:

+
+from polymorphic import PolymorphicModel, ShowFieldType
+
+class ModelA(ShowFieldType, PolymorphicModel):
+    field1 = models.CharField(max_length=10)
+
+

You may also use ShowFieldContent or ShowFieldTypeAndContent to display +additional information when printing querysets (or converting them to text).

+
+
+
+

Custom Managers, Querysets & Manager Inheritance

+
+

Using a Custom Manager

+

A nice feature of Django is the possibility to define one's own custom object managers. +This is fully supported with django_polymorphic: For creating a custom polymorphic +manager class, just derive your manager from PolymorphicManager instead of +models.Manager. Just as with vanilla Django, in your model class, you should +explicitly add the default manager first, and then your custom manager:

+
+from polymorphic import PolymorphicModel, PolymorphicManager
+
+class MyOrderedManager(PolymorphicManager):
+    def get_query_set(self):
+        return super(MyOrderedManager,self).get_query_set().order_by('some_field')
+
+class MyModel(PolymorphicModel):
+    objects = PolymorphicManager()    # add the default polymorphic manager first
+    ordered_objects = MyOrderedManager()    # then add your own manager
+
+

The first manager defined ('objects' in the example) is used by +Django as automatic manager for several purposes, including accessing +related objects. It must not filter objects and it's safest to use +the plain PolymorphicManager here.

+
+
+

Manager Inheritance

+

Polymorphic models inherit/propagate all managers from their +base models, as long as these are polymorphic. This means that all +managers defined in polymorphic base models work just the same as if +they were defined in the new model.

+

An example (inheriting from MyModel above):

+
+class MyModel2(MyModel):
+    pass
+
+# Managers inherited from MyModel:
+# the regular 'objects' manager and the custom 'ordered_objects' manager
+>>> MyModel2.objects.all()
+>>> MyModel2.ordered_objects.all()
+
+
+
+

Using a Custom Queryset Class

+

The PolymorphicManager class accepts one initialization argument, +which is the queryset class the manager should use. Just as with vanilla Django, +you may define your own custom queryset classes. Just use PolymorphicQuerySet +instead of Django's QuerySet as the base class:

+
+from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet
+
+class MyQuerySet(PolymorphicQuerySet):
+    def my_queryset_method(...):
+        ...
+
+class MyModel(PolymorphicModel):
+    my_objects=PolymorphicManager(MyQuerySet)
+    ...
+
+
+
+
+

Performance Considerations

+

The current implementation is pretty simple and does not use any +custom SQL or Django DB layer internals - it is purely based on the +standard Django ORM. Right now the query

+
+result_objects = list( ModelA.objects.filter(...) )
+
+

performs one SQL query to retrieve ModelA objects and one additional +query for each unique derived class occurring in result_objects. +The best case for retrieving 100 objects is 1 SQL query if all are +class ModelA. If 50 objects are ModelA and 50 are ModelB, then +two queries are executed. The pathological worst case is 101 db queries if +result_objects contains 100 different object types (with all of them +subclasses of ModelA).

+

Performance ist relative: when Django users create their own +polymorphic ad-hoc solution (without a tool like django_polymorphic), +this usually results in a variation of

+
+result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ]
+
+

which has really bad performance. Relative to this, the +performance of the current django_polymorphic is pretty good. +It's probably efficient enough for the majority of use cases.

+

Chunking: The implementation always requests objects in chunks of +size Polymorphic_QuerySet_objects_per_request. This limits the +complexity/duration for each query, including the pathological cases.

+
+
+

Possible Optimizations

+

PolymorphicQuerySet can be optimized to require only one SQL query +for the queryset evaluation and retrieval of all objects.

+

Basically, what ist needed is a possibility to pull in the fields +from all relevant sub-models with one SQL query. However, some deeper +digging into the Django database layer will be required in order to +make this happen.

+

An optimized version might require an SQL database. For non-SQL databases +the implementation could fall back to the current ORM-only +implementation.

+
+

SQL Complexity of an Optimized Implementation

+

With only one SQL query, one SQL join for each possible subclass +would be needed (BaseModel.__subclasses__(), recursively). +With two SQL queries, the number of joins could be reduced to the +number of actuallly occurring subclasses in the result. A final +implementation might want to use one query only if the number of +possible subclasses (and therefore joins) is not too large, and +two queries otherwise (using the first query to determine the +actually occurring subclasses, reducing the number of joins for +the second).

+

The number of joins needed for polymorphic object retrieval might +raise concerns regarding the efficiency of these database +queries. It seems likely however, that the increased number of joins +is no problem for the supported DBM systems in all realistic use cases. +Should the number of joins of the more extreme use cases turn out to +be problematic, it is possible to split any problematic query into, for example, +two queries with only half the number of joins each.

+
+
+

In General

+

Let's not forget that the above is just about optimization. +The current implementation already works well - and perhaps well +enough for the majority of applications.

+

Also, it seems that further optimization (down to one DB request) +would be restricted to a relatively small area of the code, and +be mostly independent from the rest of the module. +So it seems this optimization can be done at any later time +(like when it's needed).

+
+
+
+

Restrictions & Caveats

+
    +
  • The queryset methods values(), values_list(), select_related(), +defer() and only() are not yet fully supported (see above)
    • +
+
    +
  • Django Admin Integration: There currently is no specific admin integration, +but it would most likely make sense to have one.
    • +
+
    +
  • Diamond shaped inheritance: There seems to be a general problem +with diamond shaped multiple model inheritance with Django models +(tested with V1.1 - V1.3). +An example is here: http://code.djangoproject.com/ticket/10808. +This problem is aggravated when trying to enhance models.Model +by subclassing it instead of modifying Django core (as we do here +with PolymorphicModel).
    • +
+
    +
  • The enhanced filter-definitions/Q-objects only work as arguments +for the methods of the polymorphic querysets. Please see above +for translate_polymorphic_Q_object.
    • +
+
    +
  • Django 1.1 only - the names of polymorphic models must be unique +in the whole project, even if they are in two different apps. +This results from a restriction in the Django 1.1 "related_name" +option (fixed in Django 1.2).
    • +
+ +
    +
  • A reference (ContentType) to the real/leaf model is stored +in the base model (the base model directly inheriting from +PolymorphicModel). You need to be aware of this when using the +dumpdata management command or any other low-level +database operations. E.g. if you rename models or apps or copy +objects from one database to another, then Django's ContentType +table needs to be corrected/copied too. This is of course generally +the case for any models using Django's ContentType.
    • +
+
+
+

Project Status

+

Django_polymorphic works well for a considerable number of users now, +and no major problems have shown up for many months. +The API can be considered stable beginning with this release.

+
+ +
+ + diff --git a/DOCS.rst b/DOCS.rst index 00853b5..e7c59b7 100644 --- a/DOCS.rst +++ b/DOCS.rst @@ -1,27 +1,106 @@ +Polymorphic Models for Django +============================= + .. contents:: Table of Contents :depth: 1 -Installation / Testing -====================== + +Quickstart +=========== + +Install +------- + +After uncompressing (if necessary), in the directory "django_polymorphic", +execute (on Unix-like systems):: + + sudo python setup.py install + +Make Your Models Polymorphic +---------------------------- + +Use ``PolymorphicModel`` instead of Django's ``models.Model``, like so:: + + from polymorphic import PolymorphicModel + + class Project(PolymorphicModel): + topic = models.CharField(max_length=30) + + class ArtProject(Project): + artist = models.CharField(max_length=30) + + class ResearchProject(Project): + supervisor = models.CharField(max_length=30) + +All models inheriting from your polymorphic models will be polymorphic as well. + +Create some objects +------------------- + +>>> Project.objects.create(topic="John's Gathering") +>>> ArtProject.objects.create(topic="Sculpting with Tim", artist="T. Turner") +>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter") + +Get polymorphic query results +----------------------------- + +>>> Project.objects.all() +[ , + , + ] + +using instance_of and not_instance_of for narrowing the result to specific subtypes: + +>>> Project.objects.instance_of(ArtProject) +[ ] + +>>> Project.objects.instance_of(ArtProject) | Project.objects.instance_of(ResearchProject) +[ , + ] + +Polymorphic filtering: Let's get all projects where Mr. Turner is involved as an artist +or supervisor (note the three underscores): + +>>> Project.objects.filter( Q(ArtProject___artist = 'T. Turner') | Q(ResearchProject___supervisor = 'T. Turner') ) +[ , + ] + +What's More? +------------- + +Most of Django's standard ORM functionality is available and works as expected. +ForeignKeys, ManyToManyFields and OneToToneFields to your polymorphic +models work as shey should (polymorphic). + +In short, with django_polymorphic the Django models are much more "pythonic", i.e. +they just work as you expect them to work: very similar to ordinary python classes +(which is not the case with vanilla Django model inheritance). + +Note: In all example output, above and below, for a nicer and more informative +output the `ShowFieldType` mixin has been used (documented below). + + +More about Installation / Testing +================================= Requirements ------------ -Django 1.1 (or later) and Python 2.5/2.6. This code has been tested -on Django 1.1.1 / 1.2 beta and Python 2.5.4 / 2.6.4 on Linux. +Django 1.1 (or later) and Python 2.4 / 2.5 / 2.6. This code has been tested +on Django 1.1.1 / 1.2 beta and Python 2.4.6 / 2.5.4 / 2.6.4 on Linux. -Testing -------- +Included Test Suite +------------------- The repository (or tar file) contains a complete Django project that may be used for tests or experiments, without any installation needed. To run the included test suite, execute:: - ./manage test + ./manage test polymorphic -The management command ``pcmd.py`` in the app ``pexp`` (Polymorphic EXPerimenting) -can be used for experiments - modify this file (pexp/management/commands/pcmd.py) +The management command ``pcmd.py`` in the app ``pexp`` can be used for +experiments - modify this file (pexp/management/commands/pcmd.py) to your liking, then run:: ./manage syncdb # db is created in /var/tmp/... (settings.py) @@ -33,7 +112,8 @@ Installation In the directory "django_polymorphic", execute ``sudo python setup.py install``. Alternatively you can simply copy the ``polymorphic`` directory -(under "django_polymorphic") into your Django project dir. +(under "django_polymorphic") into your Django project dir +(e.g. if you want to distribute your project with more 'batteries included'). If you want to use the management command ``polymorphic_dumpdata``, then you need to add ``polymorphic`` to your INSTALLED_APPS setting. This is also @@ -43,173 +123,160 @@ In any case, Django's ContentType framework (``django.contrib.contenttypes``) needs to be listed in INSTALLED_APPS (usually it already is). -Defining Polymorphic Models -=========================== +More Polymorphic Functionality +============================== -To make models polymorphic, use ``PolymorphicModel`` instead of Django's -``models.Model`` as the superclass of your base model. All models -inheriting from your base class will be polymorphic as well:: +In the examples below, these models are being used:: from polymorphic import PolymorphicModel class ModelA(PolymorphicModel): field1 = models.CharField(max_length=10) - + class ModelB(ModelA): field2 = models.CharField(max_length=10) - + class ModelC(ModelB): field3 = models.CharField(max_length=10) - -Using Polymorphic Models -======================== - -Most of Django's standard ORM functionality is available -and works as expected: - -Create some objects -------------------- - - >>> ModelA.objects.create(field1='A1') - >>> ModelB.objects.create(field1='B1', field2='B2') - >>> ModelC.objects.create(field1='C1', field2='C2', field3='C3') - -Query results are polymorphic ------------------------------ - - >>> ModelA.objects.all() - . - [ , - , - ] - Filtering for classes (equivalent to python's isinstance() ): ------------------------------------------------------------- - >>> ModelA.objects.instance_of(ModelB) - . - [ , - ] - - In general, including or excluding parts of the inheritance tree:: - - ModelA.objects.instance_of(ModelB [, ModelC ...]) - ModelA.objects.not_instance_of(ModelB [, ModelC ...]) +>>> ModelA.objects.instance_of(ModelB) +. +[ , + ] + +In general, including or excluding parts of the inheritance tree:: + + ModelA.objects.instance_of(ModelB [, ModelC ...]) + ModelA.objects.not_instance_of(ModelB [, ModelC ...]) + +You can also use this feature in Q-objects (with the same result as above): + +>>> ModelA.objects.filter( Q(instance_of=ModelB) ) Polymorphic filtering (for fields in derived classes) ----------------------------------------------------- - For example, cherrypicking objects from multiple derived classes - anywhere in the inheritance tree, using Q objects (with the - syntax: ``exact model name + three _ + field name``): - - >>> ModelA.objects.filter( Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3') ) - . - [ , - ] +For example, cherrypicking objects from multiple derived classes +anywhere in the inheritance tree, using Q objects (with the +syntax: ``exact model name + three _ + field name``): + +>>> ModelA.objects.filter( Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3') ) +. +[ , + ] Combining Querysets of different types/models --------------------------------------------- - Querysets may now be regarded as object containers that allow the - aggregation of different object types - very similar to python - lists (as long as the objects are accessed through the manager of - a common base class): +Querysets may now be regarded as object containers that allow the +aggregation of different object types - very similar to python +lists (as long as the objects are accessed through the manager of +a common base class): - >>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY) - . - [ , - ] +>>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY) +. +[ , + ] + +ManyToManyField, ForeignKey, OneToOneField +------------------------------------------ + +Relationship fields referring to polymorphic models work as +expected: like polymorphic querysets they now always return the +referred objects with the same type/class these were created and +saved as. + +E.g., if in your model you define:: + + field1 = OneToOneField(ModelA) + +then field1 may now also refer to objects of type ``ModelB`` or ``ModelC``. + +A ManyToManyField example:: + + # The model holding the relation may be any kind of model, polymorphic or not + class RelatingModel(models.Model): + many2many = models.ManyToManyField('ModelA') # ManyToMany relation to a polymorphic model + + >>> o=RelatingModel.objects.create() + >>> o.many2many.add(ModelA.objects.get(id=1)) + >>> o.many2many.add(ModelB.objects.get(id=2)) + >>> o.many2many.add(ModelC.objects.get(id=3)) + + >>> o.many2many.all() + [ , + , + ] Using Third Party Models (without modifying them) ------------------------------------------------- - Third party models can be used as polymorphic models without - restrictions by subclassing them. E.g. using a third party - model as the root of a polymorphic inheritance tree:: - - from thirdparty import ThirdPartyModel - - class MyThirdPartyModel(PolymorhpicModel, ThirdPartyModel): - pass # or add fields - - Or instead integrating the third party model anywhere into an - existing polymorphic inheritance tree:: +Third party models can be used as polymorphic models without +restrictions by subclassing them. E.g. using a third party +model as the root of a polymorphic inheritance tree:: - class MyModel(SomePolymorphicModel): - my_field = models.CharField(max_length=10) - - class MyModelWithThirdParty(MyModel, ThirdPartyModel): - pass # or add fields - -ManyToManyField, ForeignKey, OneToOneField ------------------------------------------- + from thirdparty import ThirdPartyModel - Relationship fields referring to polymorphic models work as - expected: like polymorphic querysets they now always return the - referred objects with the same type/class these were created and - saved as. - - E.g., if in your model you define:: - - field1 = OneToOneField(ModelA) - - then field1 may now also refer to objects of type ``ModelB`` or ``ModelC``. - - A ManyToManyField example:: + class MyThirdPartyModel(PolymorhpicModel, ThirdPartyModel): + pass # or add fields - # The model holding the relation may be any kind of model, polymorphic or not - class RelatingModel(models.Model): - many2many = models.ManyToManyField('ModelA') # ManyToMany relation to a polymorphic model +Or instead integrating the third party model anywhere into an +existing polymorphic inheritance tree:: - >>> o=RelatingModel.objects.create() - >>> o.many2many.add(ModelA.objects.get(id=1)) - >>> o.many2many.add(ModelB.objects.get(id=2)) - >>> o.many2many.add(ModelC.objects.get(id=3)) - - >>> o.many2many.all() - [ , - , - ] + class MyModel(SomePolymorphicModel): + my_field = models.CharField(max_length=10) + + class MyModelWithThirdParty(MyModel, ThirdPartyModel): + pass # or add fields Non-Polymorphic Queries ----------------------- - >>> ModelA.base_objects.all() - . - [ , - , - ] +>>> ModelA.base_objects.all() +. +[ , + , + ] - Each polymorphic model has 'base_objects' defined as a normal - Django manager. Of course, arbitrary custom managers may be - added to the models as well. +Each polymorphic model has 'base_objects' defined as a normal +Django manager. Of course, arbitrary custom managers may be +added to the models as well. -More Queryset Methods ---------------------- +About Queryset Methods +---------------------- -+ ``annotate()`` and ``aggregate()`` work just as usual, with the +* ``annotate()`` and ``aggregate()`` work just as usual, with the addition that the ``ModelX___field`` syntax can be used for the keyword arguments (but not for the non-keyword arguments). - + + ``order_by()`` now similarly supports the ``ModelX___field`` syntax for specifying ordering through a field in a submodel. -+ ``distinct()`` works as expected. It only regards the fields of +* ``distinct()`` works as expected. It only regards the fields of the base class, but this should never make a difference. + ``select_related()`` works just as usual, but it can not (yet) be used to select relations in derived models (like ``ModelA.objects.select_related('ModelC___fieldxy')`` ) -+ ``extra()`` by default works exactly like the original version, +* ``extra()`` by default works exactly like the original version, with the resulting queryset not being polymorphic. There is experimental support for a polymorphic extra() via the keyword argument ``polymorphic=True`` (only the ``where`` and - ``order_by`` arguments of extra() should be used then). + ``order_by`` and ``params`` arguments of extra() should be used then). + The behaviour of extra() may change in the future, so it's best if you use + ``base_objects=ModelA.base_objects.extra(...)`` instead if you want to + sure to get non-polymorphic behaviour. -+ ``values()`` & ``values_list()`` currently do not return polymorphic ++ ``get_real_instances(base_objects_list_or_queryset)`` allows you to turn a + queryset or list of base model objects efficiently into the real objects. + For example, you could do ``base_objects=ModelA.base_objects.extra(...)`` and + then call ``real_objects=ModelA.objects.get_real_instances(base_objects)``. + +* ``values()`` & ``values_list()`` currently do not return polymorphic results. This may change in the future however. If you want to use these methods now, it's best if you use ``Model.base_objects.values...`` as this is guaranteed to not change. @@ -217,32 +284,52 @@ More Queryset Methods + ``defer()`` and ``only()`` are not yet supported (support will be added in the future). -manage.py dumpdata ------------------- - - Django's standard ``dumpdata`` command requires non-polymorphic - behaviour from the querysets it uses and produces incomplete - results with polymorphic models. Django_polymorphic includes - a slightly modified version, named ``polymorphic_dumpdata`` - that fixes this. Just use this command instead of Django's - (see "installation/testing"). +Using enhanced Q-objects in any Places +-------------------------------------- - Please note that there are problems using ContentType together - with Django's seralisation or fixtures (and all polymorphic models - use ContentType). This issue seems to be resolved with Django 1.2 - (changeset 11863): http://code.djangoproject.com/ticket/7052 - +Sometimes it would be nice to be able to use the enhanced filter-definitions/Q-objects +outside of polymorphic models/querysets. Example (using ``limit_choices_to`` +to filter the selection of objects in the admin):: -Custom Managers, Querysets & Inheritance -======================================== + class MyModel(models.Model): + somekey = model.ForeignKey(Model2A, + limit_choices_to = Q(instance_of=Model2B) ) + +``instance_of`` is a django_polymorphic-specific enhancement of Q objects, which the +vanilla django function ``ForeignKey`` cannot process. In such cases you can do:: + + from polymorphic import translate_polymorphic_Q_object + + class MyModel(models.Model): + somekey = model.ForeignKey(Model2A, + limit_choices_to = translate_polymorphic_Q_object( Model2A, Q(instance_of=Model2B) ) ) + +Nicely Displaying Polymorphic Querysets +--------------------------------------- + +In order to get the output as seen in all examples here, you need to use the +ShowFieldType class mixin:: + + from polymorphic import PolymorphicModel, ShowFieldType + + class ModelA(ShowFieldType, PolymorphicModel): + field1 = models.CharField(max_length=10) + +You may also use ShowFieldContent or ShowFieldTypeAndContent to display +additional information when printing querysets (or converting them to text). + + +Custom Managers, Querysets & Manager Inheritance +================================================ Using a Custom Manager ---------------------- -For creating a custom polymorphic manager class, derive your manager -from ``PolymorphicManager`` instead of ``models.Manager``. In your model -class, explicitly add the default manager first, and then your -custom manager:: +A nice feature of Django is the possibility to define one's own custom object managers. +This is fully supported with django_polymorphic: For creating a custom polymorphic +manager class, just derive your manager from ``PolymorphicManager`` instead of +``models.Manager``. Just as with vanilla Django, in your model class, you should +explicitly add the default manager first, and then your custom manager:: from polymorphic import PolymorphicModel, PolymorphicManager @@ -281,8 +368,9 @@ Using a Custom Queryset Class ----------------------------- The ``PolymorphicManager`` class accepts one initialization argument, -which is the queryset class the manager should use. A custom -custom queryset class can be defined and used like this:: +which is the queryset class the manager should use. Just as with vanilla Django, +you may define your own custom queryset classes. Just use PolymorphicQuerySet +instead of Django's QuerySet as the base class:: from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet @@ -299,20 +387,18 @@ Performance Considerations ========================== The current implementation is pretty simple and does not use any -custom SQL - it is purely based on the Django ORM. Right now the -query :: +custom SQL or Django DB layer internals - it is purely based on the +standard Django ORM. Right now the query :: result_objects = list( ModelA.objects.filter(...) ) - + performs one SQL query to retrieve ``ModelA`` objects and one additional query for each unique derived class occurring in result_objects. -The best case for retrieving 100 objects is 1 db query if all are +The best case for retrieving 100 objects is 1 SQL query if all are class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ``ModelB``, then -two queries are executed. If result_objects contains only the base model -type (``ModelA``), the polymorphic models are just as efficient as plain -Django models (in terms of executed queries). The pathological worst -case is 101 db queries if result_objects contains 100 different -object types (with all of them subclasses of ``ModelA``). +two queries are executed. The pathological worst case is 101 db queries if +result_objects contains 100 different object types (with all of them +subclasses of ``ModelA``). Performance ist relative: when Django users create their own polymorphic ad-hoc solution (without a tool like ``django_polymorphic``), @@ -322,7 +408,7 @@ this usually results in a variation of :: which has really bad performance. Relative to this, the performance of the current ``django_polymorphic`` is pretty good. -It may well be efficient enough for the majority of use cases. +It's probably efficient enough for the majority of use cases. Chunking: The implementation always requests objects in chunks of size ``Polymorphic_QuerySet_objects_per_request``. This limits the @@ -340,16 +426,12 @@ from all relevant sub-models with one SQL query. However, some deeper digging into the Django database layer will be required in order to make this happen. -A viable option might be to get the SQL query from the QuerySet -(probably from ``django.db.models.SQL.compiler.SQLCompiler.as_sql``), -making sure that all necessary joins are done, and then doing a -custom SQL request from there (like in ``SQLCompiler.execute_sql``). +An optimized version might require an SQL database. For non-SQL databases +the implementation could fall back to the current ORM-only +implementation. -An optimized version could fall back to the current ORM-only -implementation for all non-SQL databases. - -SQL Complexity --------------- +SQL Complexity of an Optimized Implementation +--------------------------------------------- With only one SQL query, one SQL join for each possible subclass would be needed (``BaseModel.__subclasses__()``, recursively). @@ -361,21 +443,18 @@ two queries otherwise (using the first query to determine the actually occurring subclasses, reducing the number of joins for the second). -A relatively large number of joins may be needed in both cases, -which raises concerns regarding the efficiency of these database -queries. It is currently unclear however, how many model classes -will actually be involved in typical use cases - the total number -of classes in the inheritance tree as well as the number of distinct -classes in query results. It may well turn out that the increased -number of joins is no problem for the DBMS in all realistic use -cases. Alternatively, if the SQL query execution time is -significantly longer even in common use cases, this may still be -acceptable in exchange for the added functionality. +The number of joins needed for polymorphic object retrieval might +raise concerns regarding the efficiency of these database +queries. It seems likely however, that the increased number of joins +is no problem for the supported DBM systems in all realistic use cases. +Should the number of joins of the more extreme use cases turn out to +be problematic, it is possible to split any problematic query into, for example, +two queries with only half the number of joins each. In General ---------- -Let's not forget that all of the above is just about optimization. +Let's not forget that the above is just about optimization. The current implementation already works well - and perhaps well enough for the majority of applications. @@ -386,73 +465,59 @@ So it seems this optimization can be done at any later time (like when it's needed). +.. _restrictions: + Restrictions & Caveats ====================== * The queryset methods ``values()``, ``values_list()``, ``select_related()``, ``defer()`` and ``only()`` are not yet fully supported (see above) -* Django 1.1 only - the names of polymorphic models must be unique - in the whole project, even if they are in two different apps. - This results from a restriction in the Django 1.1 "related_name" - option (fixed in Django 1.2). - -* Django 1.1 only - when ContentType is used in models, Django's - seralisation or fixtures cannot be used (all polymorphic models - use ContentType). This issue seems to be resolved for Django 1.2 - (changeset 11863: Fixed #7052, Added support for natural keys in serialization). - - + http://code.djangoproject.com/ticket/7052 - + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django ++ Django Admin Integration: There currently is no specific admin integration, + but it would most likely make sense to have one. * Diamond shaped inheritance: There seems to be a general problem with diamond shaped multiple model inheritance with Django models - (tested with V1.1). + (tested with V1.1 - V1.3). An example is here: http://code.djangoproject.com/ticket/10808. This problem is aggravated when trying to enhance models.Model by subclassing it instead of modifying Django core (as we do here with PolymorphicModel). -* Django Admin Integration: There currently is no admin integration, - but it surely would be nice to have one. There is a discussion about it here: - http://groups.google.de/group/django-polymorphic/browse_thread/thread/84290fe76c40c12d ++ The enhanced filter-definitions/Q-objects only work as arguments + for the methods of the polymorphic querysets. Please see above + for ``translate_polymorphic_Q_object``. -* It must be possible to instantiate the base model objects, even if your - application never does this itself. This is needed by the current - implementation of polymorphic querysets but (likely) also by Django internals. - Example: If ModelB and ModelC inherit from ModelA, and you never create - ModelA objects, django_polymorphic and Django core will still instantiate - ModelA objects for temporary purposes (and fail, if this isn't possible). +* Django 1.1 only - the names of polymorphic models must be unique + in the whole project, even if they are in two different apps. + This results from a restriction in the Django 1.1 "related_name" + option (fixed in Django 1.2). + ++ Django 1.1 only - when ContentType is used in models, Django's + seralisation or fixtures cannot be used (all polymorphic models + use ContentType). This issue seems to be resolved for Django 1.2 + (changeset 11863: Fixed #7052, Added support for natural keys in serialization). + + + http://code.djangoproject.com/ticket/7052 + + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django * A reference (``ContentType``) to the real/leaf model is stored in the base model (the base model directly inheriting from - PolymorphicModel). If a model or an app is renamed, then Django's - ContentType table needs to be corrected too, if the db content - should stay usable after the rename. - -* For all objects that are not instances of the base class, but - instances of a subclass, the base class fields are currently - transferred twice from the database (an artefact of the current - implementation's simplicity). + PolymorphicModel). You need to be aware of this when using the + ``dumpdata`` management command or any other low-level + database operations. E.g. if you rename models or apps or copy + objects from one database to another, then Django's ContentType + table needs to be corrected/copied too. This is of course generally + the case for any models using Django's ContentType. -* __getattribute__ hack: For base model inheritance back relation - fields (like basemodel_ptr), as well as implicit model inheritance - forward relation fields, Django internally tries to use our - polymorphic manager/queryset in some places, which of course it - should not. Currently this is solved with a hacky __getattribute__ - in PolymorphicModel, which causes some overhead. A minor patch to - Django core would probably get rid of that. Project Status ============== -It's important to consider that this code is very new and -to some extent still experimental. It does seem to work very -well for a number of people, but API changes, code reorganisations -or further schema changes are still a possibility. There may also -remain larger bugs and problems in the code that have not yet -been found. +Django_polymorphic works well for a considerable number of users now, +and no major problems have shown up for many months. +The API can be considered stable beginning with this release. Links diff --git a/README.html b/README.html new file mode 100644 index 0000000..08c4d95 --- /dev/null +++ b/README.html @@ -0,0 +1,366 @@ + + + + + + + + + + +
+ + +
+

Polymorphic Models for Django

+

.

+
+

Quick Start, Docs, Contributing

+ +
+
+

What is django_polymorphic good for?

+

If you work with Django's model inheritance, django_polymorphic might +save you from implementing unpleasant workarounds that make your code +messy, error-prone, and slow. Model inheritance becomes much more "pythonic" +and now just works as you as a Python programmer expect.

+
+
+

It's best to Look at an Example

+

Let's assume the models ArtProject and ResearchProject are derived +from the model Project, and let's store one of each into the database:

+
+>>> Project.objects.create(topic="John's Gathering")
+>>> ArtProject.objects.create(topic="Sculpting with Tim", artist="T. Turner")
+>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")
+
+

If we want to retrieve all our projects, we do:

+
+>>> Project.objects.all()
+
+

Using django_polymorphic, we simply get what we stored:

+
+[ <Project:         id 1, topic: "John's Gathering">,
+  <ArtProject:      id 2, topic: "Sculpting with Tim", artist: "T. Turner">,
+  <ResearchProject: id 3, topic: "Swallow Aerodynamics", supervisor: "Dr. Winter"> ]
+
+

Using vanilla Django, we get incomplete objects, which is probably not what we wanted:

+
+[ <Project: id 1, topic: "John's Gathering">,
+  <Project: id 2, topic: "Sculpting with Tim">,
+  <Project: id 3, topic: "Swallow Aerodynamics"> ]
+
+

It's very similar for ForeignKeys, ManyToManyFields or OneToOneFields.

+

In general, the effect of django_polymorphic is twofold:

+

On one hand it makes sure that model inheritance just works +as you expect, by simply ensuring that you always get back exactly the same +objects from the database you stored there - regardless how you access them. +This can save you a lot of unpleasant workarounds.

+

On the other hand, together with only few small API additions to the Django ORM, +django_polymorphic enables a much more expressive and intuitive +programming style and also very advanced object oriented +designs that are not possible with vanilla Django.

+

Fortunately, most of the heavy duty machinery that is needed for this +functionality is already present in the original Django database layer. +Django_polymorphic merely adds a rather thin layer above that, which is +all that is required to make real OO fully automatic and very easy to use, +with only minimal additions to Django's API.

+

For more information, please look at Quickstart or the complete +Installation and Usage Docs. Please also see the restrictions and caveats.

+
+
+

This is a V1.0 Beta/Testing Release

+

This release is mostly a cleanup and maintenance release that also +improves a number of minor things and fixes one (non-critical) bug.

+

Some pending API changes and corrections have been folded into this release +in order to make the upcoming V1.0 API as stable as possible.

+

This release is also about getting feedback from you in case you don't +approve of any of these changes or would like to get additional +API fixes into V1.0.

+

The release contains a considerable amount of changes in some of the more +critical parts of the software. It's intended for testing and development +environments and not for production environments. For these, it's best to +wait a few weeks for the proper V1.0 release, to allow some time for any +potential problems to turn up (if they exist).

+

If you encounter any problems please post them in the discussion group +or open an issue on GitHub or BitBucket (or send me an email).

+
+
+
+

License

+

Django_polymorphic uses the same license as Django (BSD-like).

+
+
+

API Changes

+
+

October 18 2010

+
+

polymorphic_dumpdata

+

The polymorphic_dumpdata management command is not needed anymore +and has been removed, as the regular Django dumpdata command now automatically +works correctly with polymorphic models (for all supported versions of Django).

+
+
+

Output of Queryset or Object Printing

+

In order to improve compatibility with vanilla Django, printing quersets does not use +django_polymorphic's pretty printing by default anymore. +To get the old behaviour when printing querysets, you need to replace your model definition:

+
+>>> class Project(PolymorphicModel):
+
+

by:

+
+>>> class Project(PolymorphicModel, ShowFieldType):
+
+

The mixin classes for pretty output have been renamed:

+
+ShowFieldTypes, ShowFields, ShowFieldsAndTypes
+

are now:

+
+ShowFieldType, ShowFieldContent and ShowFieldTypeAndContent
+

(the old ones still exist for compatibility)

+
+
+

Running the Test suite with Django 1.3

+

Django 1.3 requires python manage.py test polymorphic instead of +just python manage.py test.

+
+
+
+

February 22 2010, Installation Note

+

The django_polymorphic source code has been restructured +and as a result needs to be installed like a normal Django App +- either via copying the "polymorphic" directory into your +Django project or by running setup.py. Adding 'polymorphic' +to INSTALLED_APPS in settings.py is still optional, however.

+

The file polymorphic.py cannot be used as a standalone +extension module anymore (as is has been split into a number +of smaller files).

+

Importing works slightly different now: All relevant symbols are +imported directly from 'polymorphic' instead from +'polymorphic.models':

+
+# new way
+from polymorphic import PolymorphicModel, ...
+
+# old way, doesn't work anymore
+from polymorphic.models import PolymorphicModel, ...
+
+
+
+

January 26 2010: Database Schema Change

+
+
The update from January 26 changed the database schema (more info in the commit-log).
+
Sorry for any inconvenience. But this should be the final DB schema now.
+
+
+
+
+ + diff --git a/README.rst b/README.rst index 7878a9e..997566e 100644 --- a/README.rst +++ b/README.rst @@ -1,79 +1,168 @@ -Release Notes, Usage, Code --------------------------- +Polymorphic Models for Django +============================= -* Please see `here for release notes, news and discussion`_ (Google Group) -* `Many Examples`_, or full `Installation and Usage Docs`_ (or the short `Overview`_) +. + +Quick Start, Docs, Contributing +------------------------------- + +* `What is django_polymorphic good for?`_ +* `Quickstart`_, or the complete `Installation and Usage Docs`_ +* `Release Notes, News and Discussion`_ (Google Group) or Changelog_ * Download from GitHub_ or Bitbucket_, or as TGZ_ or ZIP_ -* Improve django_polymorphic: Report issues, discuss, post patch, or fork (GitHub_, Bitbucket_, Group_, Mail_) +* Improve django_polymorphic, report issues, participate, discuss, patch or fork (GitHub_, Bitbucket_, Group_, Mail_) -.. _here for release notes, news and discussion: http://groups.google.de/group/django-polymorphic/topics +.. _What is django_polymorphic good for?: #good-for +.. _release notes, news and discussion: http://groups.google.de/group/django-polymorphic/topics .. _Group: http://groups.google.de/group/django-polymorphic/topics .. _Mail: http://github.com/bconstantin/django_polymorphic/tree/master/setup.py -.. _Installation and Usage Docs: http://bserve.webhop.org/wiki/django_polymorphic/doc -.. _Many Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc#defining-polymorphic-models +.. _Installation and Usage Docs: http://bserve.webhop.org/django_polymorphic/DOCS.html +.. _Quickstart: http://bserve.webhop.org/django_polymorphic/DOCS.html#quickstart .. _GitHub: http://github.com/bconstantin/django_polymorphic .. _Bitbucket: http://bitbucket.org/bconstantin/django_polymorphic .. _TGZ: http://github.com/bconstantin/django_polymorphic/tarball/master .. _ZIP: http://github.com/bconstantin/django_polymorphic/zipball/master -.. _Overview: http://bserve.webhop.org/wiki/django_polymorphic +.. _Overview: http://bserve.webhop.org/django_polymorphic +.. _Changelog: http://bserve.webhop.org/django_polymorphic/CHANGES.html +.. _good-for: What is django_polymorphic good for? ------------------------------------ -**Example**: If we define the model ``Project`` as the base class for -our models ``ArtProject`` and ``ResearchProject``, and we store one of -each into the database, then we can do:: +If you work with Django's model inheritance, django_polymorphic might +save you from implementing unpleasant workarounds that make your code +messy, error-prone, and slow. Model inheritance becomes much more "pythonic" +and now just works as you as a Python programmer expect. - >>> Project.objects.all() - . - [ , - , - ] +It's best to Look at an Example +------------------------------- -In general: django_polymorphic implements seamless polymorphic inheritance for Django models. +Let's assume the models ``ArtProject`` and ``ResearchProject`` are derived +from the model ``Project``, and let's store one of each into the database: -The effect: objects are always returned back from the database just -as you created them, with the same type/class and fields. +>>> Project.objects.create(topic="John's Gathering") +>>> ArtProject.objects.create(topic="Sculpting with Tim", artist="T. Turner") +>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter") -It doesn't matter how these objects are retrieved: be it through the -model's own managers/querysets, ForeignKeys, ManyToManyFields -or OneToOneFields. +If we want to retrieve all our projects, we do: -As seen in this example, the resulting querysets are polymorphic, -and will typically deliver objects of several different types in -a single query result. +>>> Project.objects.all() -django_polymorphic does this only for models that explicitely enable it -(and for their submodels). +Using django_polymorphic, we simply get what we stored:: -Please see the `Documentation and Examples`_ for more information -or directly look at `more Examples`_. + [ , + , + ] -.. _Documentation and Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc -.. _more Examples: http://bserve.webhop.org/wiki/django_polymorphic/doc#defining-polymorphic-models +Using vanilla Django, we get incomplete objects, which is probably not what we wanted:: -Status ------- + [ , + , + ] -It's important to consider that this code is very new and -to some extent still experimental. Please see the docs for -current restrictions, caveats, and performance implications. +It's very similar for ForeignKeys, ManyToManyFields or OneToOneFields. -It does seem to work very well for a number of people, but -API changes, code reorganisations or further schema changes -are still a possibility. There may also remain larger bugs -and problems in the code that have not yet been found. +In general, the effect of django_polymorphic is twofold: + +On one hand it makes sure that model inheritance just works +as you expect, by simply ensuring that you always get back exactly the same +objects from the database you stored there - regardless how you access them. +This can save you a lot of unpleasant workarounds. + +On the other hand, together with only few small API additions to the Django ORM, +django_polymorphic enables a much more expressive and intuitive +programming style and also very advanced object oriented +designs that are not possible with vanilla Django. + +Fortunately, most of the heavy duty machinery that is needed for this +functionality is already present in the original Django database layer. +Django_polymorphic merely adds a rather thin layer above that, which is +all that is required to make real OO fully automatic and very easy to use, +with only minimal additions to Django's API. + +For more information, please look at `Quickstart`_ or the complete +`Installation and Usage Docs`_. Please also see the `restrictions and caveats`_. + +.. _restrictions and caveats: http://bserve.webhop.org/django_polymorphic/DOCS.html#restrictions + + +This is a V1.0 Beta/Testing Release +----------------------------------- + +This release is mostly a cleanup and maintenance release that also +improves a number of minor things and fixes one (non-critical) bug. + +Some pending API changes and corrections have been folded into this release +in order to make the upcoming V1.0 API as stable as possible. + +This release is also about getting feedback from you in case you don't +approve of any of these changes or would like to get additional +API fixes into V1.0. + +The release contains a considerable amount of changes in some of the more +critical parts of the software. It's intended for testing and development +environments and not for production environments. For these, it's best to +wait a few weeks for the proper V1.0 release, to allow some time for any +potential problems to turn up (if they exist). + +If you encounter any problems please post them in the `discussion group`_ +or open an issue on GitHub_ or BitBucket_ (or send me an email). + +.. _discussion group: http://groups.google.de/group/django-polymorphic/topics License -------- +======= -django_polymorphic uses the same license as Django (BSD-like). +Django_polymorphic uses the same license as Django (BSD-like). -API Change on February 22, plus Installation Note -------------------------------------------------- +API Changes +=========== + +October 18 2010 +------------------------------------------------------------------- + +polymorphic_dumpdata +++++++++++++++++++++ + +The polymorphic_dumpdata management command is not needed anymore +and has been removed, as the regular Django dumpdata command now automatically +works correctly with polymorphic models (for all supported versions of Django). + +Output of Queryset or Object Printing ++++++++++++++++++++++++++++++++++++++ + +In order to improve compatibility with vanilla Django, printing quersets does not use +django_polymorphic's pretty printing by default anymore. +To get the old behaviour when printing querysets, you need to replace your model definition: + +>>> class Project(PolymorphicModel): + +by: + +>>> class Project(PolymorphicModel, ShowFieldType): + +The mixin classes for pretty output have been renamed: + + ``ShowFieldTypes, ShowFields, ShowFieldsAndTypes`` + +are now: + + ``ShowFieldType, ShowFieldContent and ShowFieldTypeAndContent`` + +(the old ones still exist for compatibility) + +Running the Test suite with Django 1.3 +++++++++++++++++++++++++++++++++++++++ + +Django 1.3 requires ``python manage.py test polymorphic`` instead of +just ``python manage.py test``. + + +February 22 2010, Installation Note +----------------------------------- The django_polymorphic source code has been restructured and as a result needs to be installed like a normal Django App @@ -96,8 +185,8 @@ imported directly from 'polymorphic' instead from from polymorphic.models import PolymorphicModel, ... -Database Schema Change on January 26 ------------------------------------- +January 26 2010: Database Schema Change +----------------------------------------- | The update from January 26 changed the database schema (more info in the commit-log_). | Sorry for any inconvenience. But this should be the final DB schema now. diff --git a/diffmanagement b/diffmanagement new file mode 100755 index 0000000..d27424d --- /dev/null +++ b/diffmanagement @@ -0,0 +1,8 @@ +#!/bin/bash + +colordiff -u -w libraries-local/django-versions/django1.1/core/management/commands/dumpdata.py polymorphic/management/commands/polymorphic_dumpdata_11.py + +colordiff -u -w libraries-local/django-versions/django1.2/core/management/commands/dumpdata.py polymorphic/management/commands/polymorphic_dumpdata_12.py + +colordiff -u -w libraries-local/django-versions/django1.3/core/management/commands/dumpdata.py polymorphic/management/commands/polymorphic_dumpdata_13.py + diff --git a/pexp/management/commands/pcmd.py b/pexp/management/commands/pcmd.py index 90c7c29..58bdfa8 100644 --- a/pexp/management/commands/pcmd.py +++ b/pexp/management/commands/pcmd.py @@ -15,7 +15,7 @@ def reset_queries(): def show_queries(): print; print 'QUERIES:',len(connection.queries); pprint(connection.queries); print; connection.queries=[] - + class Command(NoArgsCommand): help = "" @@ -38,6 +38,4 @@ class Command(NoArgsCommand): o=ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter") print Project.objects.all() print - - diff --git a/pexp/management/commands/polybench.py b/pexp/management/commands/polybench.py new file mode 100644 index 0000000..918dea0 --- /dev/null +++ b/pexp/management/commands/polybench.py @@ -0,0 +1,106 @@ +# -*- coding: utf-8 -*- +""" +This module is a scratchpad for general development, testing & debugging +""" + +from django.core.management.base import NoArgsCommand +from django.db.models import connection +from pprint import pprint +import settings +import sys +from pexp.models import * + +num_objects=15000 + +def reset_queries(): + connection.queries=[] + +def show_queries(): + print; print 'QUERIES:',len(connection.queries); pprint(connection.queries); print; reset_queries() + +import time + +################################################################################### +### benchmark wrappers + +def print_timing(func, message='', iterations=1): + def wrapper(*arg): + results=[] + reset_queries() + for i in xrange(iterations): + t1 = time.time() + x = func(*arg) + t2 = time.time() + results.append((t2-t1)*1000.0) + res_sum=0 + for r in results: res_sum +=r + median = res_sum / len(results) + print '%s%-19s: %.0f ms, %i queries' % ( + message,func.func_name, + median, + len(connection.queries)/len(results) + ) + sys.stdout.flush() + return wrapper + +def run_vanilla_any_poly(func, iterations=1): + f=print_timing(func,' ', iterations) + f(nModelC) + f=print_timing(func,'poly ', iterations) + f(ModelC) + + +################################################################################### +### benchmarks + +def bench_create(model): + for i in xrange(num_objects): + model.objects.create(field1='abc'+str(i), field2='abcd'+str(i), field3='abcde'+str(i)) + #print 'count:',model.objects.count() + +def bench_load1(model): + for o in model.objects.all(): + pass + +def bench_load1_short(model): + for i in xrange(num_objects/100): + for o in model.objects.all()[:100]: + pass + +def bench_load2(model): + for o in model.objects.all(): + f1=o.field1 + f2=o.field2 + f3=o.field3 + +def bench_load2_short(model): + for i in xrange(num_objects/100): + for o in model.objects.all()[:100]: + f1=o.field1 + f2=o.field2 + f3=o.field3 + +def bench_delete(model): + model.objects.all().delete() + +################################################################################### +### Command + +class Command(NoArgsCommand): + help = "" + + def handle_noargs(self, **options): + print 'polybench - sqlite test db is stored in:',settings.SQLITE_DB_PATH + + func_list = [ + ( bench_delete, 1 ), + ( bench_create, 1 ), + ( bench_load1, 5 ), + ( bench_load1_short, 5 ), + ( bench_load2, 5 ), + ( bench_load2_short, 5 ) + ] + for f,iterations in func_list: + run_vanilla_any_poly(f,iterations=iterations) + + print diff --git a/pexp/management/commands/polymorphic_create_test_data.py b/pexp/management/commands/polymorphic_create_test_data.py new file mode 100644 index 0000000..ff2fbd0 --- /dev/null +++ b/pexp/management/commands/polymorphic_create_test_data.py @@ -0,0 +1,34 @@ +# -*- coding: utf-8 -*- +""" +This module is a scratchpad for general development, testing & debugging +""" + +from django.core.management.base import NoArgsCommand +from django.db.models import connection +from pprint import pprint +import settings + +from pexp.models import * + +def reset_queries(): + connection.queries=[] + +def show_queries(): + print; print 'QUERIES:',len(connection.queries); pprint(connection.queries); print; connection.queries=[] + +class Command(NoArgsCommand): + help = "" + + def handle_noargs(self, **options): + #print 'polycmd - sqlite test db is stored in:',settings.SQLITE_DB_PATH + print + + Project.objects.all().delete() + o=Project.objects.create(topic="John's gathering") + o=ArtProject.objects.create(topic="Sculpting with Tim", artist="T. Turner") + o=ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter") + print Project.objects.all() + print + + + diff --git a/polymorphic/__init__.py b/polymorphic/__init__.py index 9491220..9491b23 100644 --- a/polymorphic/__init__.py +++ b/polymorphic/__init__.py @@ -12,10 +12,10 @@ from manager import PolymorphicManager from query import PolymorphicQuerySet from query_translate import translate_polymorphic_Q_object from showfields import ShowFieldContent, ShowFieldType, ShowFieldTypeAndContent -#from showfields import ShowFieldTypes, ShowFields, ShowFieldsAndTypes # import old names for compatibility +from showfields import ShowFields, ShowFieldTypes, ShowFieldsAndTypes # import old names for compatibility -VERSION = (0, 5, 0, 'beta') +VERSION = (1, 0 , 0, 'beta') def get_version(): version = '%s.%s' % VERSION[0:2] diff --git a/polymorphic/query.py b/polymorphic/query.py index 93cf270..97b00d4 100644 --- a/polymorphic/query.py +++ b/polymorphic/query.py @@ -72,19 +72,23 @@ class PolymorphicQuerySet(QuerySet): a.lookup = translate_polymorphic_field_path(self.model, a.lookup) def annotate(self, *args, **kwargs): - """translate the field paths in the kwargs, then call vanilla annotate. + """translate the polymorphic field paths in the kwargs, then call vanilla annotate. _get_real_instances will do the rest of the job after executing the query.""" self._process_aggregate_args(args, kwargs) return super(PolymorphicQuerySet, self).annotate(*args, **kwargs) def aggregate(self, *args, **kwargs): - """translate the field paths in the kwargs, then call vanilla aggregate. + """translate the polymorphic field paths in the kwargs, then call vanilla aggregate. We need no polymorphic object retrieval for aggregate => switch it off.""" self._process_aggregate_args(args, kwargs) self.polymorphic_disabled = True return super(PolymorphicQuerySet, self).aggregate(*args, **kwargs) def extra(self, *args, **kwargs): + """only return polymorphic results if we get "polymorphic=True" as a keyword arg.""" + #for key in kwargs.keys(): + # if key not in ['where','order_by', 'params']: + # assert False,"""django_polymorphic: extras() does not support keyword argument %s. self.polymorphic_disabled = not bool(kwargs.pop('polymorphic', False)) return super(PolymorphicQuerySet, self).extra(*args, **kwargs) diff --git a/polymorphic/tests.py b/polymorphic/tests.py index c85e78e..52b7694 100644 --- a/polymorphic/tests.py +++ b/polymorphic/tests.py @@ -40,6 +40,12 @@ class ModelShow3(ShowFieldTypeAndContent, PolymorphicModel): field1 = models.CharField(max_length=10) m2m = models.ManyToManyField('self') +class ModelShow1_plain(PolymorphicModel): + field1 = models.CharField(max_length=10) +class ModelShow2_plain(ModelShow1_plain): + field2 = models.CharField(max_length=10) + + class Base(ShowFieldType, PolymorphicModel): field_b = models.CharField(max_length=10) class ModelX(Base): @@ -244,7 +250,7 @@ __test__ = {"doctest": """ >>> settings.DEBUG=True >>> get_version() -'0.5 beta' +'1.0 rc1' ### simple inheritance @@ -253,8 +259,7 @@ __test__ = {"doctest": """ >>> o=Model2B.objects.create(field1='B1', field2='B2') >>> o=Model2C.objects.create(field1='C1', field2='C2', field3='C3') ->>> Model2A.objects.all() -[ , + [ , , ] @@ -287,7 +292,13 @@ __test__ = {"doctest": """ [ ] >>> ModelShow3.objects.all().annotate(Count('m2m')) [ ] - + +# no pretty printing +>>> o=ModelShow1_plain.objects.create(field1='abc') +>>> o=ModelShow2_plain.objects.create(field1='abc', field2='def') +>>> ModelShow1_plain.objects.all() +[, ] + ### extra() method diff --git a/rst-to-html.py b/rst-to-html.py new file mode 100755 index 0000000..836de10 --- /dev/null +++ b/rst-to-html.py @@ -0,0 +1,19 @@ +#!/usr/bin/python + +import sys,os + +css='--stylesheet-path=rst.css' + +def conv(name): + if len(sys.argv)>1: + os.system('rst2html.py '+css+' %s.rst >%s.html' % (name, name) ) + else: + os.system('rst2html.py '+css+' %s.rst >%s.html ; firefox %s.html' % (name, name, name) ) + +conv('DOCS') +conv('README') +conv('CHANGES') + +sys.exit() + + diff --git a/rst.css b/rst.css new file mode 100644 index 0000000..f2bfab2 --- /dev/null +++ b/rst.css @@ -0,0 +1,197 @@ +h1, h2, h3, h4 { + color: #47c; +} + +a:hover { border-bottom: 1px solid #0066cc; } +a {color: #0066cc; text-decoration: none;} + + +tt { + color: #080; + } + +blockquote tt { + color: #000 +} + +.first { + margin-top: 0 } + +.last { + margin-bottom: 0 } + +/* +a.toc-backref { + text-decoration: none ; + color: black } +*/ + +dd { + margin-bottom: 0.5em } + +div.abstract { + margin: 2em 5em } + +div.abstract p.topic-title { + font-weight: bold ; + text-align: center } + +div.attention, div.caution, div.danger, div.error, div.hint, +div.important, div.note, div.tip, div.warning { + margin: 2em ; + border: medium outset ; + padding: 1em } + +div.attention p.admonition-title, div.caution p.admonition-title, +div.danger p.admonition-title, div.error p.admonition-title, +div.warning p.admonition-title { + color: red ; + font-weight: bold ; + font-family: sans-serif } + +div.hint p.admonition-title, div.important p.admonition-title, +div.note p.admonition-title, div.tip p.admonition-title { + font-weight: bold ; + font-family: sans-serif } + +div.dedication { + margin: 2em 5em ; + text-align: center ; + font-style: italic } + +div.dedication p.topic-title { + font-weight: bold ; + font-style: normal } + +div.figure { + margin-left: 2em } + +div.footer, div.header { + font-size: smaller } + +div.system-messages { + margin: 5em } + +div.system-messages h1 { + color: red } + +div.system-message { + border: medium outset ; + padding: 1em } + +div.system-message p.system-message-title { + color: red ; + font-weight: bold } + +div.topic { + margin: 2em } + +h1.title { + text-align: center } + +h2.subtitle { + text-align: center } + +hr { + width: 75% } + +ol.simple, ul.simple { + margin-bottom: 1em } + +ol.arabic { + list-style: decimal } + +ol.loweralpha { + list-style: lower-alpha } + +ol.upperalpha { + list-style: upper-alpha } + +ol.lowerroman { + list-style: lower-roman } + +ol.upperroman { + list-style: upper-roman } + +p.caption { + font-style: italic } + +p.credits { + font-style: italic ; + font-size: smaller } + +p.label { + white-space: nowrap } + +p.topic-title { + font-weight: bold } + +pre.address { + margin-bottom: 0 ; + margin-top: 0 ; + font-family: serif ; + font-size: 100% } + +pre.line-block { + font-family: serif ; + font-size: 100% } + +pre.literal-block, pre.doctest-block { + margin-left: 2em ; + margin-right: 2em ; + background-color: #eeeeee } + +span.classifier { + font-family: sans-serif ; + font-style: oblique } + +span.classifier-delimiter { + font-family: sans-serif ; + font-weight: bold } + +span.interpreted { + font-family: sans-serif } + +span.option-argument { + font-style: italic } + +span.pre { + white-space: pre } + +span.problematic { + color: red } + +table { + margin-top: 0.5em ; + margin-bottom: 0.5em } + +table.citation { + border-left: solid thin gray ; + padding-left: 0.5ex } + +table.docinfo { + margin: 2em 4em } + +table.footnote { + border-left: solid thin black ; + padding-left: 0.5ex } + +td, th { + padding-left: 0.5em ; + padding-right: 0.5em ; + vertical-align: top } + +th.docinfo-name, th.field-name { + font-weight: bold ; + text-align: left ; + white-space: nowrap } + +h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt { + font-size: 100% } + +tt, pre.literal-block, pre.doctest-block { + font-size: 115%; + line-height: 150% } + +ul.auto-toc { + list-style-type: none } diff --git a/settings.py b/settings.py index f646415..2eafaa8 100644 --- a/settings.py +++ b/settings.py @@ -9,13 +9,28 @@ ADMINS = ( MANAGERS = ADMINS -DATABASE_ENGINE = 'sqlite3' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'. -DATABASE_NAME = '/var/tmp/django-polymorphic-test-db.sqlite3' # Or path to database file if using sqlite3. -DATABASE_USER = '' # Not used with sqlite3. -DATABASE_PASSWORD = '' # Not used with sqlite3. +import django +import os -DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3. -DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3. +if os.path.ismount('/ram'): + SQLITE_DB_PATH = '/ram/django-polymorphic-test-db.sqlite3' +else: + SQLITE_DB_PATH = '/var/tmp/django-polymorphic-test-db.sqlite3' + +if django.VERSION[:2][0]>=1 and django.VERSION[:2][1]>=3: + DATABASES = { + 'default': { + 'ENGINE': 'django.db.backends.sqlite3', + 'NAME': '/var/tmp/django-polymorphic-test-db.sqlite3' + } + } +else: + DATABASE_ENGINE = 'sqlite3' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'. + DATABASE_NAME = '/var/tmp/django-polymorphic-test-db.sqlite3' # Or path to database file if using sqlite3. + DATABASE_USER = '' # Not used with sqlite3. + DATABASE_PASSWORD = '' # Not used with sqlite3. + DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3. + DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3. # Local time zone for this installation. Choices can be found here: # http://en.wikipedia.org/wiki/List_of_tz_zones_by_name diff --git a/test_all_python_versions b/test_all_python_versions deleted file mode 100755 index 5f8ef70..0000000 --- a/test_all_python_versions +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/bash - -function testit { - if which $1 ; then - if ! $1 manage.py test ; then echo ERROR ; exit 10 ; fi - else - echo "### $1 is not installed!" - fi -} - -testit python2.4 -testit python2.5 -testit python2.6 - diff --git a/test_all_versions b/test_all_versions new file mode 100755 index 0000000..4b09640 --- /dev/null +++ b/test_all_versions @@ -0,0 +1,62 @@ +#!/bin/bash + +# this test script runs "./manage.py test" for +# all supported python versions (2.4, 2.5, 2.6) +# and all supported Django versions (1.1, 1.2, 1.3) + +# it needs symbolic links named "django1.1" and "django1.2" etc. in: +# libraries-local/django-versions +# which point to the respective django versions + +cd libraries-local +rm -f django-orig +if test -e django ; then mv django django-orig ; fi +cd .. + +function restore_django { + echo "### restoring original libraries-local/django" + cd libraries-local + if test -e django-orig ; then mv django-orig django ; fi + cd . +} + +function test_python_version { + if which python$1 ; then + if ! python$1 manage.py test ; then + echo ERROR + restore_django + exit 10 + fi + else + echo + echo "### python $1 is not installed!" + echo + fi +} + +function test_all_python_versions { + test_python_version 2.4 + test_python_version 2.5 + test_python_version 2.6 +} + +function test_django_version { + if ! test -e libraries-local/django-versions/django$1 ; then + echo + echo "### django $1 is not installed!" + echo + return + fi + cd libraries-local + rm -f django + ln -s django-versions/django$1 django + cd .. + test_all_python_versions +} + +test_django_version 1.1 +test_django_version 1.2 +test_django_version 1.3 + +restore_django + diff --git a/test_dumpdata b/test_dumpdata new file mode 100755 index 0000000..77da0e2 --- /dev/null +++ b/test_dumpdata @@ -0,0 +1,26 @@ +#!/bin/bash + +rm -f /var/tmp/django-polymorphic-test-db.sqlite3 +rm -f /ram/django-polymorphic-test-db.sqlite3 +TMPFILE=/tmp/django-polymorphic-test.dump + +PYCMD="python$1" + +echo +echo "#####################################################################" +echo "### Testing dumpdata" +echo + +$PYCMD ./manage.py syncdb +$PYCMD ./manage.py polymorphic_create_test_data + +$PYCMD ./manage.py dumpdata --indent=4 pexp >$TMPFILE + +if ! diff -w $TMPFILE pexp/dumpdata_test_correct_output.txt ; then + echo "#####################################################################" + echo "ERROR: test_dumpdata failed!" + exit 10 +fi +echo "#####################################################################" +echo 'SUCCESS!' +