Briq/django-polymorphic
8
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

docs

Browse Source
This commit is contained in:
Bert Constantin
2010-11-11 17:11:01 +01:00
parent 748d10d2d5
commit ca329ff9b4
8 changed files with 309 additions and 275 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DOCS.html
+50 -91
View File
@@ -232,10 +232,9 @@ ul.auto-toc {
<li><a class="reference internal" href="#more-polymorphic-functionality" id="id5">More Polymorphic Functionality</a></li>
<li><a class="reference internal" href="#custom-managers-querysets-manager-inheritance" id="id6">Custom Managers, Querysets &amp; Manager Inheritance</a></li>
<li><a class="reference internal" href="#performance-considerations" id="id7">Performance Considerations</a></li>
<li><a class="reference internal" href="#possible-optimizations" id="id8">Possible Optimizations</a></li>
<li><a class="reference internal" href="#restrictions-caveats" id="id9">Restrictions &amp; Caveats</a></li>
<li><a class="reference internal" href="#project-status" id="id10">Project Status</a></li>
<li><a class="reference internal" href="#links" id="id11">Links</a></li>
<li><a class="reference internal" href="#restrictions-caveats" id="id8">Restrictions &amp; Caveats</a></li>
<li><a class="reference internal" href="#project-status" id="id9">Project Status</a></li>
<li><a class="reference internal" href="#links" id="id10">Links</a></li>
</ul>
</div>
</div>
@@ -269,8 +268,8 @@ class ResearchProject(Project):
<div class="section" id="create-some-objects">
<h2>Create some objects</h2>
<pre class="doctest-block">
&gt;&gt;&gt; Project.objects.create(topic=&quot;John's Gathering&quot;)
&gt;&gt;&gt; ArtProject.objects.create(topic=&quot;Sculpting with Tim&quot;, artist=&quot;T. Turner&quot;)
&gt;&gt;&gt; Project.objects.create(topic=&quot;Department Party&quot;)
&gt;&gt;&gt; ArtProject.objects.create(topic=&quot;Painting with Tim&quot;, artist=&quot;T. Turner&quot;)
&gt;&gt;&gt; ResearchProject.objects.create(topic=&quot;Swallow Aerodynamics&quot;, supervisor=&quot;Dr. Winter&quot;)
</pre>
</div>
@@ -278,7 +277,7 @@ class ResearchProject(Project):
<h2>Get polymorphic query results</h2>
<pre class="doctest-block">
&gt;&gt;&gt; Project.objects.all()
[ &lt;Project: id 1, topic &quot;John's Gathering&quot;&gt;,
[ &lt;Project: id 1, topic &quot;Department Party&quot;&gt;,
&lt;ArtProject: id 2, topic &quot;Painting with Tim&quot;, artist &quot;T. Turner&quot;&gt;,
&lt;ResearchProject: id 3, topic &quot;Swallow Aerodynamics&quot;, supervisor &quot;Dr. Winter&quot;&gt; ]
</pre>
@@ -302,21 +301,27 @@ or supervisor (note the three underscores):</p>
<p>This is basically all you need to know, as django_polymorphic mostly
works fully automatic and just delivers the expected (&quot;pythonic&quot;) results.</p>
<p>Note: In all example output, above and below, for a nicer and more informative
output the <cite>ShowFieldType</cite> mixin has been used (documented below).</p>
output the <tt class="docutils literal">ShowFieldType</tt> mixin has been used (documented below).</p>
</div>
</div>
<div class="section" id="list-of-features">
<h1><a class="toc-backref" href="#id3">List of Features</a></h1>
<ul class="simple">
<li>Fully automatic; generally makes sure that the same objects are returned
from the database that were stored there, regardless how they are retrieved</li>
<li>Fully automatic - generally makes sure that the same objects are
returned from the database that were stored there, regardless how
they are retrieved</li>
<li>Only on models that request polymorphic behaviour however (and the
models inheriting from them)</li>
<li>Full support for ForeignKeys, ManyToManyFields and OneToToneFields</li>
<li>Filtering for classes, equivalent to python's isinstance(): instance_of(...), not_instance_of(...)</li>
<li>Polymorphic filtering/ordering etc., allowing the use of fields of derived models (&quot;ArtProject___artist&quot;)</li>
<li>Filtering for classes, equivalent to python's isinstance():
<tt class="docutils literal"><span class="pre">instance_of(...)</span></tt> and <tt class="docutils literal"><span class="pre">not_instance_of(...)</span></tt></li>
<li>Polymorphic filtering/ordering etc., allowing the use of fields of
derived models (&quot;ArtProject___artist&quot;)</li>
<li>Support for user-defined custom managers</li>
<li>Automatic inheritance of custom managers</li>
<li>Support for user-defined custom queryset classes</li>
<li>Non-polymorphic queries, if needed - with no other change in features/behaviour</li>
<li>Non-polymorphic queries if needed, with no other change in
features/behaviour</li>
<li>Combining querysets of different types/models (&quot;qs3 = qs1 | qs2&quot;)</li>
<li>Nice/informative display of polymorphic queryset results</li>
</ul>
@@ -401,8 +406,8 @@ syntax: <tt class="docutils literal">exact model name + three _ + field name</tt
&lt;ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)&gt; ]
</pre>
</div>
<div class="section" id="combining-querysets-querysets-as-object-containers">
<h2>Combining Querysets / Querysets as &quot;Object Containers&quot;</h2>
<div class="section" id="combining-querysets">
<h2>Combining Querysets</h2>
<p>Querysets could now be regarded as object containers that allow the
aggregation of different object types, very similar to python
lists - as long as the objects are accessed through the manager of
@@ -465,24 +470,23 @@ class MyModelWithThirdParty(MyBaseModel, ThirdPartyModel):
</div>
<div class="section" id="non-polymorphic-queries">
<h2>Non-Polymorphic Queries</h2>
<pre class="doctest-block">
&gt;&gt;&gt; qs=ModelA.objects.non_polymorphic().all()
&gt;&gt;&gt; qs
.
[ &lt;ModelA: id 1, field1 (CharField)&gt;,
&lt;ModelA: id 2, field1 (CharField)&gt;,
&lt;ModelA: id 3, field1 (CharField)&gt; ]
</pre>
<p>If you insert <tt class="docutils literal">.non_polymorphic()</tt> anywhere into the query chain, then
django_polymorphic will simply leave out the final step of retrieving the
real objects, and the manager/queryset will return objects of the type of
the base class you used for the query, like vanilla Django would
(<tt class="docutils literal">ModelA</tt> in this example).</p>
<pre class="doctest-block">
&gt;&gt;&gt; qs=ModelA.objects.non_polymorphic().all()
&gt;&gt;&gt; qs
[ &lt;ModelA: id 1, field1 (CharField)&gt;,
&lt;ModelA: id 2, field1 (CharField)&gt;,
&lt;ModelA: id 3, field1 (CharField)&gt; ]
</pre>
<p>There are no other changes in the behaviour of the queryset. For example,
enhancements for <tt class="docutils literal">filter()</tt> or <tt class="docutils literal">instance_of()</tt> etc. still work as expected.
If you do the final step yourself, you get the usual polymorphic result:</p>
<pre class="doctest-block">
&gt;&gt;&gt; qs.get_real_instances()
&gt;&gt;&gt; ModelA.objects.get_real_instances(qs)
[ &lt;ModelA: id 1, field1 (CharField)&gt;,
&lt;ModelB: id 2, field1 (CharField), field2 (CharField)&gt;,
&lt;ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)&gt; ]
@@ -554,12 +558,14 @@ class ModelA(ShowFieldType, PolymorphicModel):
<p>You may also use ShowFieldContent or ShowFieldTypeAndContent to display
additional information when printing querysets (or converting them to text).</p>
<p>When showing field contents, they will be truncated to 20 characters. You can
modify this behaviour by setting a class variable like this:</p>
modify this behaviour by setting a class variable in your model like this:</p>
<pre class="literal-block">
class ModelA(ShowFieldType, PolymorphicModel):
polymorphic_showfield_max_field_width = 20
...
</pre>
<p>Similarly, pre-V1.0 output formatting can be re-estated by using
<tt class="docutils literal">polymorphic_showfield_old_format = True</tt>.</p>
</div>
</div>
<div class="section" id="custom-managers-querysets-manager-inheritance">
@@ -649,11 +655,6 @@ class MyModel(PolymorphicModel):
<p>The current implementation is rather simple and does not use any
custom SQL or Django DB layer internals - it is purely based on the
standard Django ORM.</p>
<p>The advantages are that the implementation naturally works on all
supported database management systems, and consists of rather
clean source code which can be easily understood and enhanced.</p>
<p>The disadvantage is that this approach can not deliver the optimum
performance as it introduces additional database queries.</p>
<p>Specifically, the query:</p>
<pre class="literal-block">
result_objects = list( ModelA.objects.filter(...) )
@@ -670,69 +671,27 @@ without a tool like django_polymorphic, this usually results in a variation of</
<pre class="literal-block">
result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ]
</pre>
<p>which has exceptionally bad performance, as it introduces one additional
SQL query for every object in the result which is not of class <tt class="docutils literal">BaseModel</tt>.
Relative to this, the performance of the current django_polymorphic
implementation is very good.</p>
<p>If your project however needs perfect performance and the current
performance implications of django_polymorphic are not acceptable, then
basically there are the two options of either foregoing of an essential aspect
of object oriented programming or optimizing django_polymorphic.</p>
<p>Foregoing the benefits of this aspect of object oriented programming
for projects that could benefit from it will however usually lead to bloated code,
unnecessary complexity and considerably more of the programmer's time to
create and update the implementation, together with the disadvantages
of a less flexible and less future-proof solution. Throwing a little more
hardware on the problem might be the least expensive solution in most cases.</p>
</div>
<div class="section" id="possible-optimizations">
<h1><a class="toc-backref" href="#id8">Possible Optimizations</a></h1>
<p>Django_polymorphic can be optimized to require only one
SQL query for the queryset evaluation and retrieval of all objects.</p>
<p>Probably all that would be needed seems support for an additional
queryset function in Django's database layer, like:</p>
<pre class="literal-block">
ModelA.objects.join_models(on=&quot;field_name&quot;, models=[ModelB, ModelC])
</pre>
<p>or, less general but more simple:</p>
<pre class="literal-block">
ModelA.objects.join_tables(on=&quot;field_name&quot;, tables=['tableB','tableC'])
</pre>
<p>This would add additional left outer joins to the query and then add
the resulting fields from this join to the result objects.
E.g. a query for <tt class="docutils literal">ModelA</tt> objects would need to join the <tt class="docutils literal">ModelB</tt>
and <tt class="docutils literal">ModelC</tt> tables on the the field <tt class="docutils literal">id</tt> and add the fields <tt class="docutils literal">field2</tt>
and <tt class="docutils literal">field3</tt> from the joined tables to the resulting objects.</p>
<p>An optimization like this might require an SQL database.
For non-SQL databases the implementation could fall back to
the current ORM-only implementation.</p>
<div class="section" id="sql-complexity-of-an-optimized-implementation">
<h2>SQL Complexity of an Optimized Implementation</h2>
<p>With only one SQL query, one SQL join for each possible subclass
would be needed (<tt class="docutils literal">BaseModel.__subclasses__()</tt>, recursively).</p>
<p>With two SQL queries, the number of joins could be reduced to the
number of actuallly occurring subclasses in the specific result.</p>
<p>A perfect implementation might want to use one query only
if the number of possible subclasses (and therefore joins) is not
too large, and two queries otherwise (using the first query to
determine the actually occurring subclasses, reducing the number
of joins for the second).</p>
<p>The number of joins needed for polymorphic object retrieval might
raise concerns regarding the efficiency of these database
queries. It seems likely however, that the increased number of joins
is no problem for the supported DBM systems in all realistic use cases.
Should the number of joins of the more extreme use cases turn out to
be problematic, it is possible to split any problematic query into, for example,
two queries with only half the number of joins each.</p>
<p>It seems that further optimization (down to one DB request)
of django_polymorphic would be restricted to a relatively small area of
the code (&quot;query.py&quot;), and be pretty much independent from the rest of the module.
Such an optimization can be done at any later time (like when it's needed).</p>
<p>which has very bad performance, as it introduces one additional
SQL query for every object in the result which is not of class <tt class="docutils literal">BaseModel</tt>.</p>
<p>Compared to these solutions, django_polymorphic has the advantage
that it only needs one sql request per <em>object type</em>, and not <em>per object</em>.</p>
<div class="section" id="performance-problems-with-postgresql-mysql-and-sqlite3">
<span id="performance"></span><h2>Performance Problems with PostgreSQL, MySQL and SQLite3</h2>
<p>Current relational DBM systems seem to be have general problems with
the SQL queries produced by object relational mappers like the Django
ORM, if these use multi-table inheritance like Django's ORM does.
The &quot;inner joins&quot; in these queries can perform very badly.
This is independent of django_polymorphic and affects all uses of
multi table Model inheritance.</p>
<p>Concrete benchmark results are forthcoming (please see discussion forum).</p>
<p>Please also see this <a class="reference external" href="http://www.jacobian.org/writing/concrete-inheritance/">post (and comments) from Jacob Kaplan-Moss</a>.</p>
</div>
</div>
<div class="section" id="restrictions-caveats">
<span id="restrictions"></span><h1><a class="toc-backref" href="#id9">Restrictions &amp; Caveats</a></h1>
<span id="restrictions"></span><h1><a class="toc-backref" href="#id8">Restrictions &amp; Caveats</a></h1>
<ul class="simple">
<li>Database Performance regarding concrete Model inheritance in general
Please see &quot;Performance Problems&quot; above.</li>
<li>Queryset methods <tt class="docutils literal">values()</tt>, <tt class="docutils literal">values_list()</tt>, <tt class="docutils literal">select_related()</tt>,
<tt class="docutils literal">defer()</tt> and <tt class="docutils literal">only()</tt> are not yet fully supported (see above).
<tt class="docutils literal">extra()</tt> has one restriction: the resulting objects are required to have
@@ -772,13 +731,13 @@ use ContentType). This issue seems to be resolved for Django 1.2
</ul>
</div>
<div class="section" id="project-status">
<h1><a class="toc-backref" href="#id10">Project Status</a></h1>
<h1><a class="toc-backref" href="#id9">Project Status</a></h1>
<p>Django_polymorphic works well for a considerable number of users now,
and no major problems have shown up for many months.
The API can be considered stable beginning with the V1.0 release.</p>
</div>
<div class="section" id="links">
<h1><a class="toc-backref" href="#id11">Links</a></h1>
<h1><a class="toc-backref" href="#id10">Links</a></h1>
<ul class="simple">
<li><a class="reference external" href="http://code.djangoproject.com/wiki/ModelInheritance">http://code.djangoproject.com/wiki/ModelInheritance</a></li>
<li><a class="reference external" href="http://lazypython.blogspot.com/2009/02/second-look-at-inheritance-and.html">http://lazypython.blogspot.com/2009/02/second-look-at-inheritance-and.html</a></li>