==========================

``QuerySet`` API reference

==========================

.. currentmodule:: django.db.models.query

This document describes the details of the ``QuerySet`` API. It builds on the

material presented in the :doc:`model </topics/db/models>` and :doc:`database

query </topics/db/queries>` guides, so you'll probably want to read and

understand those documents before reading this one.

Throughout this reference we'll use the :ref:`example blog models

<queryset-model-example>` presented in the :doc:`database query guide

</topics/db/queries>`.

.. _when-querysets-are-evaluated:

When ``QuerySet``\s are evaluated

=================================

Internally, a ``QuerySet`` can be constructed, filtered, sliced, and generally

passed around without actually hitting the database. No database activity

actually occurs until you do something to evaluate the queryset.

You can evaluate a ``QuerySet`` in the following ways:

* **Iteration.** A ``QuerySet`` is iterable, and it executes its database

  query the first time you iterate over it. For example, this will print

  the headline of all entries in the database::

      for e in Entry.objects.all():

          print(e.headline)

  Note: Don't use this if all you want to do is determine if at least one

  result exists. It's more efficient to use :meth:`~QuerySet.exists`.

* **Asynchronous iteration.**. A ``QuerySet`` can also be iterated over using

  ``async for``::

      async for e in Entry.objects.all():

        results.append(e)

  Both synchronous and asynchronous iterators of QuerySets share the same

  underlying cache.

  .. versionchanged:: 4.1

    Support for asynchronous iteration was added.

* **Slicing.** As explained in :ref:`limiting-querysets`, a ``QuerySet`` can

  be sliced, using Python's array-slicing syntax. Slicing an unevaluated

  ``QuerySet`` usually returns another unevaluated ``QuerySet``, but Django

  will execute the database query if you use the "step" parameter of slice

  syntax, and will return a list. Slicing a ``QuerySet`` that has been

  evaluated also returns a list.

  Also note that even though slicing an unevaluated ``QuerySet`` returns

  another unevaluated ``QuerySet``, modifying it further (e.g., adding

  more filters, or modifying ordering) is not allowed, since that does not

  translate well into SQL and it would not have a clear meaning either.

* **Pickling/Caching.** See the following section for details of what

  is involved when `pickling QuerySets`_. The important thing for the

  purposes of this section is that the results are read from the database.

* **repr().** A ``QuerySet`` is evaluated when you call ``repr()`` on it.

  This is for convenience in the Python interactive interpreter, so you can

  immediately see your results when using the API interactively.

* **len().** A ``QuerySet`` is evaluated when you call ``len()`` on it.

  This, as you might expect, returns the length of the result list.

  Note: If you only need to determine the number of records in the set (and

  don't need the actual objects), it's much more efficient to handle a count

  at the database level using SQL's ``SELECT COUNT(*)``. Django provides a

  :meth:`~QuerySet.count` method for precisely this reason.

* **list().** Force evaluation of a ``QuerySet`` by calling ``list()`` on

  it. For example::

      entry_list = list(Entry.objects.all())

* **bool().** Testing a ``QuerySet`` in a boolean context, such as using

  ``bool()``, ``or``, ``and`` or an ``if`` statement, will cause the query

  to be executed. If there is at least one result, the ``QuerySet`` is

  ``True``, otherwise ``False``. For example::

      if Entry.objects.filter(headline="Test"):

         print("There is at least one Entry with the headline Test")

  Note: If you only want to determine if at least one result exists (and don't

  need the actual objects), it's more efficient to use :meth:`~QuerySet.exists`.

.. _pickling QuerySets:

Pickling ``QuerySet``\s

-----------------------

If you :mod:`pickle` a ``QuerySet``, this will force all the results to be loaded

into memory prior to pickling. Pickling is usually used as a precursor to

caching and when the cached queryset is reloaded, you want the results to

already be present and ready for use (reading from the database can take some

time, defeating the purpose of caching). This means that when you unpickle a

``QuerySet``, it contains the results at the moment it was pickled, rather

than the results that are currently in the database.

If you only want to pickle the necessary information to recreate the

``QuerySet`` from the database at a later time, pickle the ``query`` attribute

of the ``QuerySet``. You can then recreate the original ``QuerySet`` (without

any results loaded) using some code like this::

    >>> import pickle

    >>> query = pickle.loads(s)     # Assuming 's' is the pickled string.

    >>> qs = MyModel.objects.all()

    >>> qs.query = query            # Restore the original 'query'.

The ``query`` attribute is an opaque object. It represents the internals of

the query construction and is not part of the public API. However, it is safe

(and fully supported) to pickle and unpickle the attribute's contents as

described here.

.. admonition:: Restrictions on ``QuerySet.values_list()``

    If you recreate :meth:`QuerySet.values_list` using the pickled ``query``

    attribute, it will be converted to :meth:`QuerySet.values`::

        >>> import pickle

        >>> qs = Blog.objects.values_list('id', 'name')

        >>> qs

        <QuerySet [(1, 'Beatles Blog')]>

        >>> reloaded_qs = Blog.objects.all()

        >>> reloaded_qs.query = pickle.loads(pickle.dumps(qs.query))

        >>> reloaded_qs

        <QuerySet [{'id': 1, 'name': 'Beatles Blog'}]>

.. admonition:: You can't share pickles between versions

    Pickles of ``QuerySets`` are only valid for the version of Django that

    was used to generate them. If you generate a pickle using Django

    version N, there is no guarantee that pickle will be readable with

    Django version N+1. Pickles should not be used as part of a long-term

    archival strategy.

    Since pickle compatibility errors can be difficult to diagnose, such as

    silently corrupted objects, a ``RuntimeWarning`` is raised when you try to

    unpickle a queryset in a Django version that is different than the one in

    which it was pickled.

.. _queryset-api:

``QuerySet`` API

================

Here's the formal declaration of a ``QuerySet``:

.. class:: QuerySet(model=None, query=None, using=None, hints=None)

    Usually when you'll interact with a ``QuerySet`` you'll use it by

    :ref:`chaining filters <chaining-filters>`. To make this work, most

    ``QuerySet`` methods return new querysets. These methods are covered in

    detail later in this section.

    The ``QuerySet`` class has the following public attributes you can use for

    introspection:

    .. attribute:: ordered

        ``True`` if the ``QuerySet`` is ordered — i.e. has an

        :meth:`order_by()` clause or a default ordering on the model.

        ``False`` otherwise.

    .. attribute:: db

        The database that will be used if this query is executed now.

    .. note::

        The ``query`` parameter to :class:`QuerySet` exists so that specialized

        query subclasses can reconstruct internal query state. The value of the

        parameter is an opaque representation of that query state and is not

        part of a public API.

.. currentmodule:: django.db.models.query.QuerySet

Methods that return new ``QuerySet``\s

--------------------------------------

Django provides a range of ``QuerySet`` refinement methods that modify either

the types of results returned by the ``QuerySet`` or the way its SQL query is

executed.

.. note::

    These methods do not run database queries, therefore they are **safe to**

    **run in asynchronous code**, and do not have separate asynchronous

    versions.

``filter()``

~~~~~~~~~~~~

.. method:: filter(*args, **kwargs)

Returns a new ``QuerySet`` containing objects that match the given lookup

parameters.

The lookup parameters (``**kwargs``) should be in the format described in

`Field lookups`_ below. Multiple parameters are joined via ``AND`` in the

underlying SQL statement.

If you need to execute more complex queries (for example, queries with ``OR`` statements),

you can use :class:`Q objects <django.db.models.Q>` (``*args``).

``exclude()``

~~~~~~~~~~~~~

.. method:: exclude(*args, **kwargs)

Returns a new ``QuerySet`` containing objects that do *not* match the given

lookup parameters.

The lookup parameters (``**kwargs``) should be in the format described in

`Field lookups`_ below. Multiple parameters are joined via ``AND`` in the

underlying SQL statement, and the whole thing is enclosed in a ``NOT()``.

This example excludes all entries whose ``pub_date`` is later than 2005-1-3

AND whose ``headline`` is "Hello"::

    Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3), headline='Hello')

In SQL terms, that evaluates to:

.. code-block:: sql

    SELECT ...

    WHERE NOT (pub_date > '2005-1-3' AND headline = 'Hello')

This example excludes all entries whose ``pub_date`` is later than 2005-1-3

OR whose headline is "Hello"::

    Entry.objects.exclude(pub_date__gt=datetime.date(2005, 1, 3)).exclude(headline='Hello')

In SQL terms, that evaluates to:

.. code-block:: sql

    SELECT ...

    WHERE NOT pub_date > '2005-1-3'

    AND NOT headline = 'Hello'

Note the second example is more restrictive.

If you need to execute more complex queries (for example, queries with ``OR`` statements),

you can use :class:`Q objects <django.db.models.Q>` (``*args``).

``annotate()``

~~~~~~~~~~~~~~

.. method:: annotate(*args, **kwargs)

Annotates each object in the ``QuerySet`` with the provided list of :doc:`query

expressions </ref/models/expressions>`. An expression may be a simple value, a

reference to a field on the model (or any related models), or an aggregate

expression (averages, sums, etc.) that has been computed over the objects that

are related to the objects in the ``QuerySet``.

Each argument to ``annotate()`` is an annotation that will be added

to each object in the ``QuerySet`` that is returned.

The aggregation functions that are provided by Django are described

in `Aggregation Functions`_ below.

Annotations specified using keyword arguments will use the keyword as

the alias for the annotation. Anonymous arguments will have an alias

generated for them based upon the name of the aggregate function and

the model field that is being aggregated. Only aggregate expressions

that reference a single field can be anonymous arguments. Everything

else must be a keyword argument.

For example, if you were manipulating a list of blogs, you may want

to determine how many entries have been made in each blog::

    >>> from django.db.models import Count

    >>> q = Blog.objects.annotate(Count('entry'))

    # The name of the first blog

    >>> q[0].name

    'Blogasaurus'

    # The number of entries on the first blog

    >>> q[0].entry__count

    42

The ``Blog`` model doesn't define an ``entry__count`` attribute by itself,

but by using a keyword argument to specify the aggregate function, you can

control the name of the annotation::

    >>> q = Blog.objects.annotate(number_of_entries=Count('entry'))

    # The number of entries on the first blog, using the name provided

    >>> q[0].number_of_entries

    42

For an in-depth discussion of aggregation, see :doc:`the topic guide on

Aggregation </topics/db/aggregation>`.

``alias()``

~~~~~~~~~~~

.. method:: alias(*args, **kwargs)

Same as :meth:`annotate`, but instead of annotating objects in the

``QuerySet``, saves the expression for later reuse with other ``QuerySet``

methods. This is useful when the result of the expression itself is not needed

but it is used for filtering, ordering, or as a part of a complex expression.

Not selecting the unused value removes redundant work from the database which

should result in better performance.

For example, if you want to find blogs with more than 5 entries, but are not

interested in the exact number of entries, you could do this::

    >>> from django.db.models import Count

    >>> blogs = Blog.objects.alias(entries=Count('entry')).filter(entries__gt=5)

``alias()`` can be used in conjunction with :meth:`annotate`, :meth:`exclude`,

:meth:`filter`, :meth:`order_by`, and :meth:`update`. To use aliased expression

with other methods (e.g. :meth:`aggregate`), you must promote it to an

annotation::

    Blog.objects.alias(entries=Count('entry')).annotate(

        entries=F('entries'),

    ).aggregate(Sum('entries'))

:meth:`filter` and :meth:`order_by` can take expressions directly, but

expression construction and usage often does not happen in the same place (for

example, ``QuerySet`` method creates expressions, for later use in views).

``alias()`` allows building complex expressions incrementally, possibly

spanning multiple methods and modules, refer to the expression parts by their

aliases and only use :meth:`annotate` for the final result.

``order_by()``

~~~~~~~~~~~~~~

.. method:: order_by(*fields)

By default, results returned by a ``QuerySet`` are ordered by the ordering

tuple given by the ``ordering`` option in the model's ``Meta``. You can

override this on a per-``QuerySet`` basis by using the ``order_by`` method.

Example::

    Entry.objects.filter(pub_date__year=2005).order_by('-pub_date', 'headline')

The result above will be ordered by ``pub_date`` descending, then by

``headline`` ascending. The negative sign in front of ``"-pub_date"`` indicates

*descending* order. Ascending order is implied. To order randomly, use ``"?"``,

like so::

    Entry.objects.order_by('?')

Note: ``order_by('?')`` queries may be expensive and slow, depending on the

database backend you're using.

To order by a field in a different model, use the same syntax as when you are

querying across model relations. That is, the name of the field, followed by a

double underscore (``__``), followed by the name of the field in the new model,

and so on for as many models as you want to join. For example::

    Entry.objects.order_by('blog__name', 'headline')

If you try to order by a field that is a relation to another model, Django will

use the default ordering on the related model, or order by the related model's

primary key if there is no :attr:`Meta.ordering

<django.db.models.Options.ordering>` specified. For example, since the ``Blog``

model has no default ordering specified::

    Entry.objects.order_by('blog')

...is identical to::

    Entry.objects.order_by('blog__id')

If ``Blog`` had ``ordering = ['name']``, then the first queryset would be

identical to::

    Entry.objects.order_by('blog__name')

You can also order by :doc:`query expressions </ref/models/expressions>` by

calling :meth:`~.Expression.asc` or :meth:`~.Expression.desc` on the

expression::

    Entry.objects.order_by(Coalesce('summary', 'headline').desc())

:meth:`~.Expression.asc` and :meth:`~.Expression.desc` have arguments

(``nulls_first`` and ``nulls_last``) that control how null values are sorted.

Be cautious when ordering by fields in related models if you are also using

:meth:`distinct()`. See the note in :meth:`distinct` for an explanation of how

related model ordering can change the expected results.

.. note::

    It is permissible to specify a multi-valued field to order the results by

    (for example, a :class:`~django.db.models.ManyToManyField` field, or the

    reverse relation of a :class:`~django.db.models.ForeignKey` field).

    Consider this case::

         class Event(Model):

            parent = models.ForeignKey(

                'self',

                on_delete=models.CASCADE,

                related_name='children',

            )

            date = models.DateField()

         Event.objects.order_by('children__date')

    Here, there could potentially be multiple ordering data for each ``Event``;

    each ``Event`` with multiple ``children`` will be returned multiple times

    into the new ``QuerySet`` that ``order_by()`` creates. In other words,

    using ``order_by()`` on the ``QuerySet`` could return more items than you

    were working on to begin with - which is probably neither expected nor

    useful.

    Thus, take care when using multi-valued field to order the results. **If**

    you can be sure that there will only be one ordering piece of data for each

    of the items you're ordering, this approach should not present problems. If

    not, make sure the results are what you expect.

There's no way to specify whether ordering should be case sensitive. With

respect to case-sensitivity, Django will order results however your database

backend normally orders them.

You can order by a field converted to lowercase with

:class:`~django.db.models.functions.Lower` which will achieve case-consistent

ordering::

    Entry.objects.order_by(Lower('headline').desc())

If you don't want any ordering to be applied to a query, not even the default

ordering, call :meth:`order_by()` with no parameters.

You can tell if a query is ordered or not by checking the

:attr:`.QuerySet.ordered` attribute, which will be ``True`` if the

``QuerySet`` has been ordered in any way.

Each ``order_by()`` call will clear any previous ordering. For example, this

query will be ordered by ``pub_date`` and not ``headline``::

    Entry.objects.order_by('headline').order_by('pub_date')

.. warning::

    Ordering is not a free operation. Each field you add to the ordering

    incurs a cost to your database. Each foreign key you add will

    implicitly include all of its default orderings as well.

    If a query doesn't have an ordering specified, results are returned from

    the database in an unspecified order. A particular ordering is guaranteed

    only when ordering by a set of fields that uniquely identify each object in

    the results. For example, if a ``name`` field isn't unique, ordering by it

    won't guarantee objects with the same name always appear in the same order.

``reverse()``

~~~~~~~~~~~~~

.. method:: reverse()

Use the ``reverse()`` method to reverse the order in which a queryset's

elements are returned. Calling ``reverse()`` a second time restores the

ordering back to the normal direction.

To retrieve the "last" five items in a queryset, you could do this::

    my_queryset.reverse()[:5]

Note that this is not quite the same as slicing from the end of a sequence in

Python. The above example will return the last item first, then the

penultimate item and so on. If we had a Python sequence and looked at

``seq[-5:]``, we would see the fifth-last item first. Django doesn't support

that mode of access (slicing from the end), because it's not possible to do it

efficiently in SQL.

Also, note that ``reverse()`` should generally only be called on a ``QuerySet``

which has a defined ordering (e.g., when querying against a model which defines

a default ordering, or when using :meth:`order_by()`). If no such ordering is

defined for a given ``QuerySet``, calling ``reverse()`` on it has no real

effect (the ordering was undefined prior to calling ``reverse()``, and will

remain undefined afterward).

``distinct()``

~~~~~~~~~~~~~~

.. method:: distinct(*fields)

Returns a new ``QuerySet`` that uses ``SELECT DISTINCT`` in its SQL query. This

eliminates duplicate rows from the query results.

By default, a ``QuerySet`` will not eliminate duplicate rows. In practice, this

is rarely a problem, because simple queries such as ``Blog.objects.all()``

don't introduce the possibility of duplicate result rows. However, if your

query spans multiple tables, it's possible to get duplicate results when a

``QuerySet`` is evaluated. That's when you'd use ``distinct()``.

.. note::

    Any fields used in an :meth:`order_by` call are included in the SQL

    ``SELECT`` columns. This can sometimes lead to unexpected results when used

    in conjunction with ``distinct()``. If you order by fields from a related

    model, those fields will be added to the selected columns and they may make

    otherwise duplicate rows appear to be distinct. Since the extra columns

    don't appear in the returned results (they are only there to support

    ordering), it sometimes looks like non-distinct results are being returned.

    Similarly, if you use a :meth:`values()` query to restrict the columns

    selected, the columns used in any :meth:`order_by()` (or default model

    ordering) will still be involved and may affect uniqueness of the results.

    The moral here is that if you are using ``distinct()`` be careful about

    ordering by related models. Similarly, when using ``distinct()`` and

    :meth:`values()` together, be careful when ordering by fields not in the

    :meth:`values()` call.

On PostgreSQL only, you can pass positional arguments (``*fields``) in order to

specify the names of fields to which the ``DISTINCT`` should apply. This

translates to a ``SELECT DISTINCT ON`` SQL query. Here's the difference. For a

normal ``distinct()`` call, the database compares *each* field in each row when

determining which rows are distinct. For a ``distinct()`` call with specified

field names, the database will only compare the specified field names.

.. note::

    When you specify field names, you *must* provide an ``order_by()`` in the

    ``QuerySet``, and the fields in ``order_by()`` must start with the fields in

    ``distinct()``, in the same order.

    For example, ``SELECT DISTINCT ON (a)`` gives you the first row for each

    value in column ``a``. If you don't specify an order, you'll get some

    arbitrary row.

Examples (those after the first will only work on PostgreSQL)::

    >>> Author.objects.distinct()

    [...]

    >>> Entry.objects.order_by('pub_date').distinct('pub_date')

    [...]

    >>> Entry.objects.order_by('blog').distinct('blog')

    [...]

    >>> Entry.objects.order_by('author', 'pub_date').distinct('author', 'pub_date')

    [...]

    >>> Entry.objects.order_by('blog__name', 'mod_date').distinct('blog__name', 'mod_date')

    [...]

    >>> Entry.objects.order_by('author', 'pub_date').distinct('author')

    [...]

.. note::

    Keep in mind that :meth:`order_by` uses any default related model ordering

    that has been defined. You might have to explicitly order by the relation

    ``_id`` or referenced field to make sure the ``DISTINCT ON`` expressions

    match those at the beginning of the ``ORDER BY`` clause. For example, if

    the ``Blog`` model defined an :attr:`~django.db.models.Options.ordering` by

    ``name``::

        Entry.objects.order_by('blog').distinct('blog')

    ...wouldn't work because the query would be ordered by ``blog__name`` thus

    mismatching the ``DISTINCT ON`` expression. You'd have to explicitly order

    by the relation ``_id`` field (``blog_id`` in this case) or the referenced

    one (``blog__pk``) to make sure both expressions match.

``values()``

~~~~~~~~~~~~

.. method:: values(*fields, **expressions)

Returns a ``QuerySet`` that returns dictionaries, rather than model instances,

when used as an iterable.

Each of those dictionaries represents an object, with the keys corresponding to

the attribute names of model objects.

This example compares the dictionaries of ``values()`` with the normal model

objects::

    # This list contains a Blog object.

    >>> Blog.objects.filter(name__startswith='Beatles')

    <QuerySet [<Blog: Beatles Blog>]>

    # This list contains a dictionary.

    >>> Blog.objects.filter(name__startswith='Beatles').values()

    <QuerySet [{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}]>

The ``values()`` method takes optional positional arguments, ``*fields``, which

specify field names to which the ``SELECT`` should be limited. If you specify

the fields, each dictionary will contain only the field keys/values for the

fields you specify. If you don't specify the fields, each dictionary will

contain a key and value for every field in the database table.

Example::

    >>> Blog.objects.values()

    <QuerySet [{'id': 1, 'name': 'Beatles Blog', 'tagline': 'All the latest Beatles news.'}]>

    >>> Blog.objects.values('id', 'name')

    <QuerySet [{'id': 1, 'name': 'Beatles Blog'}]>

The ``values()`` method also takes optional keyword arguments,

``**expressions``, which are passed through to :meth:`annotate`::

    >>> from django.db.models.functions import Lower

    >>> Blog.objects.values(lower_name=Lower('name'))

    <QuerySet [{'lower_name': 'beatles blog'}]>

You can use built-in and :doc:`custom lookups </howto/custom-lookups>` in

ordering. For example::

    >>> from django.db.models import CharField

    >>> from django.db.models.functions import Lower

    >>> CharField.register_lookup(Lower)

    >>> Blog.objects.values('name__lower')

    <QuerySet [{'name__lower': 'beatles blog'}]>

An aggregate within a ``values()`` clause is applied before other arguments

within the same ``values()`` clause. If you need to group by another value,

add it to an earlier ``values()`` clause instead. For example::

    >>> from django.db.models import Count

    >>> Blog.objects.values('entry__authors', entries=Count('entry'))

    <QuerySet [{'entry__authors': 1, 'entries': 20}, {'entry__authors': 1, 'entries': 13}]>

    >>> Blog.objects.values('entry__authors').annotate(entries=Count('entry'))

    <QuerySet [{'entry__authors': 1, 'entries': 33}]>

A few subtleties that are worth mentioning:

* If you have a field called ``foo`` that is a

  :class:`~django.db.models.ForeignKey`, the default ``values()`` call

  will return a dictionary key called ``foo_id``, since this is the name

  of the hidden model attribute that stores the actual value (the ``foo``

  attribute refers to the related model). When you are calling

  ``values()`` and passing in field names, you can pass in either ``foo``

  or ``foo_id`` and you will get back the same thing (the dictionary key

  will match the field name you passed in).

  For example::

    >>> Entry.objects.values()

    <QuerySet [{'blog_id': 1, 'headline': 'First Entry', ...}, ...]>

    >>> Entry.objects.values('blog')

    <QuerySet [{'blog': 1}, ...]>

    >>> Entry.objects.values('blog_id')

    <QuerySet [{'blog_id': 1}, ...]>

* When using ``values()`` together with :meth:`distinct()`, be aware that

  ordering can affect the results. See the note in :meth:`distinct` for

  details.

* If you use a ``values()`` clause after an :meth:`extra()` call,

  any fields defined by a ``select`` argument in the :meth:`extra()` must

  be explicitly included in the ``values()`` call. Any :meth:`extra()` call

  made after a ``values()`` call will have its extra selected fields

  ignored.

* Calling :meth:`only()` and :meth:`defer()` after ``values()`` doesn't make

  sense, so doing so will raise a ``TypeError``.

* Combining transforms and aggregates requires the use of two :meth:`annotate`

  calls, either explicitly or as keyword arguments to :meth:`values`. As above,

  if the transform has been registered on the relevant field type the first

  :meth:`annotate` can be omitted, thus the following examples are equivalent::

    >>> from django.db.models import CharField, Count

    >>> from django.db.models.functions import Lower

    >>> CharField.register_lookup(Lower)

    >>> Blog.objects.values('entry__authors__name__lower').annotate(entries=Count('entry'))

    <QuerySet [{'entry__authors__name__lower': 'test author', 'entries': 33}]>

    >>> Blog.objects.values(

    ...     entry__authors__name__lower=Lower('entry__authors__name')

    ... ).annotate(entries=Count('entry'))

    <QuerySet [{'entry__authors__name__lower': 'test author', 'entries': 33}]>

    >>> Blog.objects.annotate(

    ...     entry__authors__name__lower=Lower('entry__authors__name')

    ... ).values('entry__authors__name__lower').annotate(entries=Count('entry'))

    <QuerySet [{'entry__authors__name__lower': 'test author', 'entries': 33}]>

It is useful when you know you're only going to need values from a small number

of the available fields and you won't need the functionality of a model

instance object. It's more efficient to select only the fields you need to use.

Finally, note that you can call ``filter()``, ``order_by()``, etc. after the

``values()`` call, that means that these two calls are identical::

    Blog.objects.values().order_by('id')

    Blog.objects.order_by('id').values()

The people who made Django prefer to put all the SQL-affecting methods first,

followed (optionally) by any output-affecting methods (such as ``values()``),

but it doesn't really matter. This is your chance to really flaunt your

individualism.

You can also refer to fields on related models with reverse relations through

``OneToOneField``, ``ForeignKey`` and ``ManyToManyField`` attributes::

    >>> Blog.objects.values('name', 'entry__headline')

    <QuerySet [{'name': 'My blog', 'entry__headline': 'An entry'},

         {'name': 'My blog', 'entry__headline': 'Another entry'}, ...]>

.. warning::

   Because :class:`~django.db.models.ManyToManyField` attributes and reverse

   relations can have multiple related rows, including these can have a

   multiplier effect on the size of your result set. This will be especially

   pronounced if you include multiple such fields in your ``values()`` query,

   in which case all possible combinations will be returned.

.. admonition:: Special values for ``JSONField`` on SQLite

    Due to the way the ``JSON_EXTRACT`` and ``JSON_TYPE`` SQL functions are

    implemented on SQLite, and lack of the ``BOOLEAN`` data type,

    ``values()`` will return ``True``, ``False``, and ``None`` instead of

    ``"true"``, ``"false"``, and ``"null"`` strings for

    :class:`~django.db.models.JSONField` key transforms.

``values_list()``

~~~~~~~~~~~~~~~~~

.. method:: values_list(*fields, flat=False, named=False)

This is similar to ``values()`` except that instead of returning dictionaries,

it returns tuples when iterated over. Each tuple contains the value from the

respective field or expression passed into the ``values_list()`` call — so the

first item is the first field, etc. For example::

    >>> Entry.objects.values_list('id', 'headline')

    <QuerySet [(1, 'First entry'), ...]>

    >>> from django.db.models.functions import Lower

    >>> Entry.objects.values_list('id', Lower('headline'))

    <QuerySet [(1, 'first entry'), ...]>

If you only pass in a single field, you can also pass in the ``flat``

parameter. If ``True``, this will mean the returned results are single values,

rather than one-tuples. An example should make the difference clearer::

    >>> Entry.objects.values_list('id').order_by('id')

    <QuerySet[(1,), (2,), (3,), ...]>

    >>> Entry.objects.values_list('id', flat=True).order_by('id')

    <QuerySet [1, 2, 3, ...]>

It is an error to pass in ``flat`` when there is more than one field.

You can pass ``named=True`` to get results as a

:func:`~python:collections.namedtuple`::

    >>> Entry.objects.values_list('id', 'headline', named=True)

    <QuerySet [Row(id=1, headline='First entry'), ...]>

Using a named tuple may make use of the results more readable, at the expense

of a small performance penalty for transforming the results into a named tuple.

If you don't pass any values to ``values_list()``, it will return all the

fields in the model, in the order they were declared.

A common need is to get a specific field value of a certain model instance. To

achieve that, use ``values_list()`` followed by a ``get()`` call::

    >>> Entry.objects.values_list('headline', flat=True).get(pk=1)

    'First entry'

``values()`` and ``values_list()`` are both intended as optimizations for a

specific use case: retrieving a subset of data without the overhead of creating

a model instance. This metaphor falls apart when dealing with many-to-many and

other multivalued relations (such as the one-to-many relation of a reverse

foreign key) because the "one row, one object" assumption doesn't hold.

For example, notice the behavior when querying across a

:class:`~django.db.models.ManyToManyField`::

    >>> Author.objects.values_list('name', 'entry__headline')

    <QuerySet [('Noam Chomsky', 'Impressions of Gaza'),

     ('George Orwell', 'Why Socialists Do Not Believe in Fun'),

     ('George Orwell', 'In Defence of English Cooking'),

     ('Don Quixote', None)]>

Authors with multiple entries appear multiple times and authors without any

entries have ``None`` for the entry headline.

Similarly, when querying a reverse foreign key, ``None`` appears for entries

not having any author::

    >>> Entry.objects.values_list('authors')

    <QuerySet [('Noam Chomsky',), ('George Orwell',), (None,)]>

.. admonition:: Special values for ``JSONField`` on SQLite

    Due to the way the ``JSON_EXTRACT`` and ``JSON_TYPE`` SQL functions are

    implemented on SQLite, and lack of the ``BOOLEAN`` data type,

    ``values_list()`` will return ``True``, ``False``, and ``None`` instead of

    ``"true"``, ``"false"``, and ``"null"`` strings for

    :class:`~django.db.models.JSONField` key transforms.

``dates()``

~~~~~~~~~~~

.. method:: dates(field, kind, order='ASC')

Returns a ``QuerySet`` that evaluates to a list of :class:`datetime.date`

objects representing all available dates of a particular kind within the

contents of the ``QuerySet``.

``field`` should be the name of a ``DateField`` of your model.

``kind`` should be either ``"year"``, ``"month"``, ``"week"``, or ``"day"``.

Each :class:`datetime.date` object in the result list is "truncated" to the

given ``type``.

* ``"year"`` returns a list of all distinct year values for the field.

* ``"month"`` returns a list of all distinct year/month values for the

  field.

* ``"week"`` returns a list of all distinct year/week values for the field. All

  dates will be a Monday.

* ``"day"`` returns a list of all distinct year/month/day values for the

  field.

``order``, which defaults to ``'ASC'``, should be either ``'ASC'`` or

``'DESC'``. This specifies how to order the results.

Examples::

    >>> Entry.objects.dates('pub_date', 'year')

    [datetime.date(2005, 1, 1)]

    >>> Entry.objects.dates('pub_date', 'month')

    [datetime.date(2005, 2, 1), datetime.date(2005, 3, 1)]

    >>> Entry.objects.dates('pub_date', 'week')

    [datetime.date(2005, 2, 14), datetime.date(2005, 3, 14)]

    >>> Entry.objects.dates('pub_date', 'day')

    [datetime.date(2005, 2, 20), datetime.date(2005, 3, 20)]

    >>> Entry.objects.dates('pub_date', 'day', order='DESC')

    [datetime.date(2005, 3, 20), datetime.date(2005, 2, 20)]

    >>> Entry.objects.filter(headline__contains='Lennon').dates('pub_date', 'day')

    [datetime.date(2005, 3, 20)]

``datetimes()``

~~~~~~~~~~~~~~~

.. method:: datetimes(field_name, kind, order='ASC', tzinfo=None, is_dst=None)

Returns a ``QuerySet`` that evaluates to a list of :class:`datetime.datetime`

objects representing all available dates of a particular kind within the

contents of the ``QuerySet``.

``field_name`` should be the name of a ``DateTimeField`` of your model.

``kind`` should be either ``"year"``, ``"month"``, ``"week"``, ``"day"``,

``"hour"``, ``"minute"``, or ``"second"``. Each :class:`datetime.datetime`

object in the result list is "truncated" to the given ``type``.

``order``, which defaults to ``'ASC'``, should be either ``'ASC'`` or

``'DESC'``. This specifies how to order the results.

``tzinfo`` defines the time zone to which datetimes are converted prior to

truncation. Indeed, a given datetime has different representations depending

on the time zone in use. This parameter must be a :class:`datetime.tzinfo`

object. If it's ``None``, Django uses the :ref:`current time zone

<default-current-time-zone>`. It has no effect when :setting:`USE_TZ` is

``False``.

``is_dst`` indicates whether or not ``pytz`` should interpret nonexistent and

ambiguous datetimes in daylight saving time. By default (when ``is_dst=None``),

``pytz`` raises an exception for such datetimes.

.. deprecated:: 4.0

    The ``is_dst`` parameter is deprecated and will be removed in Django 5.0.

.. _database-time-zone-definitions:

.. note::

    This function performs time zone conversions directly in the database.

    As a consequence, your database must be able to interpret the value of

    ``tzinfo.tzname(None)``. This translates into the following requirements:

    - SQLite: no requirements. Conversions are performed in Python.

    - PostgreSQL: no requirements (see `Time Zones`_).

    - Oracle: no requirements (see `Choosing a Time Zone File`_).

    - MySQL: load the time zone tables with `mysql_tzinfo_to_sql`_.

    .. _Time Zones: https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-TIMEZONES

    .. _Choosing a Time Zone File: https://docs.oracle.com/en/database/oracle/

       oracle-database/18/nlspg/datetime-data-types-and-time-zone-support.html

       #GUID-805AB986-DE12-4FEA-AF56-5AABCD2132DF

    .. _mysql_tzinfo_to_sql: https://dev.mysql.com/doc/refman/en/mysql-tzinfo-to-sql.html

``none()``

~~~~~~~~~~

.. method:: none()

Calling ``none()`` will create a queryset that never returns any objects and no

query will be executed when accessing the results. A ``qs.none()`` queryset

is an instance of ``EmptyQuerySet``.

Examples::

    >>> Entry.objects.none()

    <QuerySet []>

    >>> from django.db.models.query import EmptyQuerySet

    >>> isinstance(Entry.objects.none(), EmptyQuerySet)

    True

``all()``

~~~~~~~~~

.. method:: all()

Returns a *copy* of the current ``QuerySet`` (or ``QuerySet`` subclass).  This

can be useful in situations where you might want to pass in either a model

manager or a ``QuerySet`` and do further filtering on the result. After calling

``all()`` on either object, you'll definitely have a ``QuerySet`` to work with.

When a ``QuerySet`` is :ref:`evaluated <when-querysets-are-evaluated>`, it

typically caches its results. If the data in the database might have changed

since a ``QuerySet`` was evaluated, you can get updated results for the same

query by calling ``all()`` on a previously evaluated ``QuerySet``.

``union()``

~~~~~~~~~~~

.. method:: union(*other_qs, all=False)

Uses SQL's ``UNION`` operator to combine the results of two or more

``QuerySet``\s. For example:

    >>> qs1.union(qs2, qs3)

The ``UNION`` operator selects only distinct values by default. To allow

duplicate values, use the ``all=True`` argument.

``union()``, ``intersection()``, and ``difference()`` return model instances

of the type of the first ``QuerySet`` even if the arguments are ``QuerySet``\s

of other models. Passing different models works as long as the ``SELECT`` list

is the same in all ``QuerySet``\s (at least the types, the names don't matter

as long as the types are in the same order). In such cases, you must use the

column names from the first ``QuerySet`` in ``QuerySet`` methods applied to the

resulting ``QuerySet``. For example::

    >>> qs1 = Author.objects.values_list('name')

    >>> qs2 = Entry.objects.values_list('headline')

    >>> qs1.union(qs2).order_by('name')

In addition, only ``LIMIT``, ``OFFSET``, ``COUNT(*)``, ``ORDER BY``, and

specifying columns (i.e. slicing, :meth:`count`, :meth:`exists`,

:meth:`order_by`, and :meth:`values()`/:meth:`values_list()`) are allowed

on the resulting ``QuerySet``. Further, databases place restrictions on

what operations are allowed in the combined queries. For example, most

databases don't allow ``LIMIT`` or ``OFFSET`` in the combined queries.

``intersection()``

~~~~~~~~~~~~~~~~~~

.. method:: intersection(*other_qs)

Uses SQL's ``INTERSECT`` operator to return the shared elements of two or more

``QuerySet``\s. For example:

    >>> qs1.intersection(qs2, qs3)

See :meth:`union` for some restrictions.

``difference()``

~~~~~~~~~~~~~~~~

.. method:: difference(*other_qs)

Uses SQL's ``EXCEPT`` operator to keep only elements present in the

``QuerySet`` but not in some other ``QuerySet``\s. For example::

    >>> qs1.difference(qs2, qs3)

See :meth:`union` for some restrictions.

``select_related()``

~~~~~~~~~~~~~~~~~~~~

.. method:: select_related(*fields)

Returns a ``QuerySet`` that will "follow" foreign-key relationships, selecting

additional related-object data when it executes its query. This is a

performance booster which results in a single more complex query but means

later use of foreign-key relationships won't require database queries.

The following examples illustrate the difference between plain lookups and

``select_related()`` lookups. Here's standard lookup::

    # Hits the database.

    e = Entry.objects.get(id=5)

    # Hits the database again to get the related Blog object.

    b = e.blog

And here's ``select_related`` lookup::

    # Hits the database.

    e = Entry.objects.select_related('blog').get(id=5)

    # Doesn't hit the database, because e.blog has been prepopulated

    # in the previous query.

    b = e.blog

You can use ``select_related()`` with any queryset of objects::

    from django.utils import timezone

    # Find all the blogs with entries scheduled to be published in the future.

    blogs = set()

    for e in Entry.objects.filter(pub_date__gt=timezone.now()).select_related('blog'):

        # Without select_related(), this would make a database query for each

        # loop iteration in order to fetch the related blog for each entry.

        blogs.add(e.blog)

The order of ``filter()`` and ``select_related()`` chaining isn't important.

These querysets are equivalent::

    Entry.objects.filter(pub_date__gt=timezone.now()).select_related('blog')

    Entry.objects.select_related('blog').filter(pub_date__gt=timezone.now())

You can follow foreign keys in a similar way to querying them. If you have the

following models::

    from django.db import models

    class City(models.Model):

        # ...

        pass

    class Person(models.Model):

        # ...

        hometown = models.ForeignKey(

            City,

            on_delete=models.SET_NULL,

            blank=True,

            null=True,

        )

    class Book(models.Model):

        # ...

        author = models.ForeignKey(Person, on_delete=models.CASCADE)

... then a call to ``Book.objects.select_related('author__hometown').get(id=4)``

will cache the related ``Person`` *and* the related ``City``::

    # Hits the database with joins to the author and hometown tables.

    b = Book.objects.select_related('author__hometown').get(id=4)

    p = b.author         # Doesn't hit the database.

    c = p.hometown       # Doesn't hit the database.

    # Without select_related()...

    b = Book.objects.get(id=4)  # Hits the database.

    p = b.author         # Hits the database.

    c = p.hometown       # Hits the database.

You can refer to any :class:`~django.db.models.ForeignKey` or

:class:`~django.db.models.OneToOneField` relation in the list of fields

passed to ``select_related()``.

You can also refer to the reverse direction of a

:class:`~django.db.models.OneToOneField` in the list of fields passed to

``select_related`` — that is, you can traverse a

:class:`~django.db.models.OneToOneField` back to the object on which the field

is defined. Instead of specifying the field name, use the :attr:`related_name

<django.db.models.ForeignKey.related_name>` for the field on the related object.

There may be some situations where you wish to call ``select_related()`` with a

lot of related objects, or where you don't know all of the relations. In these

cases it is possible to call ``select_related()`` with no arguments. This will

follow all non-null foreign keys it can find - nullable foreign keys must be

specified. This is not recommended in most cases as it is likely to make the

underlying query more complex, and return more data, than is actually needed.

If you need to clear the list of related fields added by past calls of

``select_related`` on a ``QuerySet``, you can pass ``None`` as a parameter::

   >>> without_relations = queryset.select_related(None)

Chaining ``select_related`` calls works in a similar way to other methods -

that is that ``select_related('foo', 'bar')`` is equivalent to

``select_related('foo').select_related('bar')``.

``prefetch_related()``

~~~~~~~~~~~~~~~~~~~~~~

.. method:: prefetch_related(*lookups)

Returns a ``QuerySet`` that will automatically retrieve, in a single batch,

related objects for each of the specified lookups.

This has a similar purpose to ``select_related``, in that both are designed to

stop the deluge of database queries that is caused by accessing related objects,

but the strategy is quite different.

``select_related`` works by creating an SQL join and including the fields of the

related object in the ``SELECT`` statement. For this reason, ``select_related``

gets the related objects in the same database query. However, to avoid the much

larger result set that would result from joining across a 'many' relationship,

``select_related`` is limited to single-valued relationships - foreign key and

one-to-one.

``prefetch_related``, on the other hand, does a separate lookup for each

relationship, and does the 'joining' in Python. This allows it to prefetch

many-to-many and many-to-one objects, which cannot be done using

``select_related``, in addition to the foreign key and one-to-one relationships

that are supported by ``select_related``. It also supports prefetching of

:class:`~django.contrib.contenttypes.fields.GenericRelation` and

:class:`~django.contrib.contenttypes.fields.GenericForeignKey`, however, it

must be restricted to a homogeneous set of results. For example, prefetching

objects referenced by a ``GenericForeignKey`` is only supported if the query

is restricted to one ``ContentType``.

For example, suppose you have these models::

    from django.db import models

    class Topping(models.Model):

        name = models.CharField(max_length=30)

    class Pizza(models.Model):

        name = models.CharField(max_length=50)

        toppings = models.ManyToManyField(Topping)

        def __str__(self):

            return "%s (%s)" % (

                self.name,

                ", ".join(topping.name for topping in self.toppings.all()),

            )

and run::

    >>> Pizza.objects.all()

    ["Hawaiian (ham, pineapple)", "Seafood (prawns, smoked salmon)"...

The problem with this is that every time ``Pizza.__str__()`` asks for

``self.toppings.all()`` it has to query the database, so

``Pizza.objects.all()`` will run a query on the Toppings table for **every**

item in the Pizza ``QuerySet``.

We can reduce to just two queries using ``prefetch_related``:

    >>> Pizza.objects.prefetch_related('toppings')

This implies a ``self.toppings.all()`` for each ``Pizza``; now each time

``self.toppings.all()`` is called, instead of having to go to the database for

the items, it will find them in a prefetched ``QuerySet`` cache that was

populated in a single query.

That is, all the relevant toppings will have been fetched in a single query,

and used to make ``QuerySets`` that have a pre-filled cache of the relevant

results; these ``QuerySets`` are then used in the ``self.toppings.all()`` calls.

The additional queries in ``prefetch_related()`` are executed after the

``QuerySet`` has begun to be evaluated and the primary query has been executed.

If you have an iterable of model instances, you can prefetch related attributes

on those instances using the :func:`~django.db.models.prefetch_related_objects`

function.

Note that the result cache of the primary ``QuerySet`` and all specified related

objects will then be fully loaded into memory. This changes the typical

behavior of ``QuerySets``, which normally try to avoid loading all objects into

memory before they are needed, even after a query has been executed in the

database.

.. note::

    Remember that, as always with ``QuerySets``, any subsequent chained methods

    which imply a different database query will ignore previously cached

    results, and retrieve data using a fresh database query. So, if you write

    the following:

        >>> pizzas = Pizza.objects.prefetch_related('toppings')

        >>> [list(pizza.toppings.filter(spicy=True)) for pizza in pizzas]

    ...then the fact that ``pizza.toppings.all()`` has been prefetched will not

    help you. The ``prefetch_related('toppings')`` implied

    ``pizza.toppings.all()``, but ``pizza.toppings.filter()`` is a new and

    different query. The prefetched cache can't help here; in fact it hurts

    performance, since you have done a database query that you haven't used. So

    use this feature with caution!

    Also, if you call the database-altering methods

    :meth:`~django.db.models.fields.related.RelatedManager.add`,

    :meth:`~django.db.models.fields.related.RelatedManager.remove`,

    :meth:`~django.db.models.fields.related.RelatedManager.clear` or

    :meth:`~django.db.models.fields.related.RelatedManager.set`, on

    :class:`related managers<django.db.models.fields.related.RelatedManager>`,

    any prefetched cache for the relation will be cleared.

You can also use the normal join syntax to do related fields of related

fields. Suppose we have an additional model to the example above::

    class Restaurant(models.Model):

        pizzas = models.ManyToManyField(Pizza, related_name='restaurants')

        best_pizza = models.ForeignKey(Pizza, related_name='championed_by', on_delete=models.CASCADE)

The following are all legal:

    >>> Restaurant.objects.prefetch_related('pizzas__toppings')

This will prefetch all pizzas belonging to restaurants, and all toppings

belonging to those pizzas. This will result in a total of 3 database queries -

one for the restaurants, one for the pizzas, and one for the toppings.

    >>> Restaurant.objects.prefetch_related('best_pizza__toppings')

This will fetch the best pizza and all the toppings for the best pizza for each

restaurant. This will be done in 3 database queries - one for the restaurants,

one for the 'best pizzas', and one for the toppings.

The ``best_pizza`` relationship could also be fetched using ``select_related``

to reduce the query count to 2::

    >>> Restaurant.objects.select_related('best_pizza').prefetch_related('best_pizza__toppings')

Since the prefetch is executed after the main query (which includes the joins

needed by ``select_related``), it is able to detect that the ``best_pizza``

objects have already been fetched, and it will skip fetching them again.

Chaining ``prefetch_related`` calls will accumulate the lookups that are

prefetched. To clear any ``prefetch_related`` behavior, pass ``None`` as a

parameter:

   >>> non_prefetched = qs.prefetch_related(None)

One difference to note when using ``prefetch_related`` is that objects created

by a query can be shared between the different objects that they are related to

i.e. a single Python model instance can appear at more than one point in the

tree of objects that are returned. This will normally happen with foreign key

relationships. Typically this behavior will not be a problem, and will in fact

save both memory and CPU time.

While ``prefetch_related`` supports prefetching ``GenericForeignKey``

relationships, the number of queries will depend on the data. Since a

``GenericForeignKey`` can reference data in multiple tables, one query per table

referenced is needed, rather than one query for all the items. There could be

additional queries on the ``ContentType`` table if the relevant rows have not

already been fetched.

``prefetch_related`` in most cases will be implemented using an SQL query that

uses the 'IN' operator. This means that for a large ``QuerySet`` a large 'IN' clause

could be generated, which, depending on the database, might have performance

problems of its own when it comes to parsing or executing the SQL query. Always

profile for your use case!

.. versionchanged:: 4.1

    If you use ``iterator()`` to run the query, ``prefetch_related()``

    calls will only be observed if a value for ``chunk_size`` is provided.

You can use the :class:`~django.db.models.Prefetch` object to further control

the prefetch operation.

In its simplest form ``Prefetch`` is equivalent to the traditional string based

lookups:

    >>> from django.db.models import Prefetch

    >>> Restaurant.objects.prefetch_related(Prefetch('pizzas__toppings'))

You can provide a custom queryset with the optional ``queryset`` argument.

This can be used to change the default ordering of the queryset:

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas__toppings', queryset=Toppings.objects.order_by('name')))

Or to call :meth:`~django.db.models.query.QuerySet.select_related()` when

applicable to reduce the number of queries even further:

    >>> Pizza.objects.prefetch_related(

    ...     Prefetch('restaurants', queryset=Restaurant.objects.select_related('best_pizza')))

You can also assign the prefetched result to a custom attribute with the optional

``to_attr`` argument. The result will be stored directly in a list.

This allows prefetching the same relation multiple times with a different

``QuerySet``; for instance:

    >>> vegetarian_pizzas = Pizza.objects.filter(vegetarian=True)

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas', to_attr='menu'),

    ...     Prefetch('pizzas', queryset=vegetarian_pizzas, to_attr='vegetarian_menu'))

Lookups created with custom ``to_attr`` can still be traversed as usual by other

lookups:

    >>> vegetarian_pizzas = Pizza.objects.filter(vegetarian=True)

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas', queryset=vegetarian_pizzas, to_attr='vegetarian_menu'),

    ...     'vegetarian_menu__toppings')

Using ``to_attr`` is recommended when filtering down the prefetch result as it is

less ambiguous than storing a filtered result in the related manager's cache:

    >>> queryset = Pizza.objects.filter(vegetarian=True)

    >>>

    >>> # Recommended:

    >>> restaurants = Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas', queryset=queryset, to_attr='vegetarian_pizzas'))

    >>> vegetarian_pizzas = restaurants[0].vegetarian_pizzas

    >>>

    >>> # Not recommended:

    >>> restaurants = Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas', queryset=queryset))

    >>> vegetarian_pizzas = restaurants[0].pizzas.all()

Custom prefetching also works with single related relations like

forward ``ForeignKey`` or ``OneToOneField``. Generally you'll want to use

:meth:`select_related()` for these relations, but there are a number of cases

where prefetching with a custom ``QuerySet`` is useful:

* You want to use a ``QuerySet`` that performs further prefetching

  on related models.

* You want to prefetch only a subset of the related objects.

* You want to use performance optimization techniques like

  :meth:`deferred fields <defer()>`:

    >>> queryset = Pizza.objects.only('name')

    >>>

    >>> restaurants = Restaurant.objects.prefetch_related(

    ...     Prefetch('best_pizza', queryset=queryset))

When using multiple databases, ``Prefetch`` will respect your choice of

database. If the inner query does not specify a database, it will use the

database selected by the outer query. All of the following are valid::

    >>> # Both inner and outer queries will use the 'replica' database

    >>> Restaurant.objects.prefetch_related('pizzas__toppings').using('replica')

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas__toppings'),

    ... ).using('replica')

    >>>

    >>> # Inner will use the 'replica' database; outer will use 'default' database

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas__toppings', queryset=Toppings.objects.using('replica')),

    ... )

    >>>

    >>> # Inner will use 'replica' database; outer will use 'cold-storage' database

    >>> Restaurant.objects.prefetch_related(

    ...     Prefetch('pizzas__toppings', queryset=Toppings.objects.using('replica')),

    ... ).using('cold-storage')

.. note::

    The ordering of lookups matters.

    Take the following examples:

       >>> prefetch_related('pizzas__toppings', 'pizzas')

    This works even though it's unordered because ``'pizzas__toppings'``

    already contains all the needed information, therefore the second argument

    ``'pizzas'`` is actually redundant.

        >>> prefetch_related('pizzas__toppings', Prefetch('pizzas', queryset=Pizza.objects.all()))

    This will raise a ``ValueError`` because of the attempt to redefine the

    queryset of a previously seen lookup. Note that an implicit queryset was

    created to traverse ``'pizzas'`` as part of the ``'pizzas__toppings'``

    lookup.

        >>> prefetch_related('pizza_list__toppings', Prefetch('pizzas', to_attr='pizza_list'))

    This will trigger an ``AttributeError`` because ``'pizza_list'`` doesn't exist yet

    when ``'pizza_list__toppings'`` is being processed.

    This consideration is not limited to the use of ``Prefetch`` objects. Some

    advanced techniques may require that the lookups be performed in a

    specific order to avoid creating extra queries; therefore it's recommended

    to always carefully order ``prefetch_related`` arguments.

``extra()``

~~~~~~~~~~~

.. method:: extra(select=None, where=None, params=None, tables=None, order_by=None, select_params=None)

Sometimes, the Django query syntax by itself can't easily express a complex

``WHERE`` clause. For these edge cases, Django provides the ``extra()``

``QuerySet`` modifier — a hook for injecting specific clauses into the SQL

generated by a ``QuerySet``.

.. admonition:: Use this method as a last resort

    This is an old API that we aim to deprecate at some point in the future.

    Use it only if you cannot express your query using other queryset methods.

    If you do need to use it, please `file a ticket

    <https://code.djangoproject.com/newticket>`_ using the `QuerySet.extra

    keyword <https://code.djangoproject.com/query?status=assigned&status=new&keywords=~QuerySet.extra>`_

    with your use case (please check the list of existing tickets first) so

    that we can enhance the QuerySet API to allow removing ``extra()``. We are

    no longer improving or fixing bugs for this method.

    For example, this use of ``extra()``::

        >>> qs.extra(

        ...     select={'val': "select col from sometable where othercol = %s"},

        ...     select_params=(someparam,),

        ... )

    is equivalent to::

        >>> qs.annotate(val=RawSQL("select col from sometable where othercol = %s", (someparam,)))

    The main benefit of using :class:`~django.db.models.expressions.RawSQL` is

    that you can set ``output_field`` if needed. The main downside is that if

    you refer to some table alias of the queryset in the raw SQL, then it is

    possible that Django might change that alias (for example, when the

    queryset is used as a subquery in yet another query).

.. warning::

    You should be very careful whenever you use ``extra()``. Every time you use

    it, you should escape any parameters that the user can control by using

    ``params`` in order to protect against SQL injection attacks.

    You also must not quote placeholders in the SQL string. This example is

    vulnerable to SQL injection because of the quotes around ``%s``:

    .. code-block:: sql

        SELECT col FROM sometable WHERE othercol = '%s'  # unsafe!

    You can read more about how Django's :ref:`SQL injection protection

    <sql-injection-protection>` works.

By definition, these extra lookups may not be portable to different database

engines (because you're explicitly writing SQL code) and violate the DRY

principle, so you should avoid them if possible.

Specify one or more of ``params``, ``select``, ``where`` or ``tables``. None

of the arguments is required, but you should use at least one of them.

* ``select``

  The ``select`` argument lets you put extra fields in the ``SELECT``

  clause.  It should be a dictionary mapping attribute names to SQL

  clauses to use to calculate that attribute.

  Example::

      Entry.objects.extra(select={'is_recent': "pub_date > '2006-01-01'"})

  As a result, each ``Entry`` object will have an extra attribute,

  ``is_recent``, a boolean representing whether the entry's ``pub_date``

  is greater than Jan. 1, 2006.

  Django inserts the given SQL snippet directly into the ``SELECT``

  statement, so the resulting SQL of the above example would be something like:

  .. code-block:: sql

      SELECT blog_entry.*, (pub_date > '2006-01-01') AS is_recent

      FROM blog_entry;

  The next example is more advanced; it does a subquery to give each

  resulting ``Blog`` object an ``entry_count`` attribute, an integer count

  of associated ``Entry`` objects::

      Blog.objects.extra(

          select={

              'entry_count': 'SELECT COUNT(*) FROM blog_entry WHERE blog_entry.blog_id = blog_blog.id'

          },

      )

  In this particular case, we're exploiting the fact that the query will

  already contain the ``blog_blog`` table in its ``FROM`` clause.

  The resulting SQL of the above example would be:

  .. code-block:: sql

      SELECT blog_blog.*, (SELECT COUNT(*) FROM blog_entry WHERE blog_entry.blog_id = blog_blog.id) AS entry_count

      FROM blog_blog;

  Note that the parentheses required by most database engines around

  subqueries are not required in Django's ``select`` clauses. Also note

  that some database backends, such as some MySQL versions, don't support

  subqueries.

  In some rare cases, you might wish to pass parameters to the SQL

  fragments in ``extra(select=...)``. For this purpose, use the

  ``select_params`` parameter.

  This will work, for example::

      Blog.objects.extra(

          select={'a': '%s', 'b': '%s'},

          select_params=('one', 'two'),

      )

  If you need to use a literal ``%s`` inside your select string, use

  the sequence ``%%s``.

* ``where`` / ``tables``

  You can define explicit SQL ``WHERE`` clauses — perhaps to perform

  non-explicit joins — by using ``where``. You can manually add tables to

  the SQL ``FROM`` clause by using ``tables``.

  ``where`` and ``tables`` both take a list of strings. All ``where``

  parameters are "AND"ed to any other search criteria.

  Example::

      Entry.objects.extra(where=["foo='a' OR bar = 'a'", "baz = 'a'"])

  ...translates (roughly) into the following SQL:

  .. code-block:: sql

      SELECT * FROM blog_entry WHERE (foo='a' OR bar='a') AND (baz='a')

  Be careful when using the ``tables`` parameter if you're specifying

  tables that are already used in the query. When you add extra tables

  via the ``tables`` parameter, Django assumes you want that table

  included an extra time, if it is already included. That creates a

  problem, since the table name will then be given an alias. If a table

  appears multiple times in an SQL statement, the second and subsequent

  occurrences must use aliases so the database can tell them apart. If

  you're referring to the extra table you added in the extra ``where``

  parameter this is going to cause errors.

  Normally you'll only be adding extra tables that don't already appear

  in the query. However, if the case outlined above does occur, there are

  a few solutions. First, see if you can get by without including the

  extra table and use the one already in the query. If that isn't

  possible, put your ``extra()`` call at the front of the queryset

  construction so that your table is the first use of that table.

  Finally, if all else fails, look at the query produced and rewrite your

  ``where`` addition to use the alias given to your extra table. The

  alias will be the same each time you construct the queryset in the same

  way, so you can rely upon the alias name to not change.

* ``order_by``

  If you need to order the resulting queryset using some of the new

  fields or tables you have included via ``extra()`` use the ``order_by``

  parameter to ``extra()`` and pass in a sequence of strings. These

  strings should either be model fields (as in the normal

  :meth:`order_by()` method on querysets), of the form

  ``table_name.column_name`` or an alias for a column that you specified

  in the ``select`` parameter to ``extra()``.

  For example::

      q = Entry.objects.extra(select={'is_recent': "pub_date > '2006-01-01'"})

      q = q.extra(order_by = ['-is_recent'])

  This would sort all the items for which ``is_recent`` is true to the

  front of the result set (``True`` sorts before ``False`` in a

  descending ordering).

  This shows, by the way, that you can make multiple calls to ``extra()``

  and it will behave as you expect (adding new constraints each time).

* ``params``

  The ``where`` parameter described above may use standard Python

  database string placeholders — ``'%s'`` to indicate parameters the

  database engine should automatically quote. The ``params`` argument is

  a list of any extra parameters to be substituted.

  Example::

      Entry.objects.extra(where=['headline=%s'], params=['Lennon'])

  Always use ``params`` instead of embedding values directly into

  ``where`` because ``params`` will ensure values are quoted correctly

  according to your particular backend. For example, quotes will be

  escaped correctly.

  Bad::

      Entry.objects.extra(where=["headline='Lennon'"])

  Good::

      Entry.objects.extra(where=['headline=%s'], params=['Lennon'])

.. warning::

    If you are performing queries on MySQL, note that MySQL's silent type coercion

    may cause unexpected results when mixing types. If you query on a string

    type column, but with an integer value, MySQL will coerce the types of all values

    in the table to an integer before performing the comparison. For example, if your

    table contains the values ``'abc'``, ``'def'`` and you query for ``WHERE mycolumn=0``,

    both rows will match. To prevent this, perform the correct typecasting

    before using the value in a query.

``defer()``

~~~~~~~~~~~

.. method:: defer(*fields)

In some complex data-modeling situations, your models might contain a lot of

fields, some of which could contain a lot of data (for example, text fields),

or require expensive processing to convert them to Python objects. If you are

using the results of a queryset in some situation where you don't know

if you need those particular fields when you initially fetch the data, you can

tell Django not to retrieve them from the database.

This is done by passing the names of the fields to not load to ``defer()``::

    Entry.objects.defer("headline", "body")

A queryset that has deferred fields will still return model instances. Each

deferred field will be retrieved from the database if you access that field

(one at a time, not all the deferred fields at once).

.. note::

    Deferred fields will not lazy-load like this from asynchronous code.

    Instead, you will get a ``SynchronousOnlyOperation`` exception. If you are

    writing asynchronous code, you should not try to access any fields that you

    ``defer()``.

You can make multiple calls to ``defer()``. Each call adds new fields to the

deferred set::

    # Defers both the body and headline fields.

    Entry.objects.defer("body").filter(rating=5).defer("headline")

The order in which fields are added to the deferred set does not matter.

Calling ``defer()`` with a field name that has already been deferred is

harmless (the field will still be deferred).

You can defer loading of fields in related models (if the related models are

loading via :meth:`select_related()`) by using the standard double-underscore

notation to separate related fields::

    Blog.objects.select_related().defer("entry__headline", "entry__body")

If you want to clear the set of deferred fields, pass ``None`` as a parameter

to ``defer()``::

    # Load all fields immediately.

    my_queryset.defer(None)

Some fields in a model won't be deferred, even if you ask for them. You can

never defer the loading of the primary key. If you are using

:meth:`select_related()` to retrieve related models, you shouldn't defer the

loading of the field that connects from the primary model to the related

one, doing so will result in an error.

.. note::

    The ``defer()`` method (and its cousin, :meth:`only()`, below) are only for

    advanced use-cases. They provide an optimization for when you have analyzed

    your queries closely and understand *exactly* what information you need and

    have measured that the difference between returning the fields you need and

    the full set of fields for the model will be significant.

    Even if you think you are in the advanced use-case situation, **only use**

    ``defer()`` **when you cannot, at queryset load time, determine if you will

    need the extra fields or not**. If you are frequently loading and using a

    particular subset of your data, the best choice you can make is to

    normalize your models and put the non-loaded data into a separate model

    (and database table). If the columns *must* stay in the one table for some

    reason, create a model with ``Meta.managed = False`` (see the

    :attr:`managed attribute <django.db.models.Options.managed>` documentation)

    containing just the fields you normally need to load and use that where you

    might otherwise call ``defer()``. This makes your code more explicit to the

    reader, is slightly faster and consumes a little less memory in the Python

    process.

    For example, both of these models use the same underlying database table::

        class CommonlyUsedModel(models.Model):

            f1 = models.CharField(max_length=10)

            class Meta:

                managed = False

                db_table = 'app_largetable'

        class ManagedModel(models.Model):

            f1 = models.CharField(max_length=10)

            f2 = models.CharField(max_length=10)

            class Meta:

                db_table = 'app_largetable'

        # Two equivalent QuerySets:

        CommonlyUsedModel.objects.all()

        ManagedModel.objects.defer('f2')

    If many fields need to be duplicated in the unmanaged model, it may be best

    to create an abstract model with the shared fields and then have the

    unmanaged and managed models inherit from the abstract model.

.. note::

    When calling :meth:`~django.db.models.Model.save()` for instances with

    deferred fields, only the loaded fields will be saved. See

    :meth:`~django.db.models.Model.save()` for more details.

``only()``

~~~~~~~~~~

.. method:: only(*fields)

The ``only()`` method is essentially the opposite of :meth:`defer`. Only the

fields passed into this method and that are *not* already specified as deferred

are loaded immediately when the queryset is evaluated.

If you have a model where almost all the fields need to be deferred, using

``only()`` to specify the complementary set of fields can result in simpler

code.

Suppose you have a model with fields ``name``, ``age`` and ``biography``. The

following two querysets are the same, in terms of deferred fields::

    Person.objects.defer("age", "biography")

    Person.objects.only("name")

Whenever you call ``only()`` it *replaces* the set of fields to load

immediately. The method's name is mnemonic: **only** those fields are loaded

immediately; the remainder are deferred. Thus, successive calls to ``only()``

result in only the final fields being considered::

    # This will defer all fields except the headline.

    Entry.objects.only("body", "rating").only("headline")

Since ``defer()`` acts incrementally (adding fields to the deferred list), you

can combine calls to ``only()`` and ``defer()`` and things will behave

logically::

    # Final result is that everything except "headline" is deferred.

    Entry.objects.only("headline", "body").defer("body")

    # Final result loads headline immediately.

    Entry.objects.defer("body").only("headline", "body")

All of the cautions in the note for the :meth:`defer` documentation apply to

``only()`` as well. Use it cautiously and only after exhausting your other

options.

Using :meth:`only` and omitting a field requested using :meth:`select_related`

is an error as well.

As with ``defer()``, you cannot access the non-loaded fields from asynchronous

code and expect them to load. Instead, you will get a

``SynchronousOnlyOperation`` exception. Ensure that all fields you might access

are in your ``only()`` call.

.. note::

    When calling :meth:`~django.db.models.Model.save()` for instances with

    deferred fields, only the loaded fields will be saved. See

    :meth:`~django.db.models.Model.save()` for more details.

.. note::

    When using :meth:`defer` after ``only()`` the fields in :meth:`defer` will

    override ``only()`` for fields that are listed in both.

``using()``

~~~~~~~~~~~

.. method:: using(alias)

This method is for controlling which database the ``QuerySet`` will be

evaluated against if you are using more than one database.  The only argument

this method takes is the alias of a database, as defined in

:setting:`DATABASES`.

For example::

    # queries the database with the 'default' alias.

    >>> Entry.objects.all()

    # queries the database with the 'backup' alias

    >>> Entry.objects.using('backup')

``select_for_update()``

~~~~~~~~~~~~~~~~~~~~~~~

.. method:: select_for_update(nowait=False, skip_locked=False, of=(), no_key=False)

Returns a queryset that will lock rows until the end of the transaction,

generating a ``SELECT ... FOR UPDATE`` SQL statement on supported databases.

For example::

    from django.db import transaction

    entries = Entry.objects.select_for_update().filter(author=request.user)

    with transaction.atomic():

        for entry in entries:

            ...

When the queryset is evaluated (``for entry in entries`` in this case), all

matched entries will be locked until the end of the transaction block, meaning

that other transactions will be prevented from changing or acquiring locks on

them.

Usually, if another transaction has already acquired a lock on one of the

selected rows, the query will block until the lock is released. If this is

not the behavior you want, call ``select_for_update(nowait=True)``. This will

make the call non-blocking. If a conflicting lock is already acquired by

another transaction, :exc:`~django.db.DatabaseError` will be raised when the

queryset is evaluated. You can also ignore locked rows by using

``select_for_update(skip_locked=True)`` instead. The ``nowait`` and

``skip_locked`` are mutually exclusive and attempts to call

``select_for_update()`` with both options enabled will result in a

:exc:`ValueError`.

By default, ``select_for_update()`` locks all rows that are selected by the

query. For example, rows of related objects specified in :meth:`select_related`

are locked in addition to rows of the queryset's model. If this isn't desired,

specify the related objects you want to lock in ``select_for_update(of=(...))``

using the same fields syntax as :meth:`select_related`. Use the value ``'self'``

to refer to the queryset's model.

.. admonition:: Lock parents models in ``select_for_update(of=(...))``

    If you want to lock parents models when using :ref:`multi-table inheritance

    <multi-table-inheritance>`, you must specify parent link fields (by default

    ``<parent_model_name>_ptr``) in the ``of`` argument. For example::

        Restaurant.objects.select_for_update(of=('self', 'place_ptr'))

.. admonition:: Using ``select_for_update(of=(...))`` with specified fields

    If you want to lock models and specify selected fields, e.g. using

    :meth:`values`, you must select at least one field from each model in the

    ``of`` argument. Models without selected fields will not be locked.

On PostgreSQL only, you can pass ``no_key=True`` in order to acquire a weaker

lock, that still allows creating rows that merely reference locked rows

(through a foreign key, for example) while the lock is in place. The

PostgreSQL documentation has more details about `row-level lock modes

<https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-ROWS>`_.

You can't use ``select_for_update()`` on nullable relations::

    >>> Person.objects.select_related('hometown').select_for_update()

    Traceback (most recent call last):

    ...

    django.db.utils.NotSupportedError: FOR UPDATE cannot be applied to the nullable side of an outer join

To avoid that restriction, you can exclude null objects if you don't care about

them::

    >>> Person.objects.select_related('hometown').select_for_update().exclude(hometown=None)

    <QuerySet [<Person: ...)>, ...]>

The ``postgresql``, ``oracle``, and ``mysql`` database backends support

``select_for_update()``. However, MariaDB only supports the ``nowait``

argument, MariaDB 10.6+ also supports the ``skip_locked`` argument, and MySQL

8.0.1+ supports the ``nowait``, ``skip_locked``, and ``of`` arguments. The

``no_key`` argument is only supported on PostgreSQL.

Passing ``nowait=True``, ``skip_locked=True``, ``no_key=True``, or ``of`` to

``select_for_update()`` using database backends that do not support these

options, such as MySQL, raises a :exc:`~django.db.NotSupportedError`. This

prevents code from unexpectedly blocking.

Evaluating a queryset with ``select_for_update()`` in autocommit mode on

backends which support ``SELECT ... FOR UPDATE`` is a

:exc:`~django.db.transaction.TransactionManagementError` error because the

rows are not locked in that case. If allowed, this would facilitate data

corruption and could easily be caused by calling code that expects to be run in

a transaction outside of one.

Using ``select_for_update()`` on backends which do not support

``SELECT ... FOR UPDATE`` (such as SQLite) will have no effect.

``SELECT ... FOR UPDATE`` will not be added to the query, and an error isn't

raised if ``select_for_update()`` is used in autocommit mode.

.. warning::

    Although ``select_for_update()`` normally fails in autocommit mode, since

    :class:`~django.test.TestCase` automatically wraps each test in a

    transaction, calling ``select_for_update()`` in a ``TestCase`` even outside

    an :func:`~django.db.transaction.atomic()` block will (perhaps unexpectedly)

    pass without raising a ``TransactionManagementError``. To properly test

    ``select_for_update()`` you should use

    :class:`~django.test.TransactionTestCase`.

.. admonition:: Certain expressions may not be supported

    PostgreSQL doesn't support ``select_for_update()`` with

    :class:`~django.db.models.expressions.Window` expressions.

.. versionchanged:: 4.0

    The ``skip_locked`` argument was allowed on MariaDB 10.6+.

``raw()``

~~~~~~~~~

.. method:: raw(raw_query, params=(), translations=None, using=None)

Takes a raw SQL query, executes it, and returns a

``django.db.models.query.RawQuerySet`` instance. This ``RawQuerySet`` instance

can be iterated over just like a normal ``QuerySet`` to provide object

instances.

See the :doc:`/topics/db/sql` for more information.

.. warning::

  ``raw()`` always triggers a new query and doesn't account for previous

  filtering. As such, it should generally be called from the ``Manager`` or

  from a fresh ``QuerySet`` instance.

Operators that return new ``QuerySet``\s

----------------------------------------

Combined querysets must use the same model.

AND (``&``)

~~~~~~~~~~~

Combines two ``QuerySet``\s using the SQL ``AND`` operator.

The following are equivalent::

    Model.objects.filter(x=1) & Model.objects.filter(y=2)

    Model.objects.filter(x=1, y=2)

    from django.db.models import Q

    Model.objects.filter(Q(x=1) & Q(y=2))

SQL equivalent:

.. code-block:: sql

    SELECT ... WHERE x=1 AND y=2

OR (``|``)

~~~~~~~~~~

Combines two ``QuerySet``\s using the SQL ``OR`` operator.

The following are equivalent::

    Model.objects.filter(x=1) | Model.objects.filter(y=2)

    from django.db.models import Q

    Model.objects.filter(Q(x=1) | Q(y=2))

SQL equivalent:

.. code-block:: sql

    SELECT ... WHERE x=1 OR y=2

``|`` is not a commutative operation, as different (though equivalent) queries

may be generated.

XOR (``^``)

~~~~~~~~~~~

.. versionadded:: 4.1

Combines two ``QuerySet``\s using the SQL ``XOR`` operator.

The following are equivalent::

    Model.objects.filter(x=1) ^ Model.objects.filter(y=2)

    from django.db.models import Q

    Model.objects.filter(Q(x=1) ^ Q(y=2))

SQL equivalent:

.. code-block:: sql

    SELECT ... WHERE x=1 XOR y=2

.. note::

    ``XOR`` is natively supported on MariaDB and MySQL. On other databases,

    ``x ^ y ^ ... ^ z`` is converted to an equivalent:

    .. code-block:: sql

        (x OR y OR ... OR z) AND

        1=(

            (CASE WHEN x THEN 1 ELSE 0 END) +

            (CASE WHEN y THEN 1 ELSE 0 END) +

            ...

            (CASE WHEN z THEN 1 ELSE 0 END) +

        )

Methods that do not return ``QuerySet``\s

-----------------------------------------

The following ``QuerySet`` methods evaluate the ``QuerySet`` and return