Skip to content

Commit 2f172d8

Browse files
authored
bpo-17005: Move topological sort functionality to its own module (GH-20558)
The topological sort functionality that was introduced initially in the functools module has been moved to a new graphlib module to better accommodate the new tools and keep the original scope of the functools module.
1 parent 491a3d3 commit 2f172d8

File tree

10 files changed

+714
-717
lines changed

10 files changed

+714
-717
lines changed

Doc/library/datatypes.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,4 @@ The following modules are documented in this chapter:
3333
pprint.rst
3434
reprlib.rst
3535
enum.rst
36+
graphlib.rst

Doc/library/functools.rst

Lines changed: 1 addition & 195 deletions
Original file line numberDiff line numberDiff line change
@@ -543,184 +543,6 @@ The :mod:`functools` module defines the following functions:
543543
.. versionadded:: 3.8
544544

545545

546-
.. class:: TopologicalSorter(graph=None)
547-
548-
Provides functionality to topologically sort a graph of hashable nodes.
549-
550-
A topological order is a linear ordering of the vertices in a graph such that
551-
for every directed edge u -> v from vertex u to vertex v, vertex u comes
552-
before vertex v in the ordering. For instance, the vertices of the graph may
553-
represent tasks to be performed, and the edges may represent constraints that
554-
one task must be performed before another; in this example, a topological
555-
ordering is just a valid sequence for the tasks. A complete topological
556-
ordering is possible if and only if the graph has no directed cycles, that
557-
is, if it is a directed acyclic graph.
558-
559-
If the optional *graph* argument is provided it must be a dictionary
560-
representing a directed acyclic graph where the keys are nodes and the values
561-
are iterables of all predecessors of that node in the graph (the nodes that
562-
have edges that point to the value in the key). Additional nodes can be added
563-
to the graph using the :meth:`~TopologicalSorter.add` method.
564-
565-
In the general case, the steps required to perform the sorting of a given
566-
graph are as follows:
567-
568-
* Create an instance of the :class:`TopologicalSorter` with an optional
569-
initial graph.
570-
* Add additional nodes to the graph.
571-
* Call :meth:`~TopologicalSorter.prepare` on the graph.
572-
* While :meth:`~TopologicalSorter.is_active` is ``True``, iterate over
573-
the nodes returned by :meth:`~TopologicalSorter.get_ready` and
574-
process them. Call :meth:`~TopologicalSorter.done` on each node as it
575-
finishes processing.
576-
577-
In case just an immediate sorting of the nodes in the graph is required and
578-
no parallelism is involved, the convenience method
579-
:meth:`TopologicalSorter.static_order` can be used directly:
580-
581-
.. doctest::
582-
583-
>>> graph = {"D": {"B", "C"}, "C": {"A"}, "B": {"A"}}
584-
>>> ts = TopologicalSorter(graph)
585-
>>> tuple(ts.static_order())
586-
('A', 'C', 'B', 'D')
587-
588-
The class is designed to easily support parallel processing of the nodes as
589-
they become ready. For instance::
590-
591-
topological_sorter = TopologicalSorter()
592-
593-
# Add nodes to 'topological_sorter'...
594-
595-
topological_sorter.prepare()
596-
while topological_sorter.is_active():
597-
for node in topological_sorter.get_ready():
598-
# Worker threads or processes take nodes to work on off the
599-
# 'task_queue' queue.
600-
task_queue.put(node)
601-
602-
# When the work for a node is done, workers put the node in
603-
# 'finalized_tasks_queue' so we can get more nodes to work on.
604-
# The definition of 'is_active()' guarantees that, at this point, at
605-
# least one node has been placed on 'task_queue' that hasn't yet
606-
# been passed to 'done()', so this blocking 'get()' must (eventually)
607-
# succeed. After calling 'done()', we loop back to call 'get_ready()'
608-
# again, so put newly freed nodes on 'task_queue' as soon as
609-
# logically possible.
610-
node = finalized_tasks_queue.get()
611-
topological_sorter.done(node)
612-
613-
.. method:: add(node, *predecessors)
614-
615-
Add a new node and its predecessors to the graph. Both the *node* and all
616-
elements in *predecessors* must be hashable.
617-
618-
If called multiple times with the same node argument, the set of
619-
dependencies will be the union of all dependencies passed in.
620-
621-
It is possible to add a node with no dependencies (*predecessors* is not
622-
provided) or to provide a dependency twice. If a node that has not been
623-
provided before is included among *predecessors* it will be automatically
624-
added to the graph with no predecessors of its own.
625-
626-
Raises :exc:`ValueError` if called after :meth:`~TopologicalSorter.prepare`.
627-
628-
.. method:: prepare()
629-
630-
Mark the graph as finished and check for cycles in the graph. If any cycle
631-
is detected, :exc:`CycleError` will be raised, but
632-
:meth:`~TopologicalSorter.get_ready` can still be used to obtain as many
633-
nodes as possible until cycles block more progress. After a call to this
634-
function, the graph cannot be modified, and therefore no more nodes can be
635-
added using :meth:`~TopologicalSorter.add`.
636-
637-
.. method:: is_active()
638-
639-
Returns ``True`` if more progress can be made and ``False`` otherwise.
640-
Progress can be made if cycles do not block the resolution and either
641-
there are still nodes ready that haven't yet been returned by
642-
:meth:`TopologicalSorter.get_ready` or the number of nodes marked
643-
:meth:`TopologicalSorter.done` is less than the number that have been
644-
returned by :meth:`TopologicalSorter.get_ready`.
645-
646-
The :meth:`~TopologicalSorter.__bool__` method of this class defers to
647-
this function, so instead of::
648-
649-
if ts.is_active():
650-
...
651-
652-
if possible to simply do::
653-
654-
if ts:
655-
...
656-
657-
Raises :exc:`ValueError` if called without calling
658-
:meth:`~TopologicalSorter.prepare` previously.
659-
660-
.. method:: done(*nodes)
661-
662-
Marks a set of nodes returned by :meth:`TopologicalSorter.get_ready` as
663-
processed, unblocking any successor of each node in *nodes* for being
664-
returned in the future by a call to :meth:`TopologicalSorter.get_ready`.
665-
666-
Raises :exc:`ValueError` if any node in *nodes* has already been marked as
667-
processed by a previous call to this method or if a node was not added to
668-
the graph by using :meth:`TopologicalSorter.add`, if called without
669-
calling :meth:`~TopologicalSorter.prepare` or if node has not yet been
670-
returned by :meth:`~TopologicalSorter.get_ready`.
671-
672-
.. method:: get_ready()
673-
674-
Returns a ``tuple`` with all the nodes that are ready. Initially it
675-
returns all nodes with no predecessors, and once those are marked as
676-
processed by calling :meth:`TopologicalSorter.done`, further calls will
677-
return all new nodes that have all their predecessors already processed.
678-
Once no more progress can be made, empty tuples are returned.
679-
680-
Raises :exc:`ValueError` if called without calling
681-
:meth:`~TopologicalSorter.prepare` previously.
682-
683-
.. method:: static_order()
684-
685-
Returns an iterable of nodes in a topological order. Using this method
686-
does not require to call :meth:`TopologicalSorter.prepare` or
687-
:meth:`TopologicalSorter.done`. This method is equivalent to::
688-
689-
def static_order(self):
690-
self.prepare()
691-
while self.is_active():
692-
node_group = self.get_ready()
693-
yield from node_group
694-
self.done(*node_group)
695-
696-
The particular order that is returned may depend on the specific order in
697-
which the items were inserted in the graph. For example:
698-
699-
.. doctest::
700-
701-
>>> ts = TopologicalSorter()
702-
>>> ts.add(3, 2, 1)
703-
>>> ts.add(1, 0)
704-
>>> print([*ts.static_order()])
705-
[2, 0, 1, 3]
706-
707-
>>> ts2 = TopologicalSorter()
708-
>>> ts2.add(1, 0)
709-
>>> ts2.add(3, 2, 1)
710-
>>> print([*ts2.static_order()])
711-
[0, 2, 1, 3]
712-
713-
This is due to the fact that "0" and "2" are in the same level in the
714-
graph (they would have been returned in the same call to
715-
:meth:`~TopologicalSorter.get_ready`) and the order between them is
716-
determined by the order of insertion.
717-
718-
719-
If any cycle is detected, :exc:`CycleError` will be raised.
720-
721-
.. versionadded:: 3.9
722-
723-
724546
.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
725547

726548
Update a *wrapper* function to look like the *wrapped* function. The optional
@@ -829,20 +651,4 @@ callable, weak referencable, and can have attributes. There are some important
829651
differences. For instance, the :attr:`~definition.__name__` and :attr:`__doc__` attributes
830652
are not created automatically. Also, :class:`partial` objects defined in
831653
classes behave like static methods and do not transform into bound methods
832-
during instance attribute look-up.
833-
834-
835-
Exceptions
836-
----------
837-
The :mod:`functools` module defines the following exception classes:
838-
839-
.. exception:: CycleError
840-
841-
Subclass of :exc:`ValueError` raised by :meth:`TopologicalSorter.prepare` if cycles exist
842-
in the working graph. If multiple cycles exist, only one undefined choice among them will
843-
be reported and included in the exception.
844-
845-
The detected cycle can be accessed via the second element in the :attr:`~CycleError.args`
846-
attribute of the exception instance and consists in a list of nodes, such that each node is,
847-
in the graph, an immediate predecessor of the next node in the list. In the reported list,
848-
the first and the last node will be the same, to make it clear that it is cyclic.
654+
during instance attribute look-up.

0 commit comments

Comments
 (0)