graph998%python.module
fault.project

Project Graph processing tools.

Provides Queue for managing the processing of projects in dependency order.

Usage

from fault.system import files
from fault.projects import system as lsf
from fault.projects import graph
ctx = lsf.Context()
# Configure paths making up the project context.
ctx.connect(files.root@"/product/path")
ctx.load()
q = Queue()
ext = q.extend(ctx)

def process(projects):
	# Print dependency order for this example.
	for x in projects:
		print(x)
	return projects # Signals completion in this example.

# Unconditionally take initial set.
jobs = list(q.take(4))
while not q.terminal() and jobs:
	finished_set = process(jobs)
	# Notify queue state of completions.
	jobs = list(q.finish(*finished_set).take(4))

Engineering

The functions here could be easily refactored into a properly generalized form, but considering the code size and the benefits of a self-contained implementation, it seemed reasonable to leave the weight here. Even in the case of some duplicate effort, this should likely stay as-is.

typing
import

collections0
import

itertools0
import

collect90%
function

collect(context)

Collect the projects and requirements from the system.Context instance.

structure210%
function

structure(collected)

Structure the projects and the required factors produced by collect into a graph that may be processed with sequence.

sequence120%
function

sequence(pair)

Sequence the nodes of a project graph according to the completion order recognized from the generator send calls.

Queue0
class

object

State object for processing projects in dependency order.

Queue__init__30%
method

__init__(self)

Allocate instance; follow with extend.

Queueextend137%
method
typing.List[str]

extend(self, context)

Extend the queue using the projects contained within context.

Returns any out-of-context requirements.

Queuestatus20%
method

status(self)

A tuple of integers telling how many projects have been reported as finished and the total number of projects.

Queuefinish120%
method

finish(self, *projects)

Note the projects as being processed.

Returns instance for chaining with take.

Queuetake516%
method
typing.Iterator[object]

take(self, quantity)

Retrieve enqueued project IIDs in graph order.

Queueinterrupt20%
method

interrupt(self)

Halt the queue causing terminal to return True and take to return empty lists. The internal state will be discarded, but entries may still be finished.

Queueterminal420%
method
bool

terminal(self)

Whether or not the queue has released all projects via take.

This can return True prior to a final finish call.