math_spec.where_parser

pyparsing-based parser for where strings — grammar and AST only.

Parses strings like "p_max > 0 AND NOT is_must_run" into an AST. What a mask means is the consumer's business: it evaluates the AST against the data it holds.

ConnectiveWhereNode = NotNode | AndNode | OrNode module-attribute

PredicateOperator = Literal['<=', '>=', '==', '!=', '<', '>'] module-attribute

TypedPredicateNode = ParameterComparisonNode | ParameterDefinedNode | VariableDefinedNode | DimensionComparisonNode | DimensionPositionNode | LookupComparisonNode | LookupPairComparisonNode | LookupDefinedNode module-attribute

UnresolvedWhereNode = UnresolvedNameNode | UnresolvedComparisonNode | UnresolvedPositionNode module-attribute

WhereNode = BooleanLiteralNode | UnresolvedNameNode | UnresolvedComparisonNode | UnresolvedPositionNode | DimensionPositionNode | ParameterDefinedNode | VariableDefinedNode | ParameterComparisonNode | DimensionComparisonNode | LookupComparisonNode | LookupPairComparisonNode | LookupDefinedNode | NotNode | AndNode | OrNode module-attribute

AndNode(left, right) dataclass

left instance-attribute

right instance-attribute

BooleanLiteralNode(value) dataclass

value instance-attribute

DimensionComparisonNode(name, op, value) dataclass

Compare a dimension's own coordinates against a literal.

name instance-attribute

op instance-attribute

value instance-attribute

DimensionPositionNode(name, op, position, by=None) dataclass

Compare where a row sits along a dimension against a position — position(snapshot) == 0.

Both sides are integers, negative counting from the end; comparing coordinates against the label at a position would read differently on an axis whose coordinates do not arrive sorted (#32). With by the position is counted within each group the lookup makes.

by = None class-attribute instance-attribute

name instance-attribute

op instance-attribute

position instance-attribute

LookupComparisonNode(name, over, op, value) dataclass

Compare a lookup's values against a literal — period_of == 2030.

over is the dimension the lookup maps out of, copied off the declaration during resolution so the frame check and every consumer read it here rather than looking the lookup up again.

name instance-attribute

op instance-attribute

over instance-attribute

value instance-attribute

LookupDefinedNode(name, over) dataclass

True where the named lookup has a value — the partial-lookup case.

A lookup may be partial: a null says the label belongs to no group (a generator on no bus, a line with one open end). This is how a declaration asks for the labels that do map, spelled as a bare name exactly as a parameter's definedness is.

name instance-attribute

over instance-attribute

LookupPairComparisonNode(name, other, over, op) dataclass

Compare two lookups over one dimension — from != to.

The one comparison whose both sides are structure: two maps out of the same dimension, tested row by row on that dimension's own table. Over different dims there is no row to compare them on, which resolution refuses.

name instance-attribute

op instance-attribute

other instance-attribute

over instance-attribute

NotNode(operand) dataclass

operand instance-attribute

OrNode(left, right) dataclass

left instance-attribute

right instance-attribute

ParameterComparisonNode(name, op, value) dataclass

Compare a parameter against a literal, element-wise.

name instance-attribute

op instance-attribute

value instance-attribute

ParameterDefinedNode(name) dataclass

True wherever the named parameter is non-null and finite.

name instance-attribute

UnresolvedComparisonNode(name, op, value, quoted=False) dataclass

A comparison against an unresolved name. resolution.py types it.

name instance-attribute

op instance-attribute

quoted = False class-attribute instance-attribute

value instance-attribute

UnresolvedNameNode(name) dataclass

A bare name — unresolved. resolution.py types it.

name instance-attribute

UnresolvedPositionNode(dimension, op, position, by=None) dataclass

position(dim) <op> i before the name is checked.

Kept apart from :class:UnresolvedComparisonNode because its left-hand side is not a name but an application to one, which no bare name can carry. resolution.py types it into :class:DimensionPositionNode.

by = None class-attribute instance-attribute

dimension instance-attribute

op instance-attribute

position instance-attribute

VariableDefinedNode(name) dataclass

True at the coordinates where the named variable exists.

The variable counterpart of :class:ParameterDefinedNode, and spelled the same way — a bare name. A parameter's bare name asks whether it has a value here; a variable's asks whether it exists here.

name instance-attribute

atoms(where)

Every node in where that reads a declaration, connectives removed.

A predicate is a tree of :data:ConnectiveWhereNode over leaves that each name one declaration, so every question about what a mask reads is asked of the leaves and answered by taking them together. A boolean literal reads nothing and yields nothing.

RAISES	DESCRIPTION
AssertionError	An unresolved node, which is a pass running before resolution rather than a predicate with a property to read.

Source code in src/math_spec/where_parser.py

def atoms(where: WhereNode) -> Iterator[TypedPredicateNode]:
    """Every node in *where* that reads a declaration, connectives removed.

    A predicate is a tree of :data:`ConnectiveWhereNode` over leaves that each
    name one declaration, so every question about what a mask *reads* is asked
    of the leaves and answered by taking them together. A boolean literal reads
    nothing and yields nothing.

    Raises:
        AssertionError: An unresolved node, which is a pass running before
            resolution rather than a predicate with a property to read.
    """
    if isinstance(where, NotNode):
        yield from atoms(where.operand)
    elif isinstance(where, (AndNode, OrNode)):
        yield from atoms(where.left)
        yield from atoms(where.right)
    elif isinstance(where, UnresolvedWhereNode):
        msg = f'{type(where).__name__} reached a predicate walk unresolved.'
        raise AssertionError(msg)
    elif not isinstance(where, BooleanLiteralNode):
        yield where

dims_read(where, name_dims)

Which dims where reads, given what each declared name is read through.

The dim rule for the predicate side, stated once: a mask is read at the coordinates its leaves are read at. A parameter is read through its own dims, a variable through the frame it is declared over, a comparison on a dimension through that dimension, and a lookup through the dimension it maps out of — a lookup being read on the dim it leaves, not the one it lands in.

A consumer masking rows needs this to know which coordinates a mask can restrict, and answering it separately is the mistake what-counts-as-language.md forbids: two consumers deciding differently would mask the same model differently, with no error anywhere.

PARAMETER	DESCRIPTION
where	A resolved predicate. TYPE: WhereNode
name_dims	Every declared name to the dims it is read through — parameters by their dims and variables by their foreach, one flat mapping because the language has one flat namespace. TYPE: Mapping[str, Sequence[str]]

RETURNS	DESCRIPTION
frozenset[str]	The dims read, which is empty for a predicate over nothing but
frozenset[str]	literals.

Source code in src/math_spec/where_parser.py

def dims_read(where: WhereNode, name_dims: Mapping[str, Sequence[str]]) -> frozenset[str]:
    """Which dims *where* reads, given what each declared name is read through.

    The dim rule for the predicate side, stated once: **a mask is read at the
    coordinates its leaves are read at**. A parameter is read through its own
    dims, a variable through the frame it is declared over, a comparison on a
    dimension through that dimension, and a lookup through the dimension it
    maps out of — a lookup being read on the dim it leaves, not the one it
    lands in.

    A consumer masking rows needs this to know which coordinates a mask can
    restrict, and answering it separately is the mistake
    ``what-counts-as-language.md`` forbids: two consumers deciding differently
    would mask the same model differently, with no error anywhere.

    Args:
        where: A resolved predicate.
        name_dims: Every declared name to the dims it is read through —
            parameters by their ``dims`` and variables by their ``foreach``,
            one flat mapping because the language has one flat namespace.

    Returns:
        The dims read, which is empty for a predicate over nothing but
        literals.
    """
    return frozenset(dim for atom in atoms(where) for dim in _atom_dims(atom, name_dims))

names_read(where)

Which declarations where names — the parameters, lookups and variables its leaves test.

The name rule to :func:dims_read's dim rule, and its complement: a leaf is read at a dimension and of a declaration, and where dims_read gives the first this gives the second. A comparison on a dimension names no declaration — a coordinate is not data to feed — so it adds nothing here; its dimension is dims_read's. A lookup pair names both maps it compares.

A consumer asking which parameters a mask gates on — whether the corpus ever varies one, say — asks it here rather than walking the leaves itself, the what-counts-as-language.md rule that two consumers must not answer it differently.

RETURNS	DESCRIPTION
frozenset[str]	The parameter, lookup and variable names read, empty for a predicate
frozenset[str]	over nothing but literals and dimensions.

Source code in src/math_spec/where_parser.py

def names_read(where: WhereNode) -> frozenset[str]:
    """Which declarations *where* names — the parameters, lookups and variables its leaves test.

    The name rule to :func:`dims_read`'s dim rule, and its complement: a leaf is
    read *at* a dimension and *of* a declaration, and where ``dims_read`` gives
    the first this gives the second. A comparison on a dimension names no
    declaration — a coordinate is not data to feed — so it adds nothing here;
    its dimension is ``dims_read``'s. A lookup pair names both maps it compares.

    A consumer asking which parameters a mask gates on — whether the corpus
    ever varies one, say — asks it here rather than walking the leaves itself,
    the ``what-counts-as-language.md`` rule that two consumers must not answer
    it differently.

    Returns:
        The parameter, lookup and variable names read, empty for a predicate
        over nothing but literals and dimensions.
    """
    return frozenset(name for atom in atoms(where) for name in _atom_names(atom))

parse_where(text)

Parse a where string into an AST.

RAISES	DESCRIPTION
SchemaError	If text is not a where string of the language.

Source code in src/math_spec/where_parser.py

def parse_where(text: str) -> WhereNode:
    """Parse a where string into an AST.

    Raises:
        SchemaError: If *text* is not a where string of the language.
    """
    try:
        result = _WHERE_GRAMMAR.parse_string(text, parse_all=True)
    except pp.ParseException as e:
        msg = f'Failed to parse where string: {text!r}\n{e}'
        if _INDEX_CALL.search(text):
            msg += _INDEX_REWRITE
        raise SchemaError(msg) from e
    return cast('WhereNode', result[0])