10.2. typped.builtin_parse_methods

These are predefined, built-in parsing routines designed to parse various general types of constructs. These functions are all made into methods of the PrattParser class simply because that namespace is convenient to use. When calling as a parser method the first argument should be omitted, e.g., parser.def_literal("k_lpar") The functions can also be called directly, passed the parser as the first argument.

Looking at the source code can be useful for ideas of how to implement general constructs which are not covered by a builtin routine.

typped.builtin_parse_methods.def_literal(parser, token_label, val_type=None, precond_fun=None, precond_priority=0, construct_label=None, val_type_override_fun=None, eval_fun=None, ast_data=None)

Defines the token with label token_label to be a literal in the syntax of the language being parsed. This method adds a head handler function to the token. Literal tokens are the leaves of the expression trees; they are things like numbers and variable names in a numerical expression. They always occur as the first (and only) token in a subexpression being evaluated by recursive_parse, so they need a head handler but not a tail handler. (Though note that the token itparser might also have a tail handler.)

A function val_type_override_fun can be passed in, taking a token and a lexer as its two arguments and returning a TypeObject instance. If it is set then it will called in the handler at parse-time to get the type to set as the val_type of the node. This can be useful for dynamic typing such as when identifiers in an interpreted language are generic variables which can holding different types. This option currently does not work for overloading on return types.

typped.builtin_parse_methods.def_multi_literals(parser, tuple_list)

An interface to the def_literal method which takes a list of tuples. The def_literal method will be called for each tuple, unpacked in the order in the tuple. Unspecified optional arguments are assigned their default values.

Usually it is better to define literal = parser.def_literal and use that as a shorter alias. This method does not allow for keyword arguments and depends on argument ordering.

typped.builtin_parse_methods.def_bracket_pair(parser, lbrac_token_label, rbrac_token_label, in_tree=True, precond_fun=None, precond_priority=0, construct_label=None, eval_fun=None, ast_data=None)

Define a matching bracket grouping operation. The returned type is set to the type of its single child (i.e., the type of the contents of the brackets). Defines a head handler for the left bracket token, so effectively gets the highest evaluation precedence. As far as types, it is treated as a function that takes one argument of wildcard type and returns whatever type the argument has.

typped.builtin_parse_methods.def_stdfun(parser, fname_token_label, lpar_token_label, rpar_token_label, comma_token_label, precond_fun=None, precond_priority=1, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None, num_args=None, token_value_key=None)

This definition of stdfun uses lookahead to the opening paren or bracket token.

Note that all tokens must be defined as literal tokens except fname_token_label (which ends up as the root of the function evaluation subtree). If the latter is also a literal token then precond_priority may need to be increased to give this use priority.

The num_args parameter is optional for specifying the number of arguments when typing is not being used. If it is set to a nonnegative number then it will automatically set arg_types to the corresponding list of None values; if arg_types is set then it is ignored. If type-checking is disabled for the parser instance then the number of arguments is instead checked by the handler function.

typped.builtin_parse_methods.def_stdfun_lpar_tail(parser, fname_token_label, lpar_token_label, rpar_token_label, comma_token_label, prec_of_lpar, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None, num_args=None, token_value_key=None)

This is an alternate version of stdfun that defines lpar as an infix operator (i.e., with a tail handler). This function works in the usual cases but the current version without preconditions may have problems distinguishing “b (” from “b(” when a multiplication jop is set. The lookahead version def_stdfun is usually preferred.

This method assumes type checking is turned on if num_arg is set.

A peek backwards to a token with label fname_token_label is included in the preconditions function. Definitions for different leading tokens will give mutually exclusive preconditions.

typped.builtin_parse_methods.def_infix_multi_op(parser, operator_token_labels, prec, assoc, repeat=False, not_in_tree=False, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None)

Takes a list of operator token labels and defines a multi-infix operator.

If repeat=True then any number of repetitions of the list of operators will be accepted. For example, a comma operator could be used to parse a full comma-separated list. When arg_types is also set use the Varargs object in the list to check the repetitions. For a single operator, repeating just has the effect of putting the arguments in a flat argument/child list instead of as nested binary operations based on left or right association. Any argument-checking is done after any node removal, which may affect the types that should be passed-in in the list arg_types of parent constructs.

If not_in_tree is false then the root node will not appear in the final parse tree (unless it is the root).

typped.builtin_parse_methods.def_infix_op(parser, operator_token_label, prec, assoc, not_in_tree=False, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None)

This just calls the more general method def_multi_infix_op.

typped.builtin_parse_methods.def_prefix_op(parser, operator_token_label, prec, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None)

Define a prefix operator. Note that head handlers do not have precedences, only tail handlers. (With respect to the looping in recursive_parse it wouldn’t make a difference.) But, within the head handler, the call to recursive_parse can be made with a nonzero precedence. This allows setting a precedence to determine the argument expressions that the prefix operators grabs up (or doesn’t).

typped.builtin_parse_methods.def_postfix_op(parser, operator_token_label, prec, allow_ignored_before=True, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None)

Define a postfix operator. If allow_ignored_before is false then no ignored token (usually whitespace) can appear immediately before the operator.

typped.builtin_parse_methods.def_jop(parser, prec, assoc, precond_fun=None, precond_priority=None, construct_label=None, val_type=None, arg_types=None, eval_fun=None, ast_data=None)

The function precond_fun is called to determine whether or not to accept a potentially-inferred a juxtaposition operator between the previously-parsed subexpression result and the next token. Note that this function have available extra_data as an attribute of its triggering token, and extra_data contains the lookbehind attribute. Through the lookbehind list the jop_precond function has access to the type information for the potential left operand but not for the potential right operand.

Note that if the juxtaposition operator always resolves to a single type signature based on its argument types then, even if overloading on return types is in effect, the jop can be effectively inferred based on type signature information.

typped.builtin_parse_methods.def_assignment_op_untyped(parser, assignment_op_token_label, prec, assoc, identifier_token_label, symbol_value_dict=None, precond_fun=None, precond_priority=0, construct_label=None, eval_fun=None, create_eval_fun=False, ast_data=None)

Define an infix assignment operator which is statically typed, with types checked at parse time. Each identifier (with token label identifier_token_label must already have a type associated with it in the symbol_type_dict. This dict and the type values in it should be set via whatever kind of a type definition construct the language uses.

A precondition checks that the l.h.s. of the assignment operator is a token with label identifier_token_label. If not an exception is raised.

An evaluation function can optionally be created automatically, but by default is not. See the def_assignment_op_dynamic routine for more details since the mechanism is the same. If eval_fun is set then that evaluation function will always be used.

This method may not correctly set the return type when overloading on return types because currently val_type_override is used to set it.

typped.builtin_parse_methods.def_assignment_op_static(parser, assignment_op_token_label, prec, assoc, identifier_token_label, symbol_value_dict=None, symbol_type_dict=None, allowed_types=None, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, eval_fun=None, create_eval_fun=False, ast_data=None)

Define an infix assignment operator which is statically typed, with types checked at parse time. Each identifier (with token label identifier_token_label must already have a type associated with it in the symbol_type_dict. This dict and the type values in it should be set via whatever kind of a type definition construct the language uses.

A precondition checks that the l.h.s. of the assignment operator is a token with label identifier_token_label. If not an exception is raised.

An evaluation function can optionally be created automatically, but by default is not. See the def_assignment_op_dynamic routine for more details since the mechanism is the same. If eval_fun is set then that evaluation function will always be used.

This method may not correctly set the return type when overloading on return types because currently val_type_override is used to set it.

typped.builtin_parse_methods.def_assignment_op_dynamic(parser, assignment_op_token_label, prec, assoc, identifier_token_label, symbol_value_dict=None, symbol_type_dict=None, allowed_types=None, precond_fun=None, precond_priority=0, construct_label=None, val_type=None, eval_fun=None, create_eval_fun=False, ast_data=None)

Define an infix assignment operator which is dynamically typed, with types checked at evaluation time (i.e., when the tree is interpreted).

A precondition checks that the l.h.s. of the assignment operator is a token with label identifier_token_label. If not an exception is raised.

No type-checking is done on the r.h.s. by default. To limit the types that can be assigned you can pass in a list or iterable of TypeObject instances as the argument allowed_types. These formal types are stored as the list attribute allowed_dynamic_assignment_types of the parser instance. An exception will be raised by the generated evaluation function if an assigned value does not have an actual type consistent with a formal type on that list. If new types are created later they can be directly appended to that list without having to overload the assignment operator.

If create_eval_fun is true (and eval_fun is not set) then an evaluation function will be created automatically. The symbol_value_dict is used to store the values, which defaults to the parser attribute of the same name.

This method may not correctly set the return type when overloading on return types because currently val_type_override is used to set it.

typped.builtin_parse_methods.def_literal_typed_from_dict(parser, token_label, symbol_value_dict=None, symbol_type_dict=None, default_type=None, default_eval_value=None, raise_if_undefined=False, eval_fun=None, create_eval_fun=False, precond_fun=None, precond_priority=1, construct_label=None)

Define a dynamically typed literal, usually a variable-name identifier. The type is looked up in the dict symbol_type_dict, keyed by the string value of the token literal.

If create_eval_fun is true (and eval_fun is not set) then this method will provides an evaluation function automatically. This function returns the value looked up from symbol_value_dict, keyed by the literal token’s string value. The default value returned by the evaluation if the symbol is not in the dict is set via default_eval_value. (Currently there must be some default rather than raising an exception, with the default default value set to None.) Setting create_eval_fun false will skip the setting of an evaluation function.

The def_assignment_op_dynamic routine should be used to handle the corresponding variable assignment operation. That is, the assignment that dynamically sets the type of the literal to the type of the assigned value (storing it in symbol_type_dict by default).

This method may not correctly set the return type when overloading on return types because currently val_type_override is used to set it.