symforce.codegen.codegen_util module

Shared helper code between codegen of all languages.

class DenseAndSparseOutputTerms(dense, sparse)

Bases: tuple

Parameters:

• dense (List[List[Expr]]) –

• sparse (List[List[Expr]]) –

dense: List[List[Expr]]

Alias for field number 0

sparse: List[List[Expr]]

Alias for field number 1

class OutputWithTerms(name, type, terms)

Bases: tuple

Parameters:

• name (str) –

• type (Any) –

• terms (Sequence[Tuple[str, str]]) –

name: str

Alias for field number 0

type: Any

Alias for field number 1

terms: Sequence[Tuple[str, str]]

Alias for field number 2

class PrintCodeResult(intermediate_terms, dense_terms, sparse_terms, total_ops)

Bases: tuple

Parameters:

• intermediate_terms (Sequence[Tuple[str, str]]) –

• dense_terms (List[OutputWithTerms]) –

• sparse_terms (List[OutputWithTerms]) –

• total_ops (int) –

intermediate_terms: Sequence[Tuple[str, str]]

Alias for field number 0

dense_terms: List[OutputWithTerms]

Alias for field number 1

sparse_terms: List[OutputWithTerms]

Alias for field number 2

total_ops: int

Alias for field number 3

class CSCFormat(kRows, kCols, kNumNonZero, kColPtrs, kRowIndices, nonzero_elements)

Bases: object

A matrix written in Compressed Sparse Column format.

Parameters:

• kRows (int) –

• kCols (int) –

• kNumNonZero (int) –

• kColPtrs (List[int]) –

• kRowIndices (List[int]) –

• nonzero_elements (List[float]) –

kRows: int

kCols: int

kNumNonZero: int

kColPtrs: List[int]

kRowIndices: List[int]

nonzero_elements: List[float]

static from_matrix(sparse_matrix)

Returns a dictionary with the metadata required to represent a matrix as a sparse matrix in CSC form.

Parameters:

sparse_matrix (Matrix) – A symbolic sf.Matrix where sparsity is given by exact zero equality.

Return type:

CSCFormat

to_matrix()

Returns a dense matrix representing this CSC sparse matrix.

Return type:

Matrix

print_code(inputs, outputs, sparse_mat_data, config, cse=True)

Return executable code lines from the given input/output values.

Parameters:

• inputs (Values) – Values object specifying names and symbolic inputs

• outputs (Values) – Values object specifying names and output expressions (written in terms of the symbolic inputs)

• sparse_mat_data (Dict[str, CSCFormat]) – Data associated with sparse matrices. sparse_mat_data["keys"] stores a list of the keys in outputs which should be treated as sparse matrices

• config (CodegenConfig) – Programming language and configuration in which the expressions are to be generated

• cse (bool) – Perform common sub-expression elimination

Returns:

• T.List[T.Tuple[str, str]] – Line of code per temporary variable

• T.List[OutputWithTerms] – Collection of lines of code per dense output variable

• T.List[OutputWithTerms] – Collection of lines of code per sparse output variable

• int – Total number of ops

Return type:

PrintCodeResult

perform_cse(output_exprs, cse_optimizations=None)

Run common sub-expression elimination on the given input/output values.

Parameters:

• output_exprs (DenseAndSparseOutputTerms) – expressions on which to perform cse

• cse_optimizations (Literal['basic'] | ~typing.Sequence[~typing.Tuple[~typing.Callable, ~typing.Callable]] | None) – optimizations to be forwarded to sf.cse

Returns:

• T_terms – Temporary variables holding the common sub-expressions found within output_exprs

• DenseAndSparseOutputTerms – output_exprs, but in terms of the returned temporaries.

Return type:

Tuple[Sequence[Tuple[Symbol, Expr]], DenseAndSparseOutputTerms]

format_symbols(inputs, dense_outputs, sparse_outputs, intermediate_terms, output_terms, config)

Reformats symbolic variables used in intermediate and outputs terms to match structure of inputs/outputs.

For example, if we have an input array "arr" with symbolic elements [arr0, arr1], we will remap symbol "arr0" to "arr[0]" and symbol "arr1" to "arr[1]".

Parameters:

• inputs (Values) –

• dense_outputs (Values) –

• sparse_outputs (Values) –

• intermediate_terms (Sequence[Tuple[Symbol, Expr]]) –

• output_terms (DenseAndSparseOutputTerms) –

• config (CodegenConfig) –

Return type:

Tuple[Sequence[Tuple[Symbol, Expr]], Sequence[Sequence[Tuple[Symbol, Expr]]], Sequence[Sequence[Tuple[Symbol, Expr]]]]

get_formatted_list(values, config, format_as_inputs)

Returns a nested list of formatted symbols, as well as a nested list of the corresponding original scalar values. For use in generated functions.

Parameters:

• values (Values) – Values object mapping keys to different objects. Here we only use the object types, not their actual values.

• config (CodegenConfig) – Programming language and configuration for when language-specific formatting is required

• format_as_inputs (bool) – True if values defines the input symbols, false if values defines output expressions.

Returns:

• flattened_formatted_symbolic_values – nested list of formatted scalar symbols

• flattened_original_values – nested list of original scalar values

Return type:

Tuple[List[List[Symbol | DataBuffer]], List[List[float]]]

get_formatted_sparse_list(sparse_outputs)

Returns a nested list of symbols for use in generated functions for sparse matrices.

Parameters:

sparse_outputs (Values) –

Return type:

List[List[float]]

load_generated_package(name, path, evict=True)

Dynamically load generated package (or module).

Parameters:

• name (str) – The full name of the package or module to load (for example, "pkg.sub_pkg" for a package called sub_pkg inside of another package pkg, or "pkg.sub_pkg.mod" for a module called mod inside of pkg.sub_pkg).

• path (str | PathLike) – The path to the directory (or __init__.py) of the package, or the python file of the module.

• evict (bool) – Whether to evict the imported package from sys.modules after loading it. This is necessary for functions generated in the sym namespace, since leaving them would make it impossible to import sym and get the symforce-sym package as expected. For this reason, attempting to load a generated package called sym with evict=False is disallowed. However, evict should be False for numba-compiled functions.

Return type:

Any

load_generated_function(func_name, path_to_package, evict=True)

Returns the function with name func_name found inside the package located at path_to_package.

Example usage:

def my_func(...):
    ...

my_codegen = Codegen.function(my_func, config=PythonConfig())
codegen_data = my_codegen.generate_function(output_dir=output_dir)
generated_func = load_generated_function("my_func", codegen_data.function_dir)
generated_func(...)

Parameters:

• path_to_package (str | PathLike) – a python package with an __init__.py containing a module defined in func_name.py which in turn defines an attribute named func_name. See the example above.

• evict (bool) – Whether to evict the imported package from sys.modules after loading it. This is necessary for functions generated in the sym namespace, since leaving them would make it impossible to import sym and get the symforce-sym package as expected. For this reason, attempting to load a generated package called sym with evict=False is disallowed. However, evict should be False for numba-compiled functions.

• func_name (str) –

Return type:

Callable

load_generated_lcmtype(package, type_name, lcmtypes_path)

Load an LCM type generated by Codegen.generate_function

Example usage:

my_codegen = Codegen(my_func, config=PythonConfig())
codegen_data = my_codegen.generate_function(output_dir=output_dir, namespace=namespace)
my_type_t = codegen_util.load_generated_lcmtype(
    namespace, "my_type_t", codegen_data.python_types_dir
)
my_type_msg = my_type_t(foo=5)

Parameters:

• package (str) – The name of the LCM package for the type

• type_name (str) – The name of the LCM type itself (not including the package)

• lcmtypes_path (str | Path) – The path to the directory containing the generated lcmtypes package

Returns:

The Python LCM type

Return type:

Type

get_base_instance(obj)

Returns an instance of the base element (e.g. Scalar, Values, Matrix, etc.) of an object. If input is a list (incl. multidimensional lists), we return an instance of one of the base elements (i.e. the first element that isn’t a list). If input is a list we assume all elements are of the same type/shape.

Parameters:

obj (Sequence[Any]) –

Return type:

Any

class LcmBindingsDirs(python_types_dir: 'Path', cpp_types_dir: 'Path')

Bases: object

Parameters:

• python_types_dir (Path) –

• cpp_types_dir (Path) –

python_types_dir: Path

cpp_types_dir: Path

generate_lcm_types(lcm_type_dir, lcm_files, lcm_output_dir=None)

Generates the language-specific type files for all symforce generated “.lcm” files.

Parameters:

• lcm_type_dir (str | PathLike) – Directory containing symforce-generated .lcm files

• lcm_files (Sequence[str]) – List of .lcm files to process

• lcm_output_dir (str | PathLike | None) –

Return type:

LcmBindingsDirs

flat_symbols_from_values(values)

Returns a flat list of unique symbols in the object for codegen Note that this does not respect storage ordering

Parameters:

values (Values) –

Return type:

List[Any]