Values Tutorial

Values objects are ordered dict-like containers used to store multiple heterogeneous objects, normally for the purpose of function generation.

Typically a Values object will first be created to defines a number of different symbolic inputs (e.g. rotations, translations, scalars, poses, cameras, etc.). Then, a second Values object will be created to describe the objects to be returned from the function, which will be composed of the symbolic elements defined by the input Values object.

# Setup
import symforce

symforce.set_symbolic_api("sympy")
symforce.set_log_level("warning")

import symforce.symbolic as sf
from symforce.notebook_util import display
from symforce.notebook_util import set_notebook_defaults
from symforce.values import Values

set_notebook_defaults()

inputs = Values(
    x=sf.Symbol("x"),
    y=sf.Rot2.symbolic("c"),
)
display(inputs)

Values(
  x: x,
  y: <Rot2 <C real=c_{re}, imag=c_{im}>>,
)

The .add() method can add a symbol using its name as the key:

inputs.add(sf.Symbol("foo"))
display(inputs)

Values(
  x: x,
  y: <Rot2 <C real=c_{re}, imag=c_{im}>>,
  foo: foo,
)

Adding sub-values are well encouraged:

x, y = sf.symbols("x y")
expr = x**2 + sf.sin(y) / x**2
inputs["states"] = Values(p=expr)
display(inputs)

Values(
  x: x,
  y: <Rot2 <C real=c_{re}, imag=c_{im}>>,
  foo: foo,
  states:   Values(
    p: x**2 + sin(y)/x**2,
  ),
)

A Values serializes to a depth-first traversed list. This means it implements StorageOps:

display(inputs.to_storage())

$\displaystyle \left[ x, \ c_{re}, \ c_{im}, \ foo, \ x^{2} + \frac{\sin{\left(y \right)}}{x^{2}}\right]$

We can also get a flattened lists of keys and values, with . separation for sub-values:

display(inputs.items_recursive())

[('x', x),
 ('y', <Rot2 <C real=c_{re}, imag=c_{im}>>),
 ('foo', foo),
 ('states.p', x**2 + sin(y)/x**2)]

Note that there is a .keys_recursive() and a .values_recursive() which return flattened lists of keys and values respectively:

display(inputs.keys_recursive())
display(inputs.values_recursive())

['x', 'y', 'foo', 'states.p']

[x, <Rot2 <C real=c_{re}, imag=c_{im}>>, foo, x**2 + sin(y)/x**2]

To fully reconstruct the types in the Values from the serialized scalars, we need an index that describes which parts of the serialized list correspond to which types. The spec is T.Dict[str, IndexEntry] where IndexEntry has attributes offset, storage_dim, datatype, shape, item_index:

index = inputs.index()
index

{'x': IndexEntry(offset=0, storage_dim=1, _module='builtins', _qualname='float', shape=None, item_index=None),
 'y': IndexEntry(offset=1, storage_dim=2, _module='symforce.geo.rot2', _qualname='Rot2', shape=None, item_index=None),
 'foo': IndexEntry(offset=3, storage_dim=1, _module='builtins', _qualname='float', shape=None, item_index=None),
 'states': IndexEntry(offset=4, storage_dim=1, _module='symforce.values.values', _qualname='Values', shape=None, item_index={'p': IndexEntry(offset=0, storage_dim=1, _module='builtins', _qualname='float', shape=None, item_index=None)})}

With a serialized list and an index, we can get the values back:

inputs2 = Values.from_storage_index(inputs.to_storage(), index)
assert inputs == inputs2
display(inputs)

Values(
  x: x,
  y: <Rot2 <C real=c_{re}, imag=c_{im}>>,
  foo: foo,
  states:   Values(
    p: x**2 + sin(y)/x**2,
  ),
)

The item_index is a recursive structure that can contain the index for a sub-values:

item_index = inputs.index()["states"].item_index
assert item_index == inputs["states"].index()

We can also set sub-values directly with dot notation in the keys. They get split up:

inputs["states.blah"] = 3
display(inputs)

Values(
  x: x,
  y: <Rot2 <C real=c_{re}, imag=c_{im}>>,
  foo: foo,
  states:   Values(
    p: x**2 + sin(y)/x**2,
    blah: 3,
  ),
)

The .attr field also allows attribute access rather than key access:

assert inputs["states.p"] is inputs["states"]["p"] is inputs.attr.states.p
display(inputs.attr.states.p)

$\displaystyle x^{2} + \frac{\sin{\left(y \right)}}{x^{2}}$

Finally, SymForce adds the concept of a name scope to namespace symbols. Within a scope block, symbol names get prefixed with the scope name:

with sf.scope("params"):
    s = sf.Symbol("cost")
display(s)

$\displaystyle params.cost$

A common use case is to call a function that adds symbols within your own name scope to avoid name collisions. You can also chain name scopes:

v = Values()
v.add(sf.Symbol("x"))
with sf.scope("foo"):
    v.add(sf.Symbol("x"))
    with sf.scope("bar"):
        v.add(sf.Symbol("x"))
display(v)
display(v.attr.foo.bar.x)

Values(
  x: x,
  foo:   Values(
    x: foo.x,
    bar:     Values(
      x: foo.bar.x,
    ),
  ),
)

$\displaystyle foo.bar.x$

The values class also provides a .scope() method that not only applies the scope to symbol names but also to keys added to the Values:

v = Values()
with v.scope("hello"):
    v["y"] = x**2
    v["z"] = sf.Symbol("z")
v

Values(
  hello:   Values(
    y: x**2,
    z: hello.z,
  ),
)

This flexible set of features provided by the Values class allows conveniently building up large expressions, and acts as the interface to code generation.

Lie Group Operations

One useful feature of Values objects is that element-wise Lie group operations on can be performed on them.

lie_vals = Values()
lie_vals["scalar"] = sf.Symbol("x")
lie_vals["rot3"] = sf.Rot3.symbolic("rot")

sub_lie_vals = Values()
sub_lie_vals["pose3"] = sf.Pose3.symbolic("pose")
sub_lie_vals["vec"] = sf.V3.symbolic("vec")

lie_vals["sub_vals"] = sub_lie_vals

display(lie_vals)

Values(
  scalar: x,
  rot3: <Rot3 <Q xyzw=[rot_x, rot_y, rot_z, rot_w]>>,
  sub_vals:   Values(
    pose3: <Pose3 R=<Rot3 <Q xyzw=[pose.R_x, pose.R_y, pose.R_z, pose.R_w]>>, t=(pose.t_0, pose.t_1, pose.t_2)>,
    vec: Matrix([
[vec_0],
[vec_1],
[vec_2]]),
  ),
)

display(lie_vals.tangent_dim())
display(len(lie_vals.to_tangent()))

$\displaystyle 13$

$\displaystyle 13$

Importantly, we can compute the jacobian of the storage space of the object with respect to its tangent space:

display(lie_vals.storage_D_tangent())

$\displaystyle \left[\begin{array}{ccccccccccccc}1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\0 & \frac{rot_{w}}{2} & - \frac{rot_{z}}{2} & \frac{rot_{y}}{2} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\0 & \frac{rot_{z}}{2} & \frac{rot_{w}}{2} & - \frac{rot_{x}}{2} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\0 & - \frac{rot_{y}}{2} & \frac{rot_{x}}{2} & \frac{rot_{w}}{2} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\0 & - \frac{rot_{x}}{2} & - \frac{rot_{y}}{2} & - \frac{rot_{z}}{2} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & \frac{pose.R_{w}}{2} & - \frac{pose.R_{z}}{2} & \frac{pose.R_{y}}{2} & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & \frac{pose.R_{z}}{2} & \frac{pose.R_{w}}{2} & - \frac{pose.R_{x}}{2} & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & - \frac{pose.R_{y}}{2} & \frac{pose.R_{x}}{2} & \frac{pose.R_{w}}{2} & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & - \frac{pose.R_{x}}{2} & - \frac{pose.R_{y}}{2} & - \frac{pose.R_{z}}{2} & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1\end{array}\right]$

This means that we can use the elements of the object to compute a residual, and then compute the jacobian of such a residual with respect to the tangent space of our values object.

residual = sf.Matrix(6, 1)
residual[0:3, 0] = lie_vals["rot3"] * lie_vals["sub_vals.vec"]
residual[3:6, 0] = lie_vals["sub_vals.pose3"] * lie_vals["sub_vals.vec"]
display(residual)

$\displaystyle \left[\begin{matrix}vec_{0} \left(- 2 rot_{y}^{2} - 2 rot_{z}^{2} + 1\right) + vec_{1} \left(- 2 rot_{w} rot_{z} + 2 rot_{x} rot_{y}\right) + vec_{2} \cdot \left(2 rot_{w} rot_{y} + 2 rot_{x} rot_{z}\right)\\vec_{0} \cdot \left(2 rot_{w} rot_{z} + 2 rot_{x} rot_{y}\right) + vec_{1} \left(- 2 rot_{x}^{2} - 2 rot_{z}^{2} + 1\right) + vec_{2} \left(- 2 rot_{w} rot_{x} + 2 rot_{y} rot_{z}\right)\\vec_{0} \left(- 2 rot_{w} rot_{y} + 2 rot_{x} rot_{z}\right) + vec_{1} \cdot \left(2 rot_{w} rot_{x} + 2 rot_{y} rot_{z}\right) + vec_{2} \left(- 2 rot_{x}^{2} - 2 rot_{y}^{2} + 1\right)\\pose.t_{0} + vec_{0} \left(- 2 pose.R_{y}^{2} - 2 pose.R_{z}^{2} + 1\right) + vec_{1} \left(- 2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y}\right) + vec_{2} \cdot \left(2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z}\right)\\pose.t_{1} + vec_{0} \cdot \left(2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y}\right) + vec_{1} \left(- 2 pose.R_{x}^{2} - 2 pose.R_{z}^{2} + 1\right) + vec_{2} \left(- 2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z}\right)\\pose.t_{2} + vec_{0} \left(- 2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z}\right) + vec_{1} \cdot \left(2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z}\right) + vec_{2} \left(- 2 pose.R_{x}^{2} - 2 pose.R_{y}^{2} + 1\right)\end{matrix}\right]$

residual_D_tangent = residual.jacobian(lie_vals)
display(residual_D_tangent.shape)
display(residual_D_tangent)

$\displaystyle \left( 6, \ 13\right)$

$\displaystyle \left[\begin{array}{ccccccccccccc}0 & vec_{1} \cdot \left(2 rot_{w} rot_{y} + 2 rot_{x} rot_{z}\right) + vec_{2} \cdot \left(2 rot_{w} rot_{z} - 2 rot_{x} rot_{y}\right) & vec_{0} \left(- 2 rot_{w} rot_{y} - 2 rot_{x} rot_{z}\right) + vec_{2} \left(rot_{w}^{2} + rot_{x}^{2} - rot_{y}^{2} - rot_{z}^{2}\right) & vec_{0} \left(- 2 rot_{w} rot_{z} + 2 rot_{x} rot_{y}\right) + vec_{1} \left(- rot_{w}^{2} - rot_{x}^{2} + rot_{y}^{2} + rot_{z}^{2}\right) & 0 & 0 & 0 & 0 & 0 & 0 & - 2 rot_{y}^{2} - 2 rot_{z}^{2} + 1 & - 2 rot_{w} rot_{z} + 2 rot_{x} rot_{y} & 2 rot_{w} rot_{y} + 2 rot_{x} rot_{z}\\0 & vec_{1} \left(- 2 rot_{w} rot_{x} + 2 rot_{y} rot_{z}\right) + vec_{2} \left(- rot_{w}^{2} + rot_{x}^{2} - rot_{y}^{2} + rot_{z}^{2}\right) & vec_{0} \cdot \left(2 rot_{w} rot_{x} - 2 rot_{y} rot_{z}\right) + vec_{2} \cdot \left(2 rot_{w} rot_{z} + 2 rot_{x} rot_{y}\right) & vec_{0} \left(rot_{w}^{2} - rot_{x}^{2} + rot_{y}^{2} - rot_{z}^{2}\right) + vec_{1} \left(- 2 rot_{w} rot_{z} - 2 rot_{x} rot_{y}\right) & 0 & 0 & 0 & 0 & 0 & 0 & 2 rot_{w} rot_{z} + 2 rot_{x} rot_{y} & - 2 rot_{x}^{2} - 2 rot_{z}^{2} + 1 & - 2 rot_{w} rot_{x} + 2 rot_{y} rot_{z}\\0 & vec_{1} \left(rot_{w}^{2} - rot_{x}^{2} - rot_{y}^{2} + rot_{z}^{2}\right) + vec_{2} \left(- 2 rot_{w} rot_{x} - 2 rot_{y} rot_{z}\right) & vec_{0} \left(- rot_{w}^{2} + rot_{x}^{2} + rot_{y}^{2} - rot_{z}^{2}\right) + vec_{2} \left(- 2 rot_{w} rot_{y} + 2 rot_{x} rot_{z}\right) & vec_{0} \cdot \left(2 rot_{w} rot_{x} + 2 rot_{y} rot_{z}\right) + vec_{1} \cdot \left(2 rot_{w} rot_{y} - 2 rot_{x} rot_{z}\right) & 0 & 0 & 0 & 0 & 0 & 0 & - 2 rot_{w} rot_{y} + 2 rot_{x} rot_{z} & 2 rot_{w} rot_{x} + 2 rot_{y} rot_{z} & - 2 rot_{x}^{2} - 2 rot_{y}^{2} + 1\\0 & 0 & 0 & 0 & vec_{1} \cdot \left(2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z}\right) + vec_{2} \cdot \left(2 pose.R_{w} pose.R_{z} - 2 pose.R_{x} pose.R_{y}\right) & vec_{0} \left(- 2 pose.R_{w} pose.R_{y} - 2 pose.R_{x} pose.R_{z}\right) + vec_{2} \left(pose.R_{w}^{2} + pose.R_{x}^{2} - pose.R_{y}^{2} - pose.R_{z}^{2}\right) & vec_{0} \left(- 2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y}\right) + vec_{1} \left(- pose.R_{w}^{2} - pose.R_{x}^{2} + pose.R_{y}^{2} + pose.R_{z}^{2}\right) & 1 & 0 & 0 & - 2 pose.R_{y}^{2} - 2 pose.R_{z}^{2} + 1 & - 2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y} & 2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z}\\0 & 0 & 0 & 0 & vec_{1} \left(- 2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z}\right) + vec_{2} \left(- pose.R_{w}^{2} + pose.R_{x}^{2} - pose.R_{y}^{2} + pose.R_{z}^{2}\right) & vec_{0} \cdot \left(2 pose.R_{w} pose.R_{x} - 2 pose.R_{y} pose.R_{z}\right) + vec_{2} \cdot \left(2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y}\right) & vec_{0} \left(pose.R_{w}^{2} - pose.R_{x}^{2} + pose.R_{y}^{2} - pose.R_{z}^{2}\right) + vec_{1} \left(- 2 pose.R_{w} pose.R_{z} - 2 pose.R_{x} pose.R_{y}\right) & 0 & 1 & 0 & 2 pose.R_{w} pose.R_{z} + 2 pose.R_{x} pose.R_{y} & - 2 pose.R_{x}^{2} - 2 pose.R_{z}^{2} + 1 & - 2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z}\\0 & 0 & 0 & 0 & vec_{1} \left(pose.R_{w}^{2} - pose.R_{x}^{2} - pose.R_{y}^{2} + pose.R_{z}^{2}\right) + vec_{2} \left(- 2 pose.R_{w} pose.R_{x} - 2 pose.R_{y} pose.R_{z}\right) & vec_{0} \left(- pose.R_{w}^{2} + pose.R_{x}^{2} + pose.R_{y}^{2} - pose.R_{z}^{2}\right) + vec_{2} \left(- 2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z}\right) & vec_{0} \cdot \left(2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z}\right) + vec_{1} \cdot \left(2 pose.R_{w} pose.R_{y} - 2 pose.R_{x} pose.R_{z}\right) & 0 & 0 & 1 & - 2 pose.R_{w} pose.R_{y} + 2 pose.R_{x} pose.R_{z} & 2 pose.R_{w} pose.R_{x} + 2 pose.R_{y} pose.R_{z} & - 2 pose.R_{x}^{2} - 2 pose.R_{y}^{2} + 1\end{array}\right]$