nunavut

Code generator built on top of pydsdl.

Nunavut uses pydsdl to generate text files using templates. While these text files are often source code this module could also be used to generate documentation or data interchange formats like JSON or XML.

The input to the nunavut library is a list of templates and a list of pydsdl.pydsdl.CompositeType objects. The latter is typically obtained by calling pydsdl:

from pydsdl import read_namespace

compound_types = read_namespace(root_namespace, include_paths)

Next a nunavut.LanguageContext is needed which is used to configure all Nunavut objects for a specific target language

from nunavut import LanguageContextBuilder

# Here we are going to generate C headers.
language_context = LanguageContextBuilder().set_target_language("c").create()

nunavut.AbstractGenerator objects require a nunavut.Namespace tree which can be built from the pydsdl type map using nunavut.build_namespace_tree():

from nunavut import build_namespace_tree

root_namespace = build_namespace_tree(compound_types,
                                      root_ns_folder,
                                      out_dir,
                                      language_context)

Putting this all together, the typical use of this library looks something like this:

from pydsdl import read_namespace
from nunavut import build_namespace_tree
from nunavut.lang import LanguageContextBuilder
from nunavut.jinja import DSDLCodeGenerator

# parse the dsdl
compound_types = read_namespace(root_namespace, include_paths)

# select a target language
language_context = LanguageContextBuilder().set_target_language("c").create()


# build the namespace tree
root_namespace = build_namespace_tree(compound_types,
                                      root_ns_folder,
                                      out_dir,
                                      language_context)

# give the root namespace to the generator and...
generator = DSDLCodeGenerator(root_namespace)

# generate all the code!
generator.generate_all()

class nunavut.AbstractGenerator(namespace: Namespace, generate_namespace_types: YesNoDefault = YesNoDefault.DEFAULT)[source]

Bases: object

Abstract base class for classes that generate source file output from a given pydsdl parser result.

Parameters:

namespace (nunavut.Namespace) – The top-level namespace to generates types at and from.
generate_namespace_types (YesNoDefault) – Set to YES to force generation files for namespaces and NO to suppress. DEFAULT will generate namespace files based on the language preference.

property namespace: Namespace: The root nunavut.Namespace for this generator.

property generate_namespace_types: bool: If true then the generator is set to emit files for nunavut.Namespace in addition to the pydsdl datatypes. If false then only files for pydsdl datatypes will be generated.

abstract get_templates(omit_serialization_support: bool = False) → Iterable[Path][source]: Enumerate all templates found in the templates path. :param bool omit_serialization_support: If True then templates needed only for serialization will be omitted. :return: A list of paths to all templates found by this Generator object.

abstract generate_all(is_dryrun: bool = False, allow_overwrite: bool = True, omit_serialization_support: bool = False, embed_auditing_info: bool = False) → Iterable[Path][source]

Generates all output for a given nunavut.Namespace and using the templates found by this object.

Parameters:

is_dryrun (bool) – If True then no output files will actually be written but all other operations will be performed.
allow_overwrite (bool) – If True then the generator will attempt to overwrite any existing files it encounters. If False then the generator will raise an error if the output file exists and the generation is not a dry-run.
omit_serialization_support (bool) – If True then the generator will emit only types without additional serialization and deserialization support and logic.
embed_auditing_info – If True then additional information about the inputs and environment used to generate source will be embedded in the generated files at the cost of build reproducibility.

Returns:

0 for success. Non-zero for errors.

Raises:

PermissionError if allow_overwrite is False and the file exists.

nunavut.build_namespace_tree(types: List[CompositeType], root_namespace_dir: str, output_dir: str, language_context: LanguageContext) → Namespace[source]

Generates a nunavut.Namespace tree.

Given a list of pydsdl types, this method returns a root nunavut.Namespace. The root nunavut.Namespace is the top of a tree where each node contains references to nested nunavut.Namespace and to any pydsdl.CompositeType instances contained within the namespace.

Parameters:

types (list) – A list of pydsdl types.
root_namespace_dir (str) – A path to the folder which is the root namespace.
output_dir (str) – The base directory under which all generated files will be created.
language_context (nunavut.LanguageContext) – The language context to use when building nunavut.Namespace objects.

Returns:

The root nunavut.Namespace.

Return type:

nunavut.Namespace

class nunavut.CodeGenerator(namespace: Namespace, generate_namespace_types: YesNoDefault = YesNoDefault.DEFAULT, templates_dir: Path | List[Path] | None = None, support_templates_dir: Path | List[Path] | None = None, use_support_templates_dir: bool = False, followlinks: bool = False, trim_blocks: bool = False, lstrip_blocks: bool = False, additional_filters: Dict[str, Callable] | None = None, additional_tests: Dict[str, Callable] | None = None, additional_globals: Dict[str, Any] | None = None, post_processors: List[PostProcessor] | None = None, builtin_template_path: str = 'templates', package_name_for_templates: str | None = None, search_policy: ResourceSearchPolicy = ResourceSearchPolicy.FIND_ALL)[source]

Bases: AbstractGenerator

Abstract base class for all Generators that build source code using Jinja templates.

Parameters:

namespace (nunavut.Namespace) – The top-level namespace to generates code at and from.
generate_namespace_types (YesNoDefault) – Set to YES to emit files for namespaces. NO will suppress namespace file generation and DEFAULT will use the language’s preference.
templates_dir (Optional[Union[pathlib.Path,List[pathlib.Path]]]) – Directories containing jinja templates. These will be available along with any built-in templates provided by the target language. The templates at these paths will take precedence masking any built-in templates where the names are the same. See jinja2.ChoiceLoader for rules on the lookup hierarchy.
support_templates_dir (Optional[Union[pathlib.Path,List[pathlib.Path]]]) – Directories containing jinja templates for the support files. These will be available along with any built-in templates provided by the target language. The templates at these paths will take precedence masking any built-in templates where the names are the same. See jinja2.ChoiceLoader for rules on the lookup hierarchy.
use_support_templates_dir (bool) – If True use the ‘support_templates_dir’ param for jinja templates, otherwise use the ‘templates_dir’ param. Defaults to False.
followlinks (bool) – If True then symbolic links will be followed when searching for templates.
trim_blocks (bool) – If this is set to True the first newline after a block is removed (block, not variable tag!).
lstrip_blocks (bool) – If this is set to True leading spaces and tabs are stripped from the start of a line to a block. Defaults to False.
additional_filters (Dict[str, Callable]) – typing.Optional jinja filters to add to the global environment using the key as the filter name and the callable as the filter.
additional_tests (Dict[str, Callable]) – typing.Optional jinja tests to add to the global environment using the key as the test name and the callable as the test.
additional_globals (Dict[str, Any]) – typing.Optional objects to add to the template environment globals collection.
post_processors (Optional[List[nunavut.postprocessors.PostProcessor]]) – A list of nunavut.postprocessors.PostProcessor
builtin_template_path – If provided overrides the folder name under which built-in templates are loaded from within a target language’s package (i.e. ignored if no target language is specified). For example, if the target language is c and this parameter was set to foo then built-in templates would be loaded from nunavut.lang.c.foo.
package_name_for_templates – If provided then the generator will look for templates within the given package name if a template is not found under a given filesystem path first (see templates_dir parameter).

Raises:

RuntimeError – If any additional filter or test attempts to replace a built-in or otherwise already defined filter or test.

property dsdl_loader: DSDLTemplateLoader: The template loader used by this generator.

property language_context: LanguageContext: The language context used by this generator.

get_templates(omit_serialization_support: bool = False) → Iterable[Path][source]

Enumerate all templates found in the templates path. TEMPLATE_SUFFIX as the suffix for the filename.

Returns:: A list of paths to all templates found by this Generator object.

abstract generate_all(is_dryrun: bool = False, allow_overwrite: bool = True, omit_serialization_support: bool = False, embed_auditing_info: bool = False) → Iterable[Path][source]

Generates all output for a given nunavut.Namespace and using the templates found by this object.

Parameters:

is_dryrun (bool) – If True then no output files will actually be written but all other operations will be performed.
allow_overwrite (bool) – If True then the generator will attempt to overwrite any existing files it encounters. If False then the generator will raise an error if the output file exists and the generation is not a dry-run.
omit_serialization_support (bool) – If True then the generator will emit only types without additional serialization and deserialization support and logic.
embed_auditing_info – If True then additional information about the inputs and environment used to generate source will be embedded in the generated files at the cost of build reproducibility.

Returns:

0 for success. Non-zero for errors.

Raises:

PermissionError if allow_overwrite is False and the file exists.

class nunavut.DSDLCodeGenerator(namespace: Namespace, **kwargs: Any)[source]

Bases: CodeGenerator

CodeGenerator implementation that generates code for a given set of DSDL types.

static filter_yamlfy(value: Any) → str[source]

Filter to, optionally, emit a dump of the dsdl input as a yaml document. Available as yamlfy in all template environments.

Example:

/*
{{ T | yamlfy }}
*/

Result Example (truncated for brevity):

/*
!!python/object:pydsdl.StructureType
_attributes:
- !!python/object:pydsdl.Field
_serializable: !!python/object:pydsdl.UnsignedIntegerType
    _bit_length: 16
    _cast_mode: &id001 !!python/object/apply:pydsdl.CastMode
    - 0
_name: value
*/

Parameters:: value – The input value to parse as yaml.
Returns:: If a yaml parser is available, a pretty dump of the given value as yaml. If a yaml parser is not available then an empty string is returned.

filter_type_to_template(value: Any) → str[source]

Template for type resolution as a filter. Available as type_to_template in all template environments.

Example:

{%- for attribute in T.attributes %}
    {%* include attribute.data_type | type_to_template %}
    {%- if not loop.last %},{% endif %}
{%- endfor %}

Parameters:: value – The input value to change into a template include path.
Returns:: A path to a template named for the type with TEMPLATE_SUFFIX

filter_type_to_include_path(value: Any, resolve: bool = False) → str[source]

Emits an include path to the output target for a given type.

Example:

# include "{{ T.my_type | type_to_include_path }}"

Result Example:

# include “foo/bar/my_type.h”

Parameters:

value (Any) – The type to emit an include for.
resolve (bool) – If True the path returned will be absolute else the path will be relative to the folder of the root namespace.

Returns:

A string path to output file for the type.

static filter_typename(value: Any) → str[source]

Filters a given token as its type name. Available as typename in all template environments.

This example supposes that T.some_value == "some string"

Example:

{{ T.some_value | typename }}

Result Example:

str

Parameters:: value – The input value to filter into a type name.
Returns:: The __name__ of the python type.

static filter_alignment_prefix(offset: BitLengthSet) → str[source]

Provides a string prefix based on a given pydsdl.BitLengthSet.

# Given
B = pydsdl.BitLengthSet(32)

# and
template = '{{ B | alignment_prefix }}'

# outputs
rendered = 'aligned'

# Given
B = pydsdl.BitLengthSet(32)
B += 1

# and
template = '{{ B | alignment_prefix }}'

# outputs
rendered = 'unaligned'

Parameters:: offset (pydsdl.BitLengthSet) – A bit length set to test for alignment.
Returns:: ‘aligned’ or ‘unaligned’ based on the state of the offset argument.

static filter_bit_length_set(values: Iterable[int] | int | None) → BitLengthSet[source]: Convert an integer or a list of integers into a pydsdl.BitLengthSet.

static filter_remove_blank_lines(text: str) → str[source]

Remove blank lines from the supplied string. Lines that contain only whitespace characters are also considered blank.

# Given
text = '''123

456

789'''

# and
template = '{{ text | remove_blank_lines }}'

# then the black lines will be removed leaving...
rendered = '''123
456
789'''

static filter_bits2bytes_ceil(n_bits: int) → int[source]: Implements int(ceil(x/8)) | x >= 0.

static filter_text_table(data: Dict, start_each_line: str, column_sep: str = ' : ', line_end: str = '\n') → str[source]

Create a text table from a dictionary of data.

# Given
table = {
    "banana": "yellow",
    "apple": "red",
    "grape": "purple"
}

# and
template = '''
{{ table | text_table("//  ", " | ", "\n") }}'''

# then
rendered = '''
//  banana | yellow
//  apple  | red
//  grape  | purple'''

static is_None(value: Any) → bool[source]: Tests if a value is None

static is_saturated(t: PrimitiveType) → bool[source]: Tests if a type is a saturated type or not.

static is_service_request(instance: Any) → bool[source]: Tests if a type is request type of a service type.

static is_service_response(instance: Any) → bool[source]: Tests if a type is response type of a service type.

static is_deprecated(instance: Any) → bool[source]: Tests if a type is marked as deprecated

generate_all(is_dryrun: bool = False, allow_overwrite: bool = True, omit_serialization_support: bool = False, embed_auditing_info: bool = False) → Iterable[Path][source]

Generates all output for a given nunavut.Namespace and using the templates found by this object.

Parameters:

is_dryrun (bool) – If True then no output files will actually be written but all other operations will be performed.
allow_overwrite (bool) – If True then the generator will attempt to overwrite any existing files it encounters. If False then the generator will raise an error if the output file exists and the generation is not a dry-run.
omit_serialization_support (bool) – If True then the generator will emit only types without additional serialization and deserialization support and logic.
embed_auditing_info – If True then additional information about the inputs and environment used to generate source will be embedded in the generated files at the cost of build reproducibility.

Returns:

0 for success. Non-zero for errors.

Raises:

PermissionError if allow_overwrite is False and the file exists.

nunavut.generate_types(language_key: str, root_namespace_dir: Path, out_dir: Path, omit_serialization_support: bool = True, is_dryrun: bool = False, allow_overwrite: bool = True, lookup_directories: Iterable[str] | None = None, allow_unregulated_fixed_port_id: bool = False, language_options: Mapping[str, Any] | None = None, include_experimental_languages: bool = False, embed_auditing_info: bool = False) → None[source]

Helper method that uses default settings and built-in templates to generate types for a given language. This method is the most direct way to generate code using Nunavut.

Parameters:

language_key (str) – The name of the language to generate source for. See the Template Language Guide for details on available language support.
root_namespace_dir (pathlib.Path) – The path to the root of the DSDL types to generate code for.
out_dir (pathlib.Path) – The path to generate code at and under.
omit_serialization_support (bool) – If True then logic used to serialize and deserialize data is omitted.
is_dryrun (bool) – If True then nothing is generated but all other activity is performed and any errors that would have occurred are reported.
allow_overwrite (bool) – If True then generated files are allowed to overwrite existing files under the out_dir path.
lookup_directories (Optional[Iterable[str]]) – Additional directories to search for dependent types referenced by the types provided under the root_namespace_dir. Types will not be generated for these unless they are used by a type in the root namespace.
allow_unregulated_fixed_port_id (bool) – If True then errors will become warning when using fixed port identifiers for unregulated datatypes.
language_options (Optional[Mapping[str, Any]]) – Opaque arguments passed through to the language objects. The supported arguments and valid values are different depending on the language specified by the language_key parameter.
include_experimental_languages (bool) – If true then experimental languages will also be available.
embed_auditing_info – If True then additional information about the inputs and environment used to generate source will be embedded in the generated files at the cost of build reproducibility.

class nunavut.LanguageConfig[source]

Bases: object

Configuration storage encapsulating parsers and other configuration format details. For any configuration type used the concept of “sections” must be maintained which requires that the top-level configuration be structured as key/value pairs with the keys using the form “nunavut.lang.[language name]”. For example, yaml configuration must have a top-level structure like this:

example_yaml = '''
    nunavut.lang.a:
        key_one: value_one
        key_two: value_two
    nunavut.lang.b:
        key_one: value_one
        key_two: value_two
    nunavut.lang.c:
        key_one: value_one
        key_two: value_two
'''

Note

The “language name” part of the section identifier must not start with a number and can contain only alphanumeric characters. That is, the section identifier must match this pattern:

nunavut.lang.[a-zA-Z]{1}w*

The values of the section data can be anything:

example_yaml = '''
    nunavut.lang.d:
        key_one:
            - is
            - a
            - list:
                where: index2
                is: a_dictionary
'''

SECTION_NAME_PATTERN = re.compile('^nunavut\\.lang\\.([a-zA-Z]{1}\\w*)$'): Required pattern for section name identifiers.

update(configuration: Any) → None[source]

Add configuration data to this configuration from a string.

Unlike add_section, this method will update section data with existing keys.

For example, the first update is to an empty configuration so it will act as a simple insert operation:

…but updating this data is now possible where sections can be added and updated:

update_section(section_name: str, configuration: Any) → None[source]: Update a section of the configuration.

sections() → Dict[str, Dict[str, Any]][source]: Get all sections of the configuration.

update_from_yaml_string(string: str) → None[source]: Update the configuration from a yaml string. Calls update() with the parsed yaml data and will raise the same exceptions.

update_from_yaml_file(f: TextIO) → None[source]: Update the configuration from a yaml file. Calls update() with the parsed yaml data and will raise the same exceptions.

set(section: str, option: str, value: Any) → None[source]

Set a configuration value.

Parameters:

section – The section to set the value in.
option – The option to set.
value – The value to set.

add_section(section_name: str) → None[source]

Add a section to the configuration.

Sections are top-level containers that contain key/value pairs of configuration of a single language type.

Parameters:: section_name – The name of the language section. This must adhere to the SECTION_NAME_PATTERN pattern.

get_config_value(section_name: str, key: str, default_value: str | None = None) → str[source]

Get an optional language property from the language configuration.

:param section_name : The name of the section to get the value from. :param str key : The config value to retrieve. :param default_value: The value to return if the key was not in the configuration. If provided this method will not raise. :type default_value : typing.Optional[str] :return: Either the value from the config or the default_value if provided. :rtype: str :raises: KeyError if the section or the key in the section does not exist and a default_value was not provided.

get_config_value_as_bool(section_name: str, key: str, default_value: bool = False) → bool[source]

Get an optional language property from the language configuration returning a boolean. The rules for boolean conversion are as follows:

# "Any string" = True
config.set('nunavut.lang.cpp', 'v', 'Any string')
assert config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "true" = True
config.set('nunavut.lang.cpp', 'v', 'true')
assert config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "TrUe" = True
config.set('nunavut.lang.cpp', 'v', 'TrUe')
assert config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "1" = True
config.set('nunavut.lang.cpp', 'v', '1')
assert config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "false" = False
config.set('nunavut.lang.cpp', 'v', 'false')
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "FaLse" = False
config.set('nunavut.lang.cpp', 'v', 'FaLse')
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "0" = False
config.set('nunavut.lang.cpp', 'v', '0')
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# "" = False
config.set('nunavut.lang.cpp', 'v', '')
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'v')

# False if not defined
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'not_a_key')

# True if not defined but default_value is True
assert not config.get_config_value_as_bool('nunavut.lang.cpp', 'not_a_key')

:param section_name : The name of the section to get the value from. :param str key : The config value to retrieve. :param bool default_value : The value to use if no value existed. :return : The config value as either True or False. :rtype : bool

get_config_value_as_dict(section_name: str, key: str, default_value: Dict | None = None) → Dict[str, Any][source]

Get a language property parsing it as a map with string keys.

Example:

:param str section_name : The name of the section to get the key from. :param str key : The config value to retrieve. :param default_value : The value to return if the key was not in the configuration. If provided this method will not raise a KeyError nor a TypeError. :type default_value : typing.Optional[typing.Mapping[str, typing.Any]] :return : Either the value from the config or the default_value if provided. :rtype : typing.Mapping[str, typing.Any] :raises : KeyError if the key does not exist and a default_value was not provided. :raises : TypeError if the value exists but is not a dict and a default_value was not provided.

get_config_value_as_list(section_name: str, key: str, default_value: List | None = None) → List[Any][source]

Get a language property parsing it as a map with string keys.

Example:

:param str section_name : The name of the section to get the key from. :param str key : The config value to retrieve. :param default_value : The value to return if the key was not in the configuration. If provided this method will not raise a KeyError nor a TypeError. :type default_value : typing.Optional[typing.List[typing.Any]] :return : Either the value from the config or the default_value if provided. :rtype : typing.List[typing.Any] :raises : KeyError if the key does not exist and a default_value was not provided. :raises : TypeError if the value exists but is not a dict and a default_value was not provided.

class nunavut.DefaultValue(value: Any)[source]

Bases: object

Represents a default value in the language configuration. Use this to differentiate between explicit values and default values when merging configuration. For example, given the following configuration:

collection = {
    'a': DefaultValue(1),
    'b': 2
}

overrides = [
    {
        'a': 3,
        'b': DefaultValue(4)
    },
    {
        'a': DefaultValue(5),
        'b': 6
    }
]

Then the merged configuration should be:

merged = {
    'a': 3,
    'b': 6
}

Other properties of DefaultValue:

assert DefaultValue(1) == 1
assert DefaultValue(1) != 2
assert DefaultValue(1) == DefaultValue(1)
assert DefaultValue(1) != DefaultValue(2)
assert eval(repr(DefaultValue(1))) == DefaultValue(1)
assert hash(DefaultValue(1)) == hash(1)
assert bool(DefaultValue(1))
assert not bool(DefaultValue(None))
repred = eval(repr(DefaultValue(8)))
assert repred.value == 8

classmethod assign_to_if_not_default(target: MutableMapping[str, Any], key: str, value: Any) → Any[source]

Assigns a value to a key in a dictionary unless the key already has a value and the value is not a DefaultValue. The one exception to this is if the value is a DefaultValue and the value for the key is already a DefaultValue. In this case the new DefaultValue value will be assigned to the key.

Parameters:

target – The dictionary to assign to.
key – The key to assign to.
value – The value to test and assign.

Returns:

The value assigned to the key. This is the value of the value parameter if it was assigned or the value of the key in the target dictionary if it was not assigned.

property value: Any: The default value.

class nunavut.Language(language_module_name: str, config: LanguageConfig, **kwargs: Any)[source]

Bases: object

Facilities for generating source code for a specific language. Concrete Language classes must be implemented by the language support package below lang and should be instantiated using nunavut.lang.LanguageClassLoader.

Parameters:

module_name (str) – The name of the nunavut.lang module that contains the concrete language type and its resources.
config (LanguageConfig) – All configuration as defined by the properties.yaml schema.
kwargs –

Opaque arguments passed through to the target
nunavut.lang.Language object. See all “WKLA” constants on this class for well-known keyword arguments.

classmethod default_filter_id_for_target(instance: Any) → str[source]

The default transformation of any object into a string.

Parameters:: instance (any) – Any object or data that either has a name property or can be converted to a string.
Returns:: Either str(instance.name) if the instance has a name property or just str(instance)

property extension: str: The extension to use for files generated in this language.

property namespace_output_stem: str | None: The name of a namespace file for this language.

property name: str: The name of the language used by the nunavut.lang module.

property support_namespace: List[str]: The hierarchical namespace used by the support software. The property is a dot separated string when specified in configuration. This property returns that value split into namespace components with the first identifier being the first index in the array, etc.

property enable_stropping: bool: Whether or not to strop identifiers for this language.

property has_standard_namespace_files: bool: Whether or not the language defines special namespace files as part of its core standard (e.g. python’s __init__).

property stable_support: bool: Whether support for this language is designated ‘stable’, and not experimental.

property named_types: Mapping[str, str]: Get a map of named types to the type name to emit for this language.

property named_values: Mapping[str, str]: Get a map of named values to the token to emit for this language.

get_support_module() → Tuple[str, Tuple[int, int, int], module | None][source]: Returns the module object for the language support files. :return: A tuple of module name, x.y.z module version, and the module object itself.

get_dependency_builder(for_type: Any) → DependencyBuilder[source]: Get a dependency builder for the given type.

abstract get_includes(dep_types: Dependencies) → List[str][source]: Get a list of include paths that are specific to this language and the options set for it. :param Dependencies dep_types: A description of the dependencies includes are needed for. :return: A list of include file paths. The list may be empty if no includes were needed.

filter_id(instance: Any, id_type: str = 'any') → str[source]

Produces a valid identifier in the language for a given object. The encoding may not be reversible.

Parameters:

instance (any) – Any object or data that either has a name property or can be converted to a string.
id_type (str) – A type of identifier. This is different for each language. For example, for C this value can be ‘typedef’, ‘macro’, ‘function’, or ‘enum’. Use ‘any’ to apply stropping rules for all identifier types to the instance.

Returns:

A token that is a valid identifier in the language, is not a reserved keyword, and is transformed in a deterministic manner based on the provided instance.

filter_short_reference_name(t: CompositeType, stropping: YesNoDefault = YesNoDefault.DEFAULT, id_type: str = 'any') → str[source]

Provides a string that is a shorted version of the full reference name omitting any namespace parts of the type.

Parameters:

t (pydsdl.CompositeType) – The DSDL type to get the reference name for.
stropping (YesNoDefault) – If DEFAULT then the stropping value configured for the target language is used else this overrides that value.
id_type (str) – A type of identifier. This is different for each language. For example, for C this value can be ‘typedef’, ‘macro’, ‘function’, or ‘enum’. Use ‘any’ to apply stropping rules for all identifier types to the instance.

get_config_value(key: str, default_value: str | None = None) → str[source]

Get an optional language property from the language configuration.

Parameters:

key (str) – The config value to retrieve.
default_value (Optional[str]) – The value to return if the key was not in the configuration. If provided this method will not raise.

Returns:

Either the value from the config or the default_value if provided.

Return type:

str

Raises:

KeyError if the section or the key in the section does not exist and a default_value was not provided.

get_config_value_as_bool(key: str, default_value: bool = False) → bool[source]

Get an optional language property from the language configuration returning a boolean.

Parameters:

key (str) – The config value to retrieve.
default_value (bool) – The value to use if no value existed.

Returns:

The config value as either True or False.

Return type:

bool

get_config_value_as_dict(key: str, default_value: Dict | None = None) → Dict[str, Any][source]

Get a language property parsing it as a map with string keys.

Parameters:

key (str) – The config value to retrieve.
default_value (Optional[Mapping[str, Any]]) – The value to return if the key was not in the configuration. If provided this method will not raise a KeyError nor a TypeError.

Returns:

Either the value from the config or the default_value if provided.

Return type:

Mapping[str, Any]

Raises:

KeyError if the key does not exist and a default_value was not provided.

Raises:

TypeError if the value exists but is not a dict and a default_value was not provided.

get_config_value_as_list(key: str, default_value: List | None = None) → List[Any][source]

Get a language property parsing it as a map with string keys.

Parameters:

key (str) – The config value to retrieve.
default_value (Optional[List[Any]]) – The value to return if the key was not in the configuration. If provided this method will not raise a KeyError nor a TypeError.

Returns:

Either the value from the config or the default_value if provided.

Return type:

List[Any]

Raises:

KeyError if the key does not exist and a default_value was not provided.

Raises:

TypeError if the value exists but is not a dict and a default_value was not provided.

get_support_files(resource_type: ResourceType = ResourceType.ANY) → Generator[Path, None, None][source]

Iterates over supporting files embedded within the Nunavut distribution.

Parameters:: resource_type – The type of support resources to enumerate.

get_option(option_key: str, default_value: Mapping[str, Any] | str | None = None) → Mapping[str, Any] | str | None[source]

Get a language option for this language.

# Values can come from defaults...
assert lang_cpp.get_option('target_endianness') == 'little'

# ... or can come from a sane default.
assert lang_cpp.get_option('foobar', 'sane_default') == 'sane_default'

Returns:: Either the value provided to the nunavut.lang.Language instance, the value from properties.yaml or the default_value.

get_templates_package_name() → str[source]: The name of the nunavut python package containing filters, types, and configuration for this language.

get_globals() → Mapping[str, Any][source]

Get all values for this language that should be available in a global context.

Returns:: A mapping of global names to global values.

get_options() → Mapping[str, Any][source]

Get all language options for this Language.

Returns:: A mapping of option names to option values.

class nunavut.LanguageContext(language_configuration: LanguageConfig, target_language: Language, supported_language_builder: Callable[[], Dict[str, Language]])[source]

Bases: object

Context object containing the current target language and all supported nunavut.lang.Language objects.

Parameters:

language_configuration – The configuration for all languages as defined by the properties.yaml schema.
target_language – The target language.
supported_language_builder – factory closure that will create nunavut.lang.Language objects for all supported languages when LanguageContext.get_target_languages() is first called.

get_language(key_or_module_name: str) → Language[source]

Get a nunavut.lang.Language object for a given language identifier.

Parameters:: key_or_module_name (str) – Either one of the Nunavut mnemonics for a supported language or the __name__ of one of the nunavut.lang.[language] python modules.
Returns:: A nunavut.lang.Language object cached by this context.
Return type:: nunavut.lang.Language

get_target_language() → Language[source]: Returns the target language for code generation.

filter_id_for_target(instance: Any, id_type: str = 'any') → str[source]

A filter that will transform a given string or pydsdl identifier into a valid identifier in the target language.

Parameters:

instance (any) – Any object or data that either has a name property or can be converted to a string.
id_type (str) – A type of identifier. This is different for each language. Use ‘any’ to apply stropping rules for all identifier types to the instance.

Returns:

A token that is a valid identifier in the target language, is not a reserved keyword, and is transformed in a deterministic manner based on the provided instance.

get_supported_languages() → Dict[str, Language][source]: Returns a collection of available language support objects.

property config: LanguageConfig: Returns the nunavut.lang.LanguageConfig object that contains the configuration for all supported languages. This is the same object that is used to instantiate the nunavut.lang.Language

class nunavut.LanguageContextBuilder(include_experimental_languages: bool = False)[source]

Bases: object

Used to instantiate new LanguageContext objects.

The simplest invocation will always work by using the LanguageContextBuilder.DEFAULT_TARGET_LANGUAGE constant:

from nunavut.lang import LanguageContextBuilder

default_language_context = LanguageContextBuilder().create()

assert LanguageContextBuilder.DEFAULT_TARGET_LANGUAGE == default_language_context.get_target_language().name

Typically a target language is specified at minimum. Also see constants on nunavut.lang.Language for well-known options that the builder can override:

from nunavut.lang import LanguageContextBuilder

customized_language_context = (
    LanguageContextBuilder()
        .set_target_language("c")
        .set_target_language_configuration_override(Language.WKCV_DEFINITION_FILE_EXTENSION, ".h")
        .create()
)

assert customized_language_context.get_target_language().extension == ".h"

Parameters:: include_experimental_languages – If set then languages that are not fully supported will be allowed otherwise any experimental languages will be missing and errors will be raised as if the language specified was unknown.

DEFAULT_TARGET_LANGUAGE = 'c': The target language used for new contexts if none is specified.

get_supported_language_names() → Iterable[str][source]

Get a list of target languages supported by Nunavut.

Returns:: An iterable of strings which are languages with special support within Nunavut templates.

property config: LanguageConfig: The configuration object that will be used to create the language context.

set_target_language_configuration_override(key: str, value: Any) → LanguageContextBuilder[source]

Stores a key and value to override in the configuration for a language target when a LanguageContext is crated. These overrides are always set under the language section of the target language.

builder = LanguageContextBuilder().set_target_language("c")

default_c_file_extension = builder.config.get_config_value(
                                LanguageClassLoader.to_language_module_name("c"),
                                Language.WKCV_DEFINITION_FILE_EXTENSION)

assert default_c_file_extension == ".h"

We can now try to override the file extension for a future “C” target language object:

builder.set_target_language_configuration_override(Language.WKCV_DEFINITION_FILE_EXTENSION, ".foo")

…but that value will not be overridden until you create the target language:

default_c_file_extension = builder.config.get_config_value(
                                LanguageClassLoader.to_language_module_name("c"),
                                Language.WKCV_DEFINITION_FILE_EXTENSION)

assert default_c_file_extension == ".h"

_ = builder.create()

overridden_c_file_extension = builder.config.get_config_value(
                                LanguageClassLoader.to_language_module_name("c"),
                                Language.WKCV_DEFINITION_FILE_EXTENSION)

assert overridden_c_file_extension == ".foo"

Note that the config is scoped by the builder but is then inherited by the language objects created by the builder:

one = (
    LanguageContextBuilder()
        .set_target_language("c")
        .set_target_language_configuration_override("foo", 1)
    )
two = (
    LanguageContextBuilder()
        .set_target_language("c")
        .set_target_language_configuration_override("foo", 2)
    )

# Here we see that the second override of "foo" does not affect the first because they
# are in different builders.

assert (
        one.create().get_target_language().get_config_value("foo")
        !=
        two.create().get_target_language().get_config_value("foo")
    )

set_target_language_extension(target_language_extension: str | None) → LanguageContextBuilder[source]

Helper method for setting the target language file extension (since this is a common override).

Calling this method is the same as doing:

LanguageContextBuilder().set_target_language_configuration_override(
    Language.WKCV_DEFINITION_FILE_EXTENSION,
    ".h")

set_target_language(target_language: str | None) → LanguageContextBuilder[source]

Set the language name to target. This can be either the name of the language, as defined by Nunavut, or it can be the language package name.

assert LanguageContextBuilder().set_target_language("c").create().get_target_language().name == "c"

assert (
    LanguageContextBuilder()
        .set_target_language(LanguageClassLoader.to_language_module_name("c"))
        .create()
        .get_target_language().name == "c"
)

Also note that, if the language name is None, the default name will be assigned internally:

target_language = LanguageContextBuilder().set_target_language(None).create().get_target_language()

assert target_language.name == LanguageContextBuilder.DEFAULT_TARGET_LANGUAGE

set_additional_config_files(additional_config_files: List[Path]) → LanguageContextBuilder[source]: Deprecated. Use add_config_files() instead.

add_config_files(*additional_config_files: Path) → LanguageContextBuilder[source]

A list of paths to additional yaml files to load as configuration. These will override any values found in the nunavut.lang.properties.yaml file and files appearing later in this list will override value found in earlier entries.

target_language_w_overrides = (
    LanguageContextBuilder()
        .set_target_language("c")
        .add_config_files(overrides_file)
        .create()
        .get_target_language()
)

target_language_no_overrides = (
    LanguageContextBuilder()
        .set_target_language("c")
        .create()
        .get_target_language()
)

assert target_language_w_overrides.extension == ".foo"
assert target_language_no_overrides.extension == ".h"

Overrides are applied as unions. For example, given this override data:

overrides_data = '''
    nunavut.lang.c:
        extension: .foo
        non-standard: bar
'''

…the standard “extension” property will be overridden and the “non-standard” property will be added.

target_language_w_overrides = (
    LanguageContextBuilder()
        .set_target_language("c")
        .add_config_files(second_overrides_file)
        .create()
        .get_target_language()
)

assert ".foo" == target_language_w_overrides.extension
assert "bar" == target_language_w_overrides.get_config_value("non-standard")

create() → LanguageContext[source]: Applies all pending configuration overrides to the internal LanguageConfig object and instantiates a LanguageContext object.

class nunavut.Namespace(full_namespace: str, root_namespace_dir: Path, base_output_path: PurePath, language_context: LanguageContext)[source]

Bases: Any

K-ary tree (where K is the largest set of data types in a single dsdl namespace) where the nodes represent dsdl namespaces and the children are the datatypes and other nested namespaces (with datatypes always being leaf nodes). This structure extends pydsdl.Any and is a pydsdl.pydsdl.CompositeType via duck typing.

Parameters:

full_namespace (str) – The full, dot-separated name of the namepace. This is expected to be a unique identifier.
root_namespace_dir (pathlib.Path) – The directory representing the dsdl namespace and containing the namespaces’s datatypes and nested namespaces.
base_output_path (pathlib.PurePath) – The base path under which all namespaces and datatypes should be generated.
language_context (LanguageContext) – The generated software language context the namespace is within.

property output_folder: Path: The folder where this namespace’s output file and datatypes are generated.

get_support_output_folder() → PurePath[source]: The folder under which support artifacts are generated.

get_language_context() → LanguageContext[source]: The generated software language context the namespace is within.

get_root_namespace() → Namespace[source]

Traverses the namespace tree up to the root and returns the root node.

Returns:: The root namespace object.

get_nested_namespaces() → Iterator[Namespace][source]: Get an iterator over all the nested namespaces within this namespace. This is a shallow iterator that only provides directly nested namespaces.

get_nested_types() → ItemsView[CompositeType, Path][source]: Get a view of a tuple relating datatypes in this namespace to the path for the type’s generated output. This is a shallow view including only the types directly within this namespace.

get_all_datatypes() → Generator[Tuple[CompositeType, Path], None, None][source]: Generates tuples relating datatypes at and below this namespace to the path for each type’s generated output.

get_all_namespaces() → Generator[Tuple[Namespace, Path], None, None][source]: Generates tuples relating nested namespaces at and below this namespace to the path for each namespace’s generated output.

get_all_types() → Generator[Tuple[Any, Path], None, None][source]: Generates tuples relating datatypes and nested namespaces at and below this namespace to the path for each type’s generated output.

find_output_path_for_type(any_type: Any) → Path[source]

Searches the entire namespace tree to find a mapping of the type to an output file path.

Parameters:: any_type (Any) – Either a Namespace or pydsdl.CompositeType to find the output path for.
Returns:: The path where a file will be generated for a given type.
Raises:: KeyError – If the type was not found in this namespace tree.

class nunavut.SupportGenerator(namespace: Namespace, **kwargs: Any)[source]

Bases: CodeGenerator

Generates output files by copying them from within the Nunavut package itself for non templates but uses jinja to generate headers from templates with the language environment provided but no T (DSDL type) global set. This generator always copies files from those returned by the file_iterator to locations under nunavut.Namespace.get_support_output_folder()

get_templates(omit_serialization_support: bool = False) → Iterable[Path][source]

Enumerate all templates found in the templates path. TEMPLATE_SUFFIX as the suffix for the filename.

Returns:: A list of paths to all templates found by this Generator object.

generate_all(is_dryrun: bool = False, allow_overwrite: bool = True, omit_serialization_support: bool = False, embed_auditing_info: bool = False) → Iterable[Path][source]

Generates all output for a given nunavut.Namespace and using the templates found by this object.

Parameters:

is_dryrun (bool) – If True then no output files will actually be written but all other operations will be performed.
allow_overwrite (bool) – If True then the generator will attempt to overwrite any existing files it encounters. If False then the generator will raise an error if the output file exists and the generation is not a dry-run.
omit_serialization_support (bool) – If True then the generator will emit only types without additional serialization and deserialization support and logic.
embed_auditing_info – If True then additional information about the inputs and environment used to generate source will be embedded in the generated files at the cost of build reproducibility.

Returns:

0 for success. Non-zero for errors.

Raises:

PermissionError if allow_overwrite is False and the file exists.

exception nunavut.UnsupportedLanguageError[source]

Bases: ValueError

Error type raised if an unsupported language type is used.

class nunavut.YesNoDefault(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Bases: Enum

Trinary type for decisions that allow a default behavior to be requested that can be different based on other contexts. For example:

def should_we_order_pizza(answer: YesNoDefault) -> bool:
    if answer == YesNoDefault.YES or (
       answer == YesNoDefault.DEFAULT and
       datetime.today().isoweekday() == 5):
        # if yes or if we are taking the default action which is to
        # order pizza on Friday, and today is Friday, then we order pizza
        return True
    else:
        return False

classmethod test_truth(ynd_value: YesNoDefault, default_value: bool) → bool[source]

Helper method to test a YesNoDefault value and return a default boolean value.

'''
    let "is YES" be Y
    let "is DEFAULT" be D where:
        if Y then not D and if D then not Y
        and "is NO" is Y = D = 0
    let "is default_value true" be d

    Y | D | d | Y or (D and d)
    1   *   *    1
    0   1   0    0
    0   1   1    1
    0   0   *    0
'''

assert YesNoDefault.test_truth(YesNoDefault.YES, False)
assert not YesNoDefault.test_truth(YesNoDefault.DEFAULT, False)
assert YesNoDefault.test_truth(YesNoDefault.DEFAULT, True)
assert not YesNoDefault.test_truth(YesNoDefault.NO, True)

exception nunavut.InternalError[source]

Bases: RuntimeError

Internal, opaque error within Nunavut.

This exception is a “should never happen” exception. If caught you’ve probably hit a bug. This is the only exception type within the library that can be use where no unit tests are covering the error (i.e. pragma: no cover branches).