Utils#

Dictionaries#

class NoDatesSafeLoader(stream)[source]#

Bases: SafeLoader

Custom override of yaml Loader class for datetime considerations.

Initialize the scanner.

classmethod remove_implicit_resolver(tag_to_remove)[source]#

Remove implicit resolvers for a particular tag.

Takes care not to modify resolvers in super classes. Solution taken from https://stackoverflow.com/a/37958106/11483674 We want to load datetimes as strings, not dates, because we go on to serialise as jsonwhich doesn’t have the advanced types of yaml, and leads to incompatibilities down the track.

load_dict_from_file(file_path: FilePathType) → dict[source]#: Safely load metadata from .yml or .json files.

exist_dict_in_list(d, ls)[source]#: Check if an identical dictionary exists in the list.

append_replace_dict_in_list(ls, d, compare_key, list_dict_deep_update: bool = True, remove_repeats: bool = True)[source]#

Update the list ls with the dict d.

Cases: 1. If d is a dict and ls a list of dicts and ints/str, then for a given compare key, if for any element of ls

(which is a dict) say: ls[3][compare_key] == d[compare_key], then it will dict_deep_update these instead of appending d to list ls. Only if compare_key is not present in any of dicts in the list ls, then d is simply appended to ls.

If d is of immutable types like str, int etc., the ls is either appended with d or not. This depends on the value of remove_repeats. If remove_repeats is False, then ls is always appended with d. If remove_repeats is True, then if value d is present then it is not appended else it is.

Parameters

ls (list) – list of a dicts or int/str or a combination. This is the object to update
d (list/str/int) – this is the object from which ls is updated.
compare_key (str) – name of the key for which to check the presence of dicts in ls which need dict_deep_update
list_dict_deep_update (bool) – whether to update a dict in ls with compare_key present OR simply replace it.
remove_repeats (bool) – keep repeated values in the updated ls

Returns

ls – updated list

Return type

list

dict_deep_update(d: Mapping, u: Mapping, append_list: bool = True, remove_repeats: bool = True, copy: bool = True, compare_key: str = 'name', list_dict_deep_update: bool = True) → Mapping[source]#

Perform an update to all nested keys of dictionary d(input) from dictionary u(updating dict).

Parameters

d (dict) – dictionary to update
u (dict) – dictionary to update from
append_list (bool) – if the item to update is a list, whether to append the lists or replace the list in d e.g. d = dict(key1=[1,2,3]), u = dict(key1=[3,4,5]). If True then updated dictionary d=dict(key1=[1,2,3,4,5]) else d=dict(key1=[3,4,5])
remove_repeats (bool) – for updating list in d[key] with list in u[key]: if true then remove repeats: list(set(ls))
copy (bool) – whether to deepcopy the input dict d

compare_key (str) – the key that is used to compare dicts (and perform update op) and update d[key] when it is a list if dicts. example:

>>> d = {
    [
        {"name": "timeseries1", "desc": "desc1 of d", "starting_time": 0.0},
        {"name": "timeseries2", "desc": "desc2"},
    ]
}
>>> u = [{"name": "timeseries1", "desc": "desc2 of u", "unit": "n.a."}]
>>> # if compare_key='name' output is below
>>> output = [
    {"name": "timeseries1", "desc": "desc2 of u", "starting_time": 0.0, "unit": "n.a."},
    {"name": "timeseries2", "desc": "desc2"},
]
>>> # else the output is:
>>> # dict with the same key will be updated instead of being appended to the list
>>> output = [
    {"name": "timeseries1", "desc": "desc1 of d", "starting_time": 0.0},
    {"name": "timeseries2", "desc": "desc2"},
    {"name": "timeseries1", "desc": "desc2 of u", "unit": "n.a."},
]

list_dict_deep_update (bool) – for back compatibility, if False, this would work as before: example: if True then for the compare_key example, the output would be:

>>> output = [
    {"name": "timeseries1", "desc": "desc2 of u", "starting_time": 0.0, "unit": "n.a."},
    {"name": "timeseries2", "desc": "desc2"},
]
>>> # if False:
>>> output = [
    {"name": "timeseries1", "desc": "desc2 of u", "starting_time": 0.0},
    {"name": "timeseries2", "desc": "desc2"},
]  # unit key is absent since it is a replacement

Returns

d – return the updated dictionary

Return type

dict

class DeepDict(*args: Any, **kwargs: Any) → None[source]#

Bases: defaultdict

A defaultdict of defaultdicts

to_dict() → dict[source]#: Turn a DeepDict into a normal dictionary

__deepcopy__(memodict={})[source]#

Parameters: memodict (dict) – unused
Return type: DeepDict

JSON Schema#

class NWBMetaDataEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]#

Bases: JSONEncoder

Constructor for JSONEncoder, with sensible defaults.

If skipkeys is false, then it is a TypeError to attempt encoding of keys that are not str, int, float or None. If skipkeys is True, such items are simply skipped.

If ensure_ascii is true, the output is guaranteed to be str objects with all incoming non-ASCII characters escaped. If ensure_ascii is false, the output can contain non-ASCII characters.

If check_circular is true, then lists, dicts, and custom encoded objects will be checked for circular references during encoding to prevent an infinite recursion (which would cause an RecursionError). Otherwise, no such check takes place.

If allow_nan is true, then NaN, Infinity, and -Infinity will be encoded as such. This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders. Otherwise, it will be a ValueError to encode such floats.

If sort_keys is true, then the output of dictionaries will be sorted by key; this is useful for regression tests to ensure that JSON serializations can be compared on a day-to-day basis.

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines. None is the most compact representation.

If specified, separators should be an (item_separator, key_separator) tuple. The default is (’, ‘, ‘: ‘) if indent is None and (‘,’, ‘: ‘) otherwise. To get the most compact JSON representation, you should specify (‘,’, ‘:’) to eliminate whitespace.

If specified, default is a function that gets called for objects that can’t otherwise be serialized. It should return a JSON encodable version of the object or raise a TypeError.

default(obj)[source]#

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

get_base_schema(tag: Optional[str] = None, root: bool = False, id_: Optional[str] = None, required: Optional[List] = None, properties: Optional[Dict] = None, **kwargs) → dict[source]#: Return the base schema used for all other schemas.

get_schema_from_method_signature(method: Callable, exclude: Optional[list] = None) → dict[source]#

Take a class method and return a json-schema of the input args.

Parameters

method (function)
exclude (list, optional)

Return type

dict

fill_defaults(schema: dict, defaults: dict, overwrite: bool = True)[source]#

Insert the values of the defaults dict as default values in the schema in place.

Parameters

schema (dict)
defaults (dict)
overwrite (bool)

unroot_schema(schema: dict)[source]#

Modify a json-schema dictionary to make it not root.

Parameters: schema (dict)

get_schema_from_hdmf_class(hdmf_class)[source]#: Get metadata schema from hdmf class.

validate_metadata(metadata: Dict[str, dict], schema: Dict[str, dict], verbose: bool = False)[source]#: Validate metadata against a schema.

Utils#

Dictionaries#

JSON Schema#

Common Reused Types#