rail.projects.catalog_template module

class rail.projects.catalog_template.RailProjectCatalogInstance(**kwargs)

Bases: Configurable

Simple class for holding information need to make a coherent catalog of files using a templated file name and iteration_vars to fill in the interpolation in the file name.

For example the path_template might be ‘a_file/{healpix}/data.parqut’ and the interation_vars would be [‘healpix’].

When called with a dict such as healpix : [3433, 3344] it would the path_template would get expanded out to two files:

a_file/3433/data.parqut a_file/3344/data.parqut

Parameters:

kwargs (Any)

check_files(**kwargs)

Check if the files in the catalog exist

Return type:

list[bool]

Parameters:

**kwargs (dict[str, Any]) – Set of interpolants and iteration_vars needed to resolve the catalog

Returns:

List of True/False values for existance of each file in catalog

Return type:

list[bool]

Notes

By default this will used cached values, to override this and force rechecking use update=True keyword argument

config_options: dict[str, StageParameter] = {'iteration_vars': Parameter(Variables to iterate over to construct catalog, type: <class 'list'>, default: [] [optional]), 'name': Parameter(Dataset name, type: <class 'str'>, default: None [required]), 'path_template': Parameter(Template for path to catalog files, type: <class 'str'>, default: None [required])}

resolve(**kwargs)

Resolve the list of files in this catalog

Return type:

list[str]

Parameters:

**kwargs (dict[str, Any]) – Set of interpolants and iteration_vars needed to resolve the catalog

Returns:

List of resolved catalog files

Return type:

list[str]

Notes

By default this will used cached values, to override this and force rechecking use update=True keyword argument

yaml_tag: str = 'CatalogInstance'

class rail.projects.catalog_template.RailProjectCatalogTemplate(**kwargs)

Bases: Configurable

Simple class for holding a template for a catalog associated with a project

The makes a coherent catalog of files using a templated file name, interpolants, and iteration_vars to fill in the interpolation in the file name.

For example the path_template might be ‘a_file/{healpix}/{flavor}_data.hdf5’ and the interpolants would be [‘flavor’] and interation_vars would be [‘healpix’].

When called with a dict such as flavor: ‘baseline, healpix : [3433, 3344] it would the path_template would get expanded out to two files:

a_file/3433/baseline_data.hdf5 a_file/3344/baseline_data.hdf5

Parameters:

kwargs (Any)

config_options: dict[str, StageParameter] = {'iteration_vars': Parameter(Variables to iterate over to construct catalog, type: <class 'list'>, default: [] [optional]), 'name': Parameter(Dataset name, type: <class 'str'>, default: None [required]), 'path_template': Parameter(Template for path to catalog files, type: <class 'str'>, default: None [required])}

make_catalog_instance(name, **kwargs)

Make and return a specific instance of this CatalogTemplate by resolving interpolants and iterating over the iteration_vars.

Return type:

RailProjectCatalogInstance

Parameters:

• name (str) – Name for the CatalogInstance object

• **kwargs (dict[str, Any]) – Interpolants needed to resolve the path template

Returns:

Newly created object

Return type:

RailProjectCatalogInstance

yaml_tag: str = 'CatalogTemplate'