imsegm.utilities.experiments module

Framework for general experiments

Copyright (C) 2014-2018 Jiri Borovec <jiri.borovec@fel.cvut.cz>

class imsegm.utilities.experiments.Experiment(params, time_stamp=True)

Bases: object

Basic experiment class

Example

>>> import shutil
>>> params = {'path_out': './my_experiments', 'name': 'My-Sample'}
>>> expt = Experiment(params)
Traceback (most recent call last):
...
FileNotFoundError: given folder "./my_experiments" does not exist!
>>> os.mkdir(params['path_out'])
>>> expt = Experiment(params, time_stamp=False)
>>> expt.run()
>>> params = expt.params.copy()
>>> del expt
>>> shutil.rmtree(params['path_out'], ignore_errors=True)

constructor

Parameters

• params (dict) – define experimental parameters

• time_stamp (bool) – add to experiment unique time stamp

_check_exist_paths()

Check all required paths in parameters whether they exist

_create_folder(time_stamp=True)

Create the experiment folder and iterate while there is no available

Parameters

time_stamp (bool) – mark if you want an unique folder per experiment

_evaluate()

_load_data(gt=True)

loading the experiment data

Parameters

gt (bool) – try to load ground truth

_perform()

_summarise()

run(gt=True)

run the main Experimental body

Parameters

gt (bool) – try to load Ground Truth

class imsegm.utilities.experiments.WrapExecuteSequence(wrap_func, iterate_vals, nb_workers=2, desc='', ordered=False)

Bases: object

wrapper for execution paralle of single thread as for …

Example

>>> it = WrapExecuteSequence(lambda x: (x, x ** 2), range(5), nb_workers=1, ordered=True)
>>> list(it)
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]
>>> it = WrapExecuteSequence(sum, [[0, 1]] * 5, nb_workers=2, desc=None)
>>> [o for o in it]
[1, 1, 1, 1, 1]
>>> it = WrapExecuteSequence(min, ([0, 1] for i in range(5)))
>>> [o for o in it]
[0, 0, 0, 0, 0]

the init of this wrapper fro parallelism

Parameters

• wrap_func – function which will be excited in the iterations

• iterate_vals ([]) – list or iterator which will ide in iterations

• nb_workers (int) – number og jobs running in parallel

• desc (str) – deception for the bar, if it is set None, bar is suppressed

• ordered (bool) – whether enforce ordering in the parallelism

imsegm.utilities.experiments.append_final_stat(out_dir, y_true, y_pred, time_sec, file_name='resultStat.txt')

append (export) statistic to existing default file

Parameters

• out_dir (str) –

• y_true (list(int)) – annotation

• y_pred (list(int)) – predictions

• time_sec (int) –

• file_name (str) –

Return str

>>> import numpy as np
>>> np.random.seed(0)
>>> y_true = np.random.randint(0, 2, 25)
>>> y_pred = np.random.randint(0, 2, 25)
>>> f_path = append_final_stat('.', y_true, y_pred, 256)
>>> os.path.exists(f_path)
True
>>> os.remove(f_path)

imsegm.utilities.experiments.create_experiment_folder(params, dir_name, stamp_unique=True, skip_load=True)

create the experiment folder and iterate while there is no available

Parameters

• params (dict) – configuration

• dir_name (str) – folder name

• stamp_unique (bool) – use unique timestamp

• skip_load (bool) – skip loading folder params

Return dict

>>> import shutil
>>> import pandas as pd
>>> p = {'path_out': '.'}
>>> p = create_experiment_folder(p, 'my_test', False, skip_load=True)
>>> pd.Series(p).sort_index()  
computer                   [...
path_exp     ...my_test_EXAMPLE
path_out                      .
dtype: object
>>> p = create_experiment_folder(p, 'my_test', False, skip_load=False)
>>> shutil.rmtree(p['path_exp'], ignore_errors=True)
>>> p = create_experiment_folder(p, 'my_test', stamp_unique=True)
>>> pd.Series(p).sort_index()  
computer                         [...
path_exp   ...my_test_EXAMPLE_...-...
path_out                            .
dtype: object
>>> shutil.rmtree(p['path_exp'], ignore_errors=True)

imsegm.utilities.experiments.create_subfolders(path_out, folders)

create subfolders in rood directory

Parameters

• path_out (str) – root dictionary

• folders (list(str)) – list of subfolders

Return int

>>> import shutil
>>> dir_name = 'sample_dir'
>>> create_subfolders('.', [dir_name])
1
>>> os.path.exists(dir_name)
True
>>> shutil.rmtree(dir_name, ignore_errors=True)

imsegm.utilities.experiments.extend_list_params(params, name_param, options)

extend the parameter list by all sub-datasets

Parameters

• params (list(dict)) – list of parameters

• name_param (str) – parameter name

• options ([]) – lost of options

Return list(dict)

>>> import pandas as pd
>>> params = extend_list_params([{'a': 1}], 'a', [3, 4])
>>> pd.DataFrame(params)[sorted(pd.DataFrame(params))]  
   a param_idx
0  3     a-2#1
1  4     a-2#2
>>> params = extend_list_params([{'a': 1}], 'b', 5)
>>> pd.DataFrame(params)[sorted(pd.DataFrame(params))]  
   a  b param_idx
0  1  5     b-1#1

imsegm.utilities.experiments.get_nb_workers(ratio)

get fraction of of available CPUs

Parameters

ratio (float) – range (0, 1)

Return int

number of workers with lower bound 1

>>> get_nb_workers(0)
1

imsegm.utilities.experiments.is_iterable(var)

check if the variable is iterable

Parameters

var –

Return bool

>>> is_iterable('abc')
False
>>> is_iterable([0])
True
>>> is_iterable((1, ))
True

imsegm.utilities.experiments.load_config_yaml(path_config)

loading the

Parameters

path_config (str) –

Return dict

>>> p_conf = './testing-congif.yaml'
>>> save_config_yaml(p_conf, {'a': 2})
>>> load_config_yaml(p_conf)
{'a': 2}
>>> os.remove(p_conf)

imsegm.utilities.experiments.save_config_yaml(path_config, config)

exporting configuration as YAML file

Parameters

• path_config (str) –

• config (dict) –

imsegm.utilities.experiments.set_experiment_logger(path_out, file_name='logging.txt', reset=True)

set the logger to file

imsegm.utilities.experiments.string_dict(d, offset=30, desc='DICTIONARY')

transform dictionary to a formatted string

Parameters

• d (dict) –

• offset (int) – length between name and value

• desc (str) – dictionary title

Return str

>>> string_dict({'abc': 123})  
'DICTIONARY: \n"abc": 123'

imsegm.utilities.experiments.try_decorator(func)

costume decorator to wrap function in try/except

Parameters

func –

Returns

imsegm.utilities.experiments.CONFIG_YAML = 'config.yml'

default file for loading/exporting experiment configuration

imsegm.utilities.experiments.CPU_COUNT = 2

total number of avalaible CPUs/treads

imsegm.utilities.experiments.FILE_LOGS = 'logging.txt'

default file for streaming experimeny messages

imsegm.utilities.experiments.FORMAT_DT = '%Y%m%d-%H%M%S'

default date-time format

imsegm.utilities.experiments.RESULTS_TXT = 'resultStat.txt'

default name of file for exporting statistics