imsegm.utilities.experiments module

Framework for general experiments

Copyright (C) 2014-2018 Jiri Borovec <jiri.borovec@fel.cvut.cz>

class imsegm.utilities.experiments.Experiment(params, time_stamp=True)[source]

Bases: object

Basic experiment class

Example

>>> import shutil
>>> params = {'path_out': './my_experiments', 'name': 'My-Sample'}
>>> expt = Experiment(params)
Traceback (most recent call last):
...
Exception: given folder "./my_experiments" does not exist!
>>> os.mkdir(params['path_out'])
>>> expt = Experiment(params, time_stamp=False)
>>> expt.run()
>>> params = expt.params.copy()
>>> del expt
>>> shutil.rmtree(params['path_out'], ignore_errors=True)

constructor

Parameters
  • params (dict) – define experimenatl parameters

  • time_stamp (bool) – add to experiment unique time stamp

_check_exist_paths()[source]

Check all required paths in parameters whether they exist

_create_folder(time_stamp=True)[source]

Create the experiment folder and iterate while there is no available

Parameters

time_stamp (bool) – mark if you want an unique folder per experiment

_evaluate()[source]
_load_data(gt=True)[source]

loading the experiment data

Parameters

gt (bool) – try to load ground truth

_perform()[source]
_summarise()[source]
run(gt=True)[source]

run the main Experimental body

Parameters

gt (bool) – try to load Ground Truth

class imsegm.utilities.experiments.WrapExecuteSequence(wrap_func, iterate_vals, nb_workers=1, desc='', ordered=False)[source]

Bases: object

wrapper for execution paralle of single thread as for …

Example

>>> it = WrapExecuteSequence(lambda x: (x, x ** 2), range(5),
...                          nb_workers=1, ordered=True)
>>> list(it)
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]
>>> it = WrapExecuteSequence(sum, [[0, 1]] * 5, nb_workers=2, desc=None)
>>> [o for o in it]
[1, 1, 1, 1, 1]
>>> it = WrapExecuteSequence(min, ([0, 1] for i in range(5)))
>>> [o for o in it]
[0, 0, 0, 0, 0]

the init of this wrapper fro parallelism

Parameters
  • wrap_func – function which will be excited in the iterations

  • iterate_vals ([]) – list or iterator which will ide in iterations

  • nb_workers (int) – number og jobs running in parallel

  • desc (str) – deception for the bar, if it is set None, bar is suppressed

  • ordered (bool) – whether enforce ordering in the parallelism

imsegm.utilities.experiments.append_final_stat(out_dir, y_true, y_pred, time_sec, file_name='resultStat.txt')[source]

append (export) statistic to existing default file

Parameters
  • out_dir (str) –

  • y_true ([int]) – annotation

  • y_pred ([int]) – predictions

  • time_sec (int) –

  • file_name (str) –

Return str

>>> import numpy as np
>>> np.random.seed(0)
>>> y_true = np.random.randint(0, 2, 25)
>>> y_pred = np.random.randint(0, 2, 25)
>>> f_path = append_final_stat('.', y_true, y_pred, 256)
>>> os.path.exists(f_path)
True
>>> os.remove(f_path)
imsegm.utilities.experiments.create_experiment_folder(params, dir_name, stamp_unique=True, skip_load=True)[source]

create the experiment folder and iterate while there is no available

Parameters
  • params (dict) – configuration

  • dir_name (str) – folder name

  • stamp_unique (bool) – use unique timestamp

  • skip_load (bool) – skip loading folder params

Return dict

>>> import shutil
>>> import pandas as pd
>>> p = {'path_out': '.'}
>>> p = create_experiment_folder(p, 'my_test', False, skip_load=True)
>>> pd.Series(p).sort_index()  
computer                   [...
path_exp      ./my_test_EXAMPLE
path_out                      .
dtype: object
>>> p = create_experiment_folder(p, 'my_test', False, skip_load=False)
>>> shutil.rmtree(p['path_exp'], ignore_errors=True)
>>> p = create_experiment_folder(p, 'my_test', stamp_unique=True)
>>> pd.Series(p).sort_index()  
computer                         [...
path_exp    ./my_test_EXAMPLE_...-...
path_out                            .
dtype: object
>>> shutil.rmtree(p['path_exp'], ignore_errors=True)
imsegm.utilities.experiments.create_subfolders(path_out, folders)[source]

create subfolders in rood directory

Parameters
  • path_out (str) – root dictionary

  • folders (list(str)) – list of subfolders

Return int

>>> import shutil
>>> dir_name = 'sample_dir'
>>> create_subfolders('.', [dir_name])
1
>>> os.path.exists(dir_name)
True
>>> shutil.rmtree(dir_name, ignore_errors=True)
imsegm.utilities.experiments.extend_list_params(params, name_param, options)[source]

extend the parameter list by all sub-datasets

Parameters
  • params (list(dict)) – list of parameters

  • name_param (str) – parameter name

  • options ([]) – lost of options

Return list(dict)

>>> import pandas as pd
>>> params = extend_list_params([{'a': 1}], 'a', [3, 4])
>>> pd.DataFrame(params)[sorted(pd.DataFrame(params))]  
   a param_idx
0  3     a-2#1
1  4     a-2#2
>>> params = extend_list_params([{'a': 1}], 'b', 5)
>>> pd.DataFrame(params)[sorted(pd.DataFrame(params))]  
   a  b param_idx
0  1  5     b-1#1
imsegm.utilities.experiments.is_iterable(var)[source]

check if the variable is iterable

Parameters

var

Return bool

>>> is_iterable('abc')
False
>>> is_iterable([0])
True
>>> is_iterable((1, ))
True
imsegm.utilities.experiments.load_config_yaml(path_config)[source]

loading the

Parameters

path_config (str) –

Return dict

>>> p_conf = './testing-congif.yaml'
>>> save_config_yaml(p_conf, {'a': 2})
>>> load_config_yaml(p_conf)
{'a': 2}
>>> os.remove(p_conf)
imsegm.utilities.experiments.nb_workers(ratio)[source]

get fraction of of available CPUs

Parameters

ratio (float) – range (0, 1)

Return int

number of workers with lower bound 1

>>> nb_workers(0)
1
imsegm.utilities.experiments.save_config_yaml(path_config, config)[source]

exporting configuration as YAML file

Parameters
  • path_config (str) –

  • config (dict) –

imsegm.utilities.experiments.set_experiment_logger(path_out, file_name='logging.txt', reset=True)[source]

set the logger to file

imsegm.utilities.experiments.string_dict(d, offset=30, desc='DICTIONARY')[source]

transform dictionary to a formatted string

Parameters
  • d (dict) –

  • offset (int) – length between name and value

  • desc (str) – dictionary title

Return str

>>> string_dict({'abc': 123})  
'DICTIONARY: \n"abc": 123'
imsegm.utilities.experiments.try_decorator(func)[source]

costume decorator to wrap function in try/except

Parameters

func

Returns

imsegm.utilities.experiments.CONFIG_YAML = 'config.yml'[source]

default file for loading/exporting experiment configuration

imsegm.utilities.experiments.CPU_COUNT = 1[source]

total number of avalaible CPUs/treads

imsegm.utilities.experiments.FILE_LOGS = 'logging.txt'[source]

default file for streaming experimeny messages

imsegm.utilities.experiments.FORMAT_DT = '%Y%m%d-%H%M%S'[source]

default date-time format

imsegm.utilities.experiments.RESULTS_TXT = 'resultStat.txt'[source]

default name of file for exporting statistics