Sample unsupervised segmentation on Color images

Image segmentation is widely used as an initial phase of many image processing tasks in computer vision and image analysis. Many recent segmentation methods use superpixels, because they reduce the size of the segmentation problem by an order of magnitude. In addition, features on superpixels are much more robust than features on pixels only. We use spatial regularization on superpixels to make segmented regions more compact. The segmentation pipeline comprises: (i) computation of superpixels; (ii) extraction of descriptors such as color and texture; (iii) soft classification, using the Gaussian Mixture Model for unsupervised learning; (iv) final segmentation using Graph Cut. We use this segmentation pipeline on four real-world applications in medical imaging. We also show that unsupervised segmentation is sufficient for some situations, and provides similar results to those obtained using trained segmentation.

Borovec, J., Svihlik, J., Kybic, J., & Habart, D. (2017). Supervised and unsupervised segmentation using superpixels, model estimation, and Graph Cut. Journal of Electronic Imaging.

[1]:
%matplotlib inline
import os, sys, glob, time
import numpy as np
from PIL import Image
import matplotlib.pyplot as plt
from skimage.segmentation import mark_boundaries
sys.path += [os.path.abspath('.'), os.path.abspath('..')] # Add path to root
import imsegm.utilities.data_io as tl_data
import imsegm.pipelines as segm_pipe

Load image

[8]:
path_dir = os.path.join(tl_data.update_path('data-images'), 'others')
print ([os.path.basename(p) for p in glob.glob(os.path.join(path_dir, '*.jpg'))])
path_img = os.path.join(path_dir, 'sea_starfish-2.jpg')

img = np.array(Image.open(path_img))

FIG_SIZE = (8. * np.array(img.shape[:2]) / np.max(img.shape))[::-1]
_= plt.imshow(img)
['stars_nb2.jpg', 'star_nb1.jpg']
../_images/notebooks_segment-2d_slic-fts-clust-gc_4_1.png

Segment Image

Set segmentation parameters:

[4]:
nb_classes = 3
sp_size = 25
sp_regul = 0.2
dict_features = {'color': ['mean', 'std', 'median']}

Estimate the model without any annotation

[5]:
model, _ = segm_pipe.estim_model_classes_group([img], nb_classes, sp_size=sp_size, sp_regul=sp_regul,
                                               dict_features=dict_features, pca_coef=None, model_type='GMM')

Perform segmentation with estimated model

[6]:
dict_debug = {}
seg, _ = segm_pipe.segment_color2d_slic_features_model_graphcut(img, model, sp_size=sp_size, sp_regul=sp_regul,
                     dict_features=dict_features, gc_regul=5., gc_edge_type='color', debug_visual=dict_debug)
/usr/local/lib/python3.5/dist-packages/sklearn/mixture/base.py:237: ConvergenceWarning: Initialization 1 did not converge. Try different init parameters, or increase max_iter, tol or check for degenerate data.
  % (init + 1), ConvergenceWarning)
[9]:
fig = plt.figure(figsize=FIG_SIZE)
plt.imshow(img)
plt.imshow(seg, alpha=0.6, cmap=plt.cm.jet)
_= plt.contour(seg, levels=np.unique(seg), colors='w')
../_images/notebooks_segment-2d_slic-fts-clust-gc_12_0.png

Visualise intermediate steps

[12]:
plt.figure(), plt.imshow(mark_boundaries(img, dict_debug['slic'], color=(1, 1, 1))), plt.title('SLIC')
plt.figure(), plt.imshow(dict_debug['slic_mean']), plt.title('SLIC mean')
plt.figure(), plt.imshow(dict_debug['img_graph_edges']), plt.title('graph edges')
for i, im_u in enumerate(dict_debug['imgs_unary_cost']):
    plt.figure(), plt.title('unary cost: %i' % i), plt.imshow(im_u)
# plt.figure(), plt.imshow(dict_debug['img_graph_segm'])
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_0.png
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_1.png
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_2.png
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_3.png
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_4.png
../_images/notebooks_segment-2d_slic-fts-clust-gc_14_5.png
[ ]: