Performance shootout - python libraries for computer vision (Part 1/2)

Update 2017-02-03

Updated to latest versions of opencv/mahotas/skimage


What are the fastest libraries for image processing or computer vision that provide python as the front-end API? In this post, we will discuss some of the simplest image operations, and see how different libraries perform in terms of speed. Note that we are interested in speed only, and not the actual output of the operations performed. This is because the image operations that we are using for the post are typically pre-processing step for end-applications (and evaluation metric might differ in different applications)

1. Contestants

We boiled down to following three most popular computer vision libraries. (Few others who didn’t make it to the list - VLFeat, SimpleCV, SimpleITK, ilastik)

1.1 OpenCV

Probably the most popular and commonly use computer vision library. It is written primarily in C++ but has bindings for Matlab, Python and Java. Pros - runs on variety of platforms (Windows/Linux/OSX, Android, iOS), fast, large active community support. Cons - lack of documentation for latest python APIs, lots of dependencies, less backward compatibility.

1.2 mahotas

Again a fast library as most of its algorithms are written in C++, mahotas is optimal choice for developers from python background with its clean python interface and seemless operations with numpy arrays. Pros - large number of implemented algorithms, clean python interface, C++ backend for speed, numpy integration. Cons - smaller community

1.3 skimage

Most popular computer vision library amid Python users. Pros - integrates well with other python package like sklearn/scipy. Cons - fewer algorithms implemented.

2. operations

We are going to evaulate following basic operations for image processing

  • read (jpeg) image file to numpy array
  • erosion
  • dilation
  • open
  • close
  • sobel
  • watershed

3. boilerplate stuff

Some functions we’ll be using for plotting. Also specifying some image paths (modify them as necessary)

import os
import numpy as np
import timeit
import cv2, mahotas, skimage, skimage.morphology
from pylab import imshow, show

import matplotlib.pyplot as plt
%matplotlib inline

lena_path = 'pershoot/lena.jpeg'

def pyplots(f1, f2, f3, f4, cmap='gray', scatter=False):
    plt.figure(figsize=(10,10))
    plt.subplot(1, 4, 1)
    plt.imshow(f1, cmap=cmap)
    plt.axis('off')
    if scatter:
        plt.scatter(x=[100, 200], y=[100, 200], c='r', s=40)
    
    for index, im in enumerate([f2, f3, f4]):
        plt.subplot(1, 4, index+2)
        if cmap is not None:
            plt.imshow(im, cmap=cmap)
        else:
            plt.imshow(im)
        plt.axis('off')

erode

kind of making thick lines skinny

fRGB = mahotas.imread(lena_path).astype(np.uint8)
f64 = mahotas.imread(lena_path, as_grey=True)
f = f64.astype(int)

otsu = mahotas.otsu(f.astype(np.uint8))
print(otsu)
fbin = f > otsu
fbin8 = fbin.astype(np.uint8)

Bc = np.eye(3)
Bc = Bc.astype(bool)
Bc8 = Bc.astype(np.uint8)

fMH = mahotas.erode(fbin, Bc)
fSK = skimage.morphology.opening(fbin8, Bc8)
fCV = cv2.erode(fbin8, Bc8)

pyplots(f, fMH, fSK, fCV)
93

_config.yml

dilate

making thin lines thicker

fMH = mahotas.dilate(fbin, Bc)
fSK = skimage.morphology.dilation(fbin8, Bc8)
fCV = cv2.dilate(fbin8, Bc8)

pyplots(f, fMH, fSK, fCV)

_config.yml

open

erosion followed by dilation; removes thin (noisy) lines/holes

fMH = mahotas.open(fbin, Bc)
fSK = skimage.morphology.opening(fbin8, Bc8)
fCV = cv2.morphologyEx(fbin8, cv2.MORPH_OPEN, Bc8)

pyplots(f, fMH, fSK, fCV)

_config.yml

close

dilation followed by erosion; fills up small holes in the image

fMH = mahotas.close(fbin, Bc)
fSK = skimage.morphology.closing(fbin8, Bc8)
fCV = cv2.morphologyEx(fbin8, cv2.MORPH_CLOSE, Bc8)

pyplots(f, fMH, fSK, fCV)

_config.yml

sobel

sobel edge detection (read more here)

fMH = mahotas.sobel(f64, just_filter=True)
fSK = skimage.filters.sobel(f64)
fCV = np.absolute(cv2.Sobel(f64,cv2.CV_64F,1,1, ksize=11))

pyplots(f, fMH, fSK, fCV)

_config.yml

watershed

background removal (read more here)

markers = np.zeros_like(f)
markers[90,100] = 1
markers[90,20] = 2
markers[90,60] = 3
markers = markers.astype(np.int32)

#fMH = mahotas.cwatershed(f, markers)
#fSK = skimage.morphology.watershed(f, markers)
#fCV = cv2.watershed(fRGB, markers)

#pyplots(f, fMH, fSK, fCV, cmap='gray', scatter=True)

plt.imshow(f, cmap='gray')
plt.axis("off")
plt.scatter(x=[90, 90, 90],
            y=[100, 20, 60],
            c='r', s=40)
<matplotlib.collections.PathCollection at 0x116bb80f0>

_config.yml

marker3d = np.zeros_like(fRGB)
marker3d[90,100,:] = [1,1,1]
marker3d[90,20,:] = [2,2,2]
marker3d[90,60,:] = [3,3,3]
marker3d = marker3d.astype(np.int32)

marker = np.zeros_like(f)
marker[90,100] = 1
marker[90,20] = 2
marker[90,60] = 3
marker = marker.astype(np.int32)

fMH = mahotas.cwatershed(fRGB, marker3d)
fMH *= otsu
fMH = fMH.astype(np.int32)
fSK = skimage.morphology.watershed(fRGB, marker3d)
fSK *= otsu
fSK = fSK.astype(np.int32)
fCV = cv2.watershed(fRGB, marker)
fCV += 1
fCV *=otsu

pyplots(fRGB, fMH, fSK, fCV, cmap='jet')

_config.yml

mahotas vs skimage vs opencv

pre ='''
import numpy as np
import cv2, mahotas, skimage
from skimage.feature import greycomatrix, greycoprops

lena_path = 'pershoot/lena.jpeg'
fRGB = mahotas.imread(lena_path, as_grey=False)
f64 = mahotas.imread(lena_path, as_grey=True)
f = f64.astype(int)

marker3d = np.zeros_like(fRGB)
marker3d[90,100,:] = [1,1,1]
marker3d[90,20,:] = [2,2,2]
marker3d[90,60,:] = [3,3,3]
marker3d = marker3d.astype(np.int32)

marker = np.zeros_like(f)
marker[90,100] = 1
marker[90,20] = 2
marker[90,60] = 3
marker = marker.astype(np.int32)

otsu = mahotas.otsu(f.astype(np.uint8))
fbin = f > otsu
fbin8 = fbin.astype(np.uint8)

Bc = np.eye(3)
Bc = Bc.astype(bool)
Bc8 = Bc.astype(np.uint8)
'''
import sys

def t(s):
    return timeit.timeit(s, setup=pre, number=10)

tests = [
    ('erode', [
        'mahotas.erode(fbin, Bc)',
        'skimage.morphology.opening(fbin8, Bc8)',
        'cv2.erode(fbin8, Bc8)',
        ]),
    ('dilate', [
        'mahotas.dilate(fbin, Bc)',
        'skimage.morphology.dilation(fbin8, Bc8)',
        'cv2.dilate(fbin8, Bc8)',
        ]),
    ('open', [
        'mahotas.open(fbin, Bc)',
        'skimage.morphology.opening(fbin8, Bc8)',
        'cv2.morphologyEx(fbin8, cv2.MORPH_OPEN, Bc8)',
        ]),
    ('close', [
        'mahotas.close(fbin, Bc)',
        'skimage.morphology.closing(fbin8, Bc8)',
        'cv2.morphologyEx(fbin8, cv2.MORPH_CLOSE, Bc8)',
        ]),
    ('sobel', [
        'mahotas.sobel(f)',
        'skimage.filters.sobel(f64)',
        'cv2.Sobel(f64,cv2.CV_64F,1,1)',
        ]),
    ('cwatershed', [
        'mahotas.cwatershed(fRGB, marker3d)',
        'skimage.morphology.watershed(fRGB, marker3d)',
        'cv2.watershed(fRGB, marker)',
        ]),
    ('haralick', [
        'mahotas.features.haralick(f)',
        'greycoprops(greycomatrix(f, [1], [0]))',
        None,
        ]),
]

print(r'%-12s|%9s |%9s |%9s |' % ('Algorithm', 'mahotas','skimage','opencv'))
for name,statements in tests:
    sys.stdout.write(r'%-12s|' % name),
    for st in statements:
        if st is None:
            sys.stdout.write('      NA |'),
        else:
            time = '%.4f' % (t(st))
            sys.stdout.write('%8s |' % time),
    sys.stdout.write('\n')

Algorithm   |  mahotas |  skimage |   opencv |
erode       |  0.0006 |  0.0208 |  0.0003 |
dilate      |  0.0002 |  0.0096 |  0.0003 |
open        |  0.0005 |  0.0176 |  0.0005 |
close       |  0.0005 |  0.0174 |  0.0007 |
sobel       |  0.0485 |  0.0230 |  0.0022 |
cwatershed  |  0.9176 |  0.7912 |  0.0030 |
haralick    |  0.2126 |  0.0182 |      NA |

AND we have a winner

OpenCV outperforms mahotas and skimage (by an order of manitude).

I performed a bit experimentation computing more intensive image features like haralick features and hu moments. Will share my findings in the next blog.

Happy learning by coding!

Written on May 1, 2015