This was an abstract and poster we’ve submitted to the Machine Vision Applications conference in 2009. I was working on my PhD investigating the shape centred features in computer vision. Superpixel segmentation was an interesting application that makes a lot of sense for these types of features.

What are superpixels?

Pixels, or picture elements are usually small and square. They are an artefact of the rasterization that is inherent to digital image processing. However, they do not represent meaningful parts of the scene. A blue sky may contain tens of thousands of pixels but actually only be one thing. Image segmentation is the discipline that tries to group the pixels that belong to the same object together, for example marking all the pixels that belong to cat in an image.

Example of a superpixel segmentation. The red boundaries are the superpixels that cluster together parts of the image.

Superpixels are an intermediate step between pixels and semantic segmentation of an image. The goal is to group small segments of an image together that are likely to belong together. So, a blue sky would maybe just be one superpixel. A human in the background of an image might be comprised of only a few superpixels. The advantage of this intermediate layer is that it allows for a more condensed representation of an image that can then be used to do more advanced computer vision on it.

Medial feature superpixel segementation

There already existed several options for creating superpixels. Our approach was informed by the shape centered features described in some of the other papers (please check there). The idea is that we have a gradient vector flow field that we can use to find shape centered interest points. From these points of interest in the image we can in turn follow the gradient field back out to the original edges and mark the area that feeds the interest point as a superpixel.

Examples of the medial feature superpixel segmentation. We apply the medial feature transformation and threshold it to derive seeds (middle) from which we grow the superpixels using a watershed algorithm (right)

We compared the performance of our superpixels to three leading superpixel algorithms by checking how well they align with object boundaries. As it turned out our algorithm actually outperformed all of them on this metric.

Results: We outperformed the state of the art algorithms on the test dataset.

The poster as PDF for download