Image Mining Extension

This extension allows the extraction of features from images so they can be further processed, e.g. by classification algorithms. The operators provided by this extension are backed by a Web service provided by NHRF.

tl_files/elico/software/rmx_image/rmext_imagemining_result.png
Using the image mining extension for analysing tissue images.

The image mining plugin contains two classes of operators plus three helper operators:

List Images

Use this operator to specify a list of directories which will be scanned for images. The operator will then create an example set containing example (row) per image and three attributes (columns): "directory" (the directory from which the image was loaded), "file" (the filename of the image) and "id" (a generated id). After that, you can modify this information or filter examples before you proceed with uploading. A typical step would be to use the "directory" attribute as the label.

Upload Images

This operator takes an example set as produced by the "List Images" operator and uploads them to the server. An "image_reference" column is then appended to the example set. This reference can be used in all subsequent operators to reference or download the image.

A typical image mining process in RapidMiner
A typical image mining process in RapidMiner

Image Transformation

This group of operators perform image transformation methods on the server. The image is not downloaded. These operators require an "image_reference" column to exist. It will generate another image_reference column to reference the result.

Feature Extraction

These operators extract features from images, transforming it in to example sets (tables) that can then be further processed by RapidMiner. In principle, a feature extraction method turns each image into a table, but most of the time, this table will have only a single row. Still, this operator generates a collection of tables, which you can merge into one, using the regular "Append" operator.

Visualize Images

If you insert this operator, you will be presented with an image inspection dialog in the result view whenever double-clicking an example in a plot. This dialog will show the different (intermediate) versions of the image generated so far. More precisely, all "image_reference" attributes in the example set will be displayable.

The image mining extension supports the following image operations:

  • skeletonizeImage: It applies on the image a skeletonization algorithm. By first transforming the image to binary. The Algorithm repeatedly removes pixels from the edges of objects in a binary image until they are reduced to single pixel wide skeletons.

  • EqualizeImage: The operator applies a histogram equalization algorithm improving automatically the contrast and other color features of the image and allowing it to reveal important structures.

  • removeNoise: This operator applies a selective median filter to grayscale images for removing noise and small artifacts.

  • AdaptiveThreshold: The operator identifies objects in the original image by applying an adaptive thresholding algorithm.

  • SharpenImage: This operator increases the contrast of the image and accentuates detail in the image or a specific selection of it.

  • CropImage: It crops the image and produces a new one based on the parameters given (x,y start of cropping, width, height of the new image).

  • ResizeImage: It resizes the image given the width and height parameters

  • RotateLeft: It rotates the image counterclockwise by 900

  • RotateRight: It rotates the image clockwise by 900

  • FlipHorizontal: It flips the image horizontally

  • FlipVertical: It flips the image vertically

  • SmoothImage: It applies a smoothing filter on the image

  • ConvertTo8Bit: It converts any given image to 8 bit (grayscale)

  • ConvertTo16Bit: It converts any given color image to 16bit

  • ConvertTo32Bit: It converts any given color image to 32bit

  • ConvertToHSB: It converts color RGB images to HSB

  • AutoThresholdImage: It performs an auto thresholding operation

  • ThresholdImage: It produces a binary image and identifies important structures within the image by applying a user defined threshold

  • ImageEdgeDetection: This operators applies a Canny-Deriche filtering algorithm for edge detection on the image. The output is a binary image containing the edges of the objects in the original image.

  • ImageErode: This operator replaces each pixel of the original image with the minimum (lightest) value in the 3×3 neighborhood. When applied on binary images, it removes pixels from the edges of black objects.

  • ImageDilate: This operator replaces each pixel with the maximum (darkest) value in the 3×3 neighborhood. When applied on binary images, it adds pixels to the edges of black objects.

  • ImageInvert: This operator inverts the colors of the original image

  • Convert2Binary: This operator will convert the original image to binary (black and white only)

  • ImageStatistics: This operator produces a number of various statistics for the image. The output is an XML file containing information about the color, the entropy, the texture and the histogram of the original image.

RapidMiner Extension

The Image Mining Extension can be downloaded from the Rapid-I Marketplace from within RapidMiner. Details are available here.

Web Service

By default, the extension uses the online version of the Web service available at http://mebioinfo.ekt.gr/axis2/services/eLICO_ImageService?wsdl. You can as well install your own instance of this Web service. For details, see the image mining web service page.

The functionality of the Image Mining Extension can best be explained through some examples:

Example A: Angiogenic images

The processing, feature extraction and quantification of Angiogenic images involves the steps of image conversion, thresholding, skeletonization and feature extraction.

User can invoke the following series of operators through the respective Web Service functions in the given order:

  • uploadImage(the image file to be processed)
  • transformImage(“ConvertTo8Bit”, “”, imageRef)
  • transformImage(“AutoThresholdImage”, “”, imageRef)
  • transformImage(“SharpenImage”, 0.5, imageRef)
  • transformImage(“skeletonizeImage”, imageRef)
  • extractFeatures(“Angiogenesis_Extraction”, imageRef)

The application of the operators on the initial image provides the following outputs in the given order:

tl_files/elico/software/rmx_image/exampleA.png
Results of the individual processing steps.

From the last image, features can be extracted.

Example B: Object Detection

This example demonstrates the specific object detection (vessels and glomerulus) related to the KUP domain. It involves steps like image conversion, image thresholding and KUP-based feature extraction.

User can invoke the following series of operators through the respective Web Service functions in the given order:

  • uploadImage(the image file to be processed)
  • transformImage(“ConvertTo8Bit”, “”, imageRef)
  • transformImage(“AutoThresholdImage”, “”, imageRef)
  • extractFeatures(“KUP_Enhance”, imageRef)
  • extractFeatures(“KUP_ObjectDetection”, imageRef)
tl_files/elico/software/rmx_image/exampleB.png
Intermediate results of the image processing. The last image shows the areas of interest in the original image.

Workflows on myExperiment

The following image mining workflows are available on myExperiment: