• Specifications
• ImageSegmentation

---

Additional information can be found on the Discussion page.

---

Target Outcome

This specification targets at volumetric reconstruction of a tissue of interest, specifically the definition of the boundaries of the tissue,

• as an image volume, and
• as a surface representation.

Prerequisites

Infrastructure

Slicer. Slicer is a free, open source software package for visualization and image analysis (BSD-style open source license, see http://www.slicer.org).

Previous Protocols

For more details, see Specifications/ExperimentationAnatomicalImaging.

Protocols

Input

Set(s) of MRI in NifTI format

Overview

1. Download the image volume for the desired specimen (NifTI, .nii).

2. Load the desired image volume into Slicer using these directions.

3. Segment the tissue of interest using the detailed tissue specific segmentation procedures.

   • NOTES:
   • Save the segmented label map regularly to avoid losing tedious work through program crashes.
   • Segmented volume can be overlayed on other image sets to evaluate boundaries, however it can only be further edited when overlayed on MRIs with the same size/resolution as the MRI from which it was originally created.

4. After completion of the segmentation, the volume should be exported as an image volume (NIFTI) using these directions.

5. Generate a triangulated surface from all segmented tissues (without smoothing) this is considered a RAW STL.

   • NOTE: Complete this step only after all tissue segmentations are completed, as it is common to make changes throughout the process.

How to navigate this page

If you are new to segmentation, it is recommended to first load an image into slicer, and read through the Viewing Options, Segmentation Procedures, and Image Processing Procedures sections to familiarize yourself with the program and the tools you will be using. Then, follow the directions in the Tissue Specific Procedures section, which will guide you through which segmentation procedures to use for each tissue type. Refer back to the detailed procedure descriptions as needed.

Image/Data Input Procedures

Load MRI data:

• NifTI:

  • File -> Add Data (or click Data icon button)

  • Click "Choose File(s) to add" button.
  • Select desired MRI(s) (.nii)
  • Click "Open".
• DICOM:

  • File -> DICOM (or click DCM icon button).

  • Click "Import" button.
  • Select directory containing DICOM image slices (.IMA).
  • Click 'Add link'.
  • CLick 'OK' after images load.
  • Select desired Series in DICOM Browser window.
  • Click 'Load'.

Load segmented label map image data (NifTI):

• File -> Add Data (or click Data icon button).

• Click 'Choose File(s) to Add'.
• Select Segmented label map image(s) (NifTI, .nii).
• Click 'Open'.
• Select 'Show Options'.

• Select 'LabelMap' for all label map volumes.

• Click 'Ok'.

Importing a segmented label map to the "segment editor" module

• Load segmented label map image as described above.
• From the “Modules” drop-down menu, select “Segment Editor”
• Check that the Master Volume is set as the desired MRI to be segmented.
• Load the label maps into the segment editor:
  • click “Segmentations”
  • scroll down and expand the “Export/Import models and label maps” tab
  • For “Operation”, select “Import”
  • Choose the desired label map from the “Input node” drop down.
  • Click “Import”

Image/Data Output Procedures

Exporting a label map from the Segment Editor module:

• If you are in the editing mode, click “Segmentations”
• Hide all segmentations besides the one that you wish to export using the eye toggle.
• Scroll down and expand the “Export/Import models and label maps” tab. For “Operation”, select “Export”
• From the “Output node” drop down, select the desired name. Note that this will override the existing label map with that name. If you wish, you may select “export to new labelmap” and then rename it as desired when saving.
• Expand the “Advanced” tab, and under “Exported Segments” drop down, select “Visible”.
• Click “Export”
• Save the label map using the directions below

Save segmented label map (as NifTI):

• File -> Save, or Save icon

• Deselect all modified/selected files in the 'File Name' field
• Select/Check the desired segmented label map volume (default type: NRRD, .nrrd)
• Set the 'File Format' to NifTI (.nii)

• Rename the label map volume as desired, see naming convention: Specifications/DataManagement

• Click 'Save'

Save triangulated surface, STL (a.k.a. Slicer Model):

• File -> Save, or Save icon

• Deselect all modified/selected files in the 'File Name' field
• Select/Check the desired triangulated surface (default type: VTK, .vtk)
• Set the 'File Format' to STL (.stl)

• Rename the triangulated surface as desired, see naming convention: Specifications/DataManagement

• Click 'Save'

Viewing Options

Under the pin drop-down menu (under >> button) in any of the 2D slicer viewers:

• Link viewers to ensure that they all display the same label map volumes.
• Two label maps can be overlayed on the MRI in order to compare segmented boundaries between two different structures. The label map you want to edit should be set as the Label Layer (icon in pin drop-down 2D viewer menu with an "L"). The other label map can be loaded as the Foreground Layer (icon with an "F").
• Change transparency of the label map in 2D viewers by changing the percent opacity by changing value near the eye (display/hide) icon: 0.0 (transparent) to 1.0 (opaque).
• Change contrast/brightness of MRI using the default mouse cursor pointer tool by clicking and dragging up/down and left/right in any of the 2D viewers.

Segmentation Procedures

Several methods for segmentation in 3DSlicer are described here. The choice of method depends on the tissue type, and personal preference. Below, in tissue specific procedures, guidelines are provided for segmentation procedures based on tissue type.

Segmentation Setup

• From the "Modules" drop-down menu, select "Segment Editor"

• Under 'Master volume' select the desired MRI to be segmented (If no MRI is loaded, load using these directions). When multiple MRI sets are loaded, in most cases the desired MRI to be segmented should be the same as the Master Volume. It is also possible to do segmentation on a different MRI than the Master Volume. For example, different MRIs can be displayed in different windows, e.g. those that are ligament specific and cartilage specific. When linked, Slicer uses interpolation for coupled viewing of the image sets that are already spatially aligned. For segmentation, one can use one image set as the master volume but segment in the other allowing high resolution segmentation volume from images with lower resolution.

• Click "Add" to create a new segmentation. The name and color can be changed by clicking on them in the segmentation list.
• If the 'Effects' toolbar is not visible, select the segmentation in the list to highlight it, and then click "Edit selected".
• You are now ready to begin segmenting

Manual Segmentation

For first time users, see https://www.slicer.org/wiki/Documentation/4.8/Modules/Editor for more information.

• Perform Segmentation Setup.

• Highlight the anatomy of interest in every slice, in each image plane orientation as appropriate (coronal, sagittal, axial).
• Different tools are available to highlight anatomy of interest in a desired label color (e.g. label1):
  • Use the Paint Effect [with an appropriate/largest radius] to fill in the area using a brush.
  • Use the Draw Effect to draw the boundaries of the area - freehand or by right mouse click to set the polygon corners and left click to fill in the polygon.
  • Use the eraser to change the behaviour of the tool, e.g. brush or pencil, to erase rather than draw.
• Tips and Shortcuts using 3DSlicer:
  • In Segment Editor Module:
    • 'z' to undo
    • 'y' to redo
    • arrow keys to move forward/background a slice

Grow-Cut Segmentation

• Perform Segmentation Setup

• Perform the following steps on roughly equally-spaced slices in each image plane orientation as appropriate (coronal, sagittal, axial). NOTE: If the borders are easily defined (ex: registration markers) then these steps only need to be performed on one or two slices in one image plane, plus cap the boundaries. If there is more ambiguity to the borders, perform the steps on 5-10 slices in one image plane, and just one or two in the others, plus cap the boundaries.
  • Use the Paint Effect [with an appropriate/largest radius] to highlight anatomy of interest (foreground) using a specific label color. Depending on the clarity of the boundary, it may be enough to just "scribble" in the area of the anatomy of interest as opposed to carefully tracing the boundary (see example images below).
  • Add another segmentation by clicking the "Add" button. Make sure it is a different color than the foreground label. Select the segmentation in the list to highlight it, whichever segmentation is highlighted is the one you are currently editing. Use the Paint Effect to highlight area around anatomy of interest (background).
  • Use the Paint Effect to "cap" boundaries of anatomy of interest (background) to prevent foreground leakage (before/after anatomy of interest appears).

• Export and Save the label maps just in case one of the following steps does not work as expected (or in case the program crashes).

• Select the "Grow from seeds" icon (under 'Effects')
• Click the "Initialize" button to view a preview of the grow cut segmentation
• Click the "Apply" button (Slicer window graphics will freeze until the grow cut algorithm is complete).
• Check the results. If you are not satisfied, click "Undo", and edit the segmentations more (ex: add more slices, or more detail to slices). Repeat until results are satisfactory.
• Remove the background label by selecting if in the segmentation list, and clicking "Remove"
• See figures below for example of segmentation using Grow Cut Segmentation with foreground (green) and background (yellow).

• For more information on Growth Cut Effect refer to the 3DSlicerWiki http://www.slicer.org/slicerWiki/index.php/Modules:GrowCutSegmentation-Documentation-3.6.

Image Processing Procedures

Label Map Smoothing

• From the Modules drop-down menu, select 'Surface Models' -> 'Label Map Smoothing'

• Set Gaussian Smoothing Parameters, Sigma based on image resolution (e.g. 0.7 for cartilage MRI, resolution = 0.35 x 0.35 x 0.7 mm)
• From 'Input Volume' drop-down menu, select desired label map
• From 'Output Volume' drop-down menu, select 'Create and rename new Volume'
• Enter new name as desired, click 'OK'
• Click Apply

Joint Smoothing

Joint smoothing is performed in the “Segment Editor” module

• Import the desired label map to the segment editor.

• Select one of the label maps in the list, Click “Edit Selected”
• Ensure that only the label maps you wish to perform joint smoothing on are visible. Hide all others using the eye toggle.
• Under “Effects” click “Smoothing”.
• In the “smoothing method” drop down select “Joint Smoothing”.
• Set the desired smoothing factor: try the default, and if it causes the label map to shrink too much, press undo, and repeat with a lower smoothing factor.
• Click “Apply”
• Check the results, and perform touch ups as needed using the tools in "Effects".

• Once you are satisfied with the results, you must export the label maps from the segment editor module before saving (as NifTI).

Generate Triangulated Surface (STL) from Label Map

Note: : In this stage a RAW STL is being created, therefore no smoothing procedures will be performed when generating the STL.

• From the Modules drop-down menu, select 'Surface Models' -> 'Model Maker'

  • Under 'IO' tab, from 'Input Volume' drop-down menu, select desired Label Map
  • In Model drop-down menu, either 'Create and rename new Hierarchy' (e.g. default 'Models'), or select an existing Model Hierarchy.
    • NOTE: all models can be generated under the same or separate model hierarchies
  • Under 'Create Model' tab, specify a desired 'Model Name' for the select Label Map.
  • FOR RAW STL:
    • Under 'Model Maker Parameters', set 'Smooth' to zero.
    • Under 'Model Maker Parameters', set 'Decimate' to zero.
  • Click Apply button.
• From Editor:

  • Select the MakeModelEffect

  • Set desired Label Map as the Merge Volume
  • If no smoothing is desired, deselect 'smooth model'
  • Click Apply button.
• From Segment Editor:
  • If you are in the editing mode, click “Segmentations”
  • Hide all segmentations besides the one that you wish to export.
  • Scroll down and expand the “Export/Import models and label maps” tab. For “Operation”, select “Export”. For the “Output type” select “Models”.
  • From the “Output node” drop down, select the desired model hierarchy
  • NOTE: all models can be generated under the same or separate model hierarchies
  • Expand the “Advanced” tab, and under “Exported Segments” drop down, select “Visible”.
  • Click “Export”

Once the STL has been generated, save it using these directions.

Tissue-Specific Procedures

Below are the guidelines for segmentation procedures based on tissue type.

Registration Markers

INPUT: general purpose MRI

Procedure for registration marker segmentation:

1. Perform Grow Cut Segmentation procedure on desired marker.

2. FOR SPHERICAL FEMUR/TIBIA MARKERS:

   • Manually Segment to fill in the screw hole on spherical registration markers on femur and tibia.

3. IF MARKERS ARE PLASTIC SPHERES WITH ULTRASOUND GEL:

   • Manually Segment to fill in bubbles using 'art' to define outer spherical boundary.

LABELS: FMR-M, FMR-L, FMR-P, TBR-M, TBR-L, TBR-P, PTR-S, PTR-M, PTR-L (includes bone and relative marker location, e.g. FMR-M is comprised of FM for femur, R for registration marker, and -M for medial).

Go back to Overview for next steps.

oks001, femur and tibia registration markers (posterior):

Bones

INPUT: cartilage MRI (sagittal)

BOUNDARY DEFINITION: Cortical bone will appear black in MR images, so outer edge of black cortical region defines the bone surface.

Procedure for bone segmentation:

1. Perform Grow Cut Segmentation procedure on desired bone

2. Perform Label Map Smoothing to remove Grow Cut boundary noise

3. Perform Manual Segmentation to more accurately define bone boundary

4. Iteratively repeat last two steps until boundary is as desired

LABELS: FMB (femur), TBB (tibia), FBB (fibula), PTB (patella).

Go back to Overview for next steps.

oks001, femur:

oks001, tibia:

oks001, patella:

oks001, fibula:

Cartilage

INPUT: cartilage MRI

BOUNDARY DEFINITION: Cartilage segmentation should be informed by bone segmentation (i.e. use bone boundary to help define cartilage boundary).

Procedure for cartilage segmentation:

1. Perform Grow Cut Segmentation procedure on desired bone

2. Perform Label Map Smoothing to remove Grow Cut boundary noise

3. Perform Manual Segmentation to more accurately define bone boundary

4. Perform Joint Smoothing between cartilage and neighboring bone

5. Perform manual touch up

LABELS: FMC (femoral), TBC-M (medial tibial), TBC-L (lateral tibial), PTC (patellar).

Go back to Overview for next steps.

oks001, femur cartilage:

oks001, patella cartilage:

oks001, medial tibia cartilage:

oks001, lateral tibia cartilage:

Menisci

INPUT: cartilage MRI

BOUNDARY DEFINITION: Menisci segmentation should be informed by cartilage segmentation (i.e. use cartilage boundary to help define meniscus boundary).

Procedure for menisci segmentation:

1. Manual Segmentation

2. Label Map Smoothing

3. Manual touch-up

4. Perform Joint Smoothing between meniscus and cartilage

5. Manual touch-up

LABELS: MNS-M (medial), MNS-L (lateral).

Go back to Overview for next steps.

oks001, medial meniscus:

oks001, lateral meniscus:

Connective Tissue - Ligaments & Tendons

INPUT: ligament MRI (boundary definition), cartilage MRI(resolution)

BOUNDARY DEFINITION: Connective tissue segmentation should be informed by bone segmentation (i.e. use bone boundary to help define connective tissue boundary).

Procedure for patellar ligament, quadriceps tendon, ACL, PCL (and any connective tissue when applicable):

• PHASE 1 (INPUT: ligament MRI):
  1. Load sagittal ligament MRI.

  2. Perform the Grow Cut Segmentation procedure on desired tissue.

  3. Modify the label map boundary manually if desired.
  PHASE 2 (INPUT: cartilage MRI):
  1. Load cartilage MRI.
  2. Overlay previous ligament MRI segmented label map (from PHASE 1) over the cartilage MRI (as Foreground Layer, NOTE: make sure to increase opacity, default is 0). For more details see viewing options
  3. Create new label map (Merge Volume) from cartilage MRI (set as Label Layer).

  4. Manually Segment the tissue using the ligament MRI label map boundaries as a guide.

  5. Joint Smoothing between ligament and bones

  6. manual touch ups

Alternatively, one can display the ligament specific MRIs and the cartilage specific MRI in different windows. When linked, Slicer uses interpolation for coupled viewing of the image sets that are already spatially aligned. In return, one can do the segmentation on interpolated ligament MRIs using the cartilage MRI as the master volume for segmentation. This allows high resolution segmentation volume from images with lower resolution directly.

Procedure for LCL, MCL:

1. Manual Segmentation

2. Label Map Smoothing

3. Manual touch-up

LABELS: ACL, PCL, LCL, MCL, PTL, QAT.

Go back to Overview for next steps.

oks001, patellar ligament:

oks001, quadriceps tendon:

oks001, ACL:

oks001, PCL:

oks001, LCL:

Sample Results

3D SURFACE REPRESENTATION OF SEGMENTED REGIONS:

Output

• Volume of tissue of interest as a binary image aligned with original MRI coordinate system (raw, without filtering and smoothing)
• Surface representation of tissue of interest in STL format in MRI coordinate system (raw, without filtering and smoothing)

Semi-Automated and Automated Segmentation methods

For details, go to ImageSegmentationAuto