Department of Image Processing

As described in "PET denoising and deconvolution using prior information from CT or MRI", information from PET images is often very important for identifying lesions as being malignant or benign. On the other hand, surgery for the lesion is almost always based on structural examination provided by MRI or CT. Therefore, it can be very useful to combine the information from both modalities (PET and CT/MRI). Such combination is called multimodal fusion and we have developed a new method for this purpose.

The proposed method is a pixel-wise multimodal image fusion of structural data (i.e. CT or MRI) and PET data (usually of significantly lower resolution and quality). The motivation is to preserve the contrast of both modalities. This means we want to be able to see important information - edges - all together. For this purpose the output data are computed in the RGB space - the two dimensional CT-PET space is mapped to the 3D RGB space, maximizing the color distance for any two CT-PET point combination. This gives us the opportunity to better distinguish small changes in either modality as well as the modalities from each other (as you can see in the results section).

Results:

The achieved results can be seen in Fig. 1. In the fused image (at the bottom), you can see CT data (grayscale) as well as PET data (the higher PET intensity, the higher saturation). The fused image can be at first sight a bit strange (unusual for the doctors), but it is not difficult to get used to it and the advantage of having all information in one image is quite obvious.

Fig. 1. Example of a CT-PET fusion: (left) CT (512x512x226) image, (right) corresponding PET (128x128x225) image and (bottom) the result of the fusion.