JP2011019768A - Image processor, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and the like, setting a plurality of local VOI (Volume of Interest) and performing alignment for data of three-dimensional image groups obtained from a medical device, thereby performing alignment with good accuracy while remarkably reducing the processing time. <P>SOLUTION: In this image processor 10, alignment is performed between three-dimensional image groups obtained by the medical device. The image processor includes: a VOI setting part 342 for setting the plurality of local three-dimensional volumes of interest in the three-dimensional image groups; a feature point extract part 342 for extracting a feature part in the preset three-dimensional volumes of interest; a corresponding point deciding part 343 for deciding corresponding points corresponding between different three-dimensional image groups concerning the extracted feature part; a parameter deciding part 344 for specifying the positions of the corresponding points to decide the optimum parameter so that the respective corresponding points between the different three-dimensional image groups coincide with each other; and an optimum alignment part 345 for performing alignment for the three-dimensional image groups based on the decided parameter. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and the like for aligning an image group obtained by a medical device.

  Diagnostic imaging apparatuses such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging) that obtain a cross-sectional image of a body without putting a scalpel into the body are widely used. In particular, a three-dimensional image is indispensable for a surgical simulation on a computer and a treatment with a radiotherapy robot (Cyber Knife), and an image processing technique capable of accurately identifying the position of a lesion is strongly desired.

  However, since CT, MRI, and the like have different properties for each modality, the information that is clearly imaged differs, and comparative interpretation is required for brain surgery and brain research. When images for different modalities are arranged for comparative interpretation, it is difficult to determine an accurate internal position, and much time is required for comparative interpretation. In addition, there are many cases where variations in accuracy are seen due to individual differences among persons who perform comparative interpretation.

  Even with images obtained with the same modality, the internal positional relationship often shifts due to differences in the body angle and imaging environment at the time of imaging due to differences in the timing of imaging. It is difficult to determine an accurate position, and a long time is required for comparative interpretation. Thus, a technique for generating a fusion image by aligning different image groups obtained from a medical device is disclosed (see Non-Patent Documents 1 and 2).

  Non-Patent Document 1 is a technique for performing alignment so that the mutual information amount is maximized with respect to the three-dimensional image data. After the image reading process and the pre-processing, the variation of parameters and the calculation of the mutual information amount by the Powell method are performed. Are repeatedly performed, and finally the parameters (movement amount, rotation amount) when the mutual information amount is maximized are determined as the optimum solution.

  Non-Patent Document 2 is a technique for aligning the two-dimensional image data of the axial plane and the sagittal plane by the ICP method (Iterative Closest Point), extracting the eye as a landmark, and corresponding slices Determine the image roughly. Then, the two-dimensional image data of the axial plane and the sagittal plane are aligned by the ICP method using the head contour image.

F.MASE, A.Collignon, D.Vandermeulen, G. Marchal and P. Suetens, "Multimodality Image Registration by Maximization of Mutual Information", IEEE Trans. On Medical Imaging, vol.16, no.2, pp.187- 198, 1997 Harada, Kohei, Kim, Tan Jukui, Ishikawa, Seiji, Yamamura, Yutaro, Yamamoto, Yoshiyoshi, “Optimal Registration Method for Head CT / MR Images Using ICP Method”, MEDICAL IMAGEING TECHNOLOGY, Japan, September 2008, Vol. . 26 No. 4, pp. 246-250

  However, since the technique shown in Non-Patent Document 1 performs alignment with three-dimensional image data for a group of hundreds of images, the amount of calculation becomes very large, and a lot of time is spent on processing time. (The experimental value is about 6000 seconds for each of about 350 CT images and MR images). In particular, when a treatment using a cyber knife or the like is performed immediately after imaging of the head, it is very important to shorten the time from imaging to treatment, and the technique shown in Non-Patent Document 1 is practical. Will be lacking.

  The technique shown in Non-Patent Document 2 performs a much faster processing time compared to Non-Patent Document 1 by aligning the two-dimensional image data on the axial plane and the sagittal plane (experimental). The value is approximately 70 seconds for each of the 350 CT images and MR images), and the alignment is performed only with the two axes of the axial plane and the sagittal plane. There is a problem that the technology is insufficient. In addition, since it is necessary to rigidly transform the three-dimensional image data after performing alignment on each of the axial plane and the sagittal plane, there is a problem that it takes some time for processing.

  Therefore, by setting a plurality of local VOIs (Volume Of Interest) for the data of the 3D image group obtained from the medical device, the processing time is greatly shortened and the accuracy is improved. An image processing apparatus and the like that perform alignment well are provided.

  (1) An image processing apparatus disclosed in the present application sets a plurality of local three-dimensional regions of interest in the three-dimensional image group in an image processing apparatus that performs alignment between three-dimensional image groups obtained by a medical device. Region-of-interest setting means, feature extraction means for extracting feature portions in the three-dimensional region of interest set by the region-of-interest setting means, and feature portions extracted by the feature extraction means between different three-dimensional image groups A corresponding point determining unit for determining a corresponding point to be determined, and a position of the corresponding point is determined so as to match each corresponding point between different three-dimensional image groups determined by the corresponding point determining unit. Parameter determining means for determining parameters, and alignment means for aligning the three-dimensional image group based on the parameters determined by the parameter determining means It is an feature.

  Thus, in the image processing device disclosed in the present application, a plurality of local 3D regions of interest in a 3D image group are set, and corresponding points between different 3D image groups in the set 3D region of interest. By optimizing them, it is possible to perform the alignment with high accuracy while greatly shortening the image alignment processing time between different three-dimensional image groups.

  (2) An image processing apparatus disclosed in the present application generates a projection image for the axial plane of the three-dimensional image group, calculates a centroid of the generated projection image, and sagittal of the three-dimensional image group. Difference calculating means for calculating a difference in height of the top of the head with respect to the surface, and the difference 3 based on the center of gravity calculated by the center of gravity calculating means and the height difference of the top of the head calculated by the difference calculating means. And initial alignment means for performing temporary alignment between the dimensional image groups.

  Thus, in the image processing device disclosed in the present application, the center of gravity of the projected image of the axial surface is calculated, the shift in the height direction of the sagittal surface is calculated, the calculated center of gravity is matched, and the height shift is calculated. By canceling and performing alignment, rough initial alignment can be performed, so that the processing time in the subsequent strict alignment processing can be limited and the processing time can be shortened.

(3) The image processing apparatus disclosed in the present application is characterized in that the processing performed by the centroid calculating means is performed by parallel processing for each different three-dimensional image group.
As described above, the image processing apparatus disclosed in the present application has an effect that the processing time can be shortened by performing the process of calculating the center of gravity for each different three-dimensional image group by parallel processing.

(4) In the image processing device disclosed in the present application, the three-dimensional image group is an image obtained by imaging a human head, and the region-of-interest setting unit includes at least three local regions that can specify the head direction. It is characterized by being set as a region of interest.
As described above, in the image processing apparatus disclosed in the present application, when the three-dimensional image group is an image obtained by imaging the human head, at least three local regions that can specify the head direction are set as the regions of interest. Therefore, even if it is a three-dimensional image group of a symmetrical head, there is an effect that the direction can be accurately specified and alignment can be performed with high accuracy.

(5) The image processing apparatus disclosed in the present application is characterized in that the set region of interest includes at least a nose region and does not include a tooth region.
As described above, in the image processing device disclosed in the present application, it is easy to capture a characteristic region by including at least the nose region in the region of interest, and the tooth region image in the CT image can be captured by not including the tooth region. There is an effect that accurate alignment can be performed by removing the disturbance.

  (6) In the image processing device disclosed in the present application, the three-dimensional image group is an image group captured by CT and / or MRI, and the CT image group that has been aligned by the alignment unit is used. A skull region extracting unit that extracts a skull region from the MRI image group that has been aligned by the positioning unit based on the skull region extracted from the CT image group, and a skull region; Skull parameter determining means for determining an optimal parameter that matches the position of the skull region from the CT image group and the skull region from the MRI image group extracted by the extracting means, and alignment by the positioning means. A brain region is extracted from the broken MRI image group, and the alignment is performed based on the brain region extracted from the MRI image group. A brain region extracting means for extracting a brain region from the CT image group that has been aligned by a stage; a brain region from the MRI image group and a brain region from the CT image group extracted by the brain region extracting means; And a brain parameter determination unit that determines an optimal parameter that matches the position, and the positioning unit again determines the parameter determined by the skull parameter determination unit and the parameter determined by the brain parameter determination unit. The three-dimensional image group is aligned.

  As described above, in the image processing device disclosed in the present application, based on the skull region extracted from the already-aligned CT image, the skull region is extracted from the MRI, the respective regions are aligned, Based on the brain region extracted from the already-aligned MRI image, the brain region is extracted from the CT image, the respective regions are aligned, and the original image group is again set using the optimum parameters calculated by the respective alignments. Therefore, it is possible to generate a processed image that takes advantage of the advantages of each modality.

  In addition, by positioning the rigid skull region and the non-rigid brain region separately, the skull region can be accurately aligned without being affected by the deformation or movement of the brain region. The brain region can be accurately aligned without being influenced by the positional relationship with the skull region and the like, and there is an effect that alignment with higher accuracy can be performed finally.

(7) The image processing apparatus disclosed in the present application determines an arbitrary reference point in the processed image aligned by the alignment unit, divides the processed image into a plurality of regions around the reference point, The adjacent areas in the divided plural areas are displayed as images of different three-dimensional image groups.
As described above, in the image processing device disclosed in the present application, an arbitrary reference point in the processed image that has been aligned is determined, the processed image is divided into a plurality of regions around the reference point, and the divided plurality of regions By displaying the adjacent regions in the images as images of different three-dimensional image groups, it is possible to comparatively interpret images of different three-dimensional image groups on one image, and simplify the image checking operation by the doctor. As a result, it is possible to easily evaluate the accuracy of alignment.

  Although the present invention has been shown as an apparatus so far, as will be apparent to those skilled in the art, the present invention can also be understood as a method and a program. These outlines of the invention do not enumerate the features essential to the present invention, and a sub-combination of these features can also be an invention.

1 is a diagram illustrating an example of a system using an image processing apparatus according to a first embodiment. 1 is a hardware configuration diagram of an image processing apparatus according to a first embodiment. 1 is a functional block diagram of an image processing apparatus according to a first embodiment. It is a figure which shows an example of CT image and MRI image processed with the image processing apparatus which concerns on 1st Embodiment. 3 is a flowchart illustrating an operation of the image processing apparatus according to the first embodiment. 6 is a flowchart showing an alignment process by an ICP method in the image processing apparatus according to the first embodiment. It is a figure which shows the process which sets VOI in the image processing apparatus which concerns on 1st Embodiment. It is the schematic diagram which extracted the feature point of VOI in the image processing apparatus which concerns on 1st Embodiment. It is a functional block diagram of the image processing apparatus which concerns on 2nd Embodiment. 10 is a flowchart illustrating an operation of the image processing apparatus according to the second embodiment. It is a 1st figure which shows an example of the process result of the image processing apparatus which concerns on 2nd Embodiment. It is a 2nd figure which shows an example of the process result of the image processing apparatus which concerns on 2nd Embodiment. It is a figure which shows an example of the display mode of the processed image in the image processing apparatus which concerns on 3rd Embodiment. It is a fusion image fused by the image processing apparatus according to the present invention.

  Embodiments of the present invention will be described below. The present invention can be implemented in many different forms. Therefore, the present invention should not be construed based only on the description of this embodiment. Also, the same reference numerals are given to the same elements throughout the present embodiment.

  In the following embodiments, the apparatus will be mainly described. However, as is apparent to those skilled in the art, the present invention can also be implemented as a method and a program for operating a computer. In addition, the present invention can be implemented in hardware, software, or hardware and software embodiments. The program can be recorded on any computer-readable medium such as a hard disk, CD-ROM, DVD-ROM, optical storage device, or magnetic storage device. Furthermore, the program can be recorded on another computer via a network.

  In each of the following embodiments, image processing is performed on a 3D image group of the head of CT and MRI. However, an image group such as CTA (CT angiography) or MRA (MR angiography) is used for image processing. Processing may be performed. Moreover, you may perform an image process with respect to the image group obtained by the same modality.

(First embodiment of the present invention)
The image processing apparatus according to this embodiment will be described with reference to FIGS. FIG. 1 is a diagram illustrating an example of a system using an image processing apparatus according to the present embodiment. The system 1 includes an image processing apparatus 10, a CT 11, and an MRI 12. The patient 2 receives a three-dimensional image of the body by CT11 and MRI12, and the captured three-dimensional image group is input to the image processing apparatus 10. The image processing apparatus 10 aligns the three-dimensional image group captured by the CT 11 and the three-dimensional image group captured by the MRI 12, and generates a processed image obtained by fusing them. The doctor 3 uses the fused processed image to perform treatment with a cyber knife, simulation of brain surgery, and brain research.

  FIG. 2 is a hardware configuration diagram of the image processing apparatus according to the present embodiment. The image processing apparatus 10 includes a CPU 21, a RAM 22, a ROM 23, a hard disk (HD) 24, a communication I / F 25, and an input / output I / F 26. The ROM 23 and the HD 24 store an operating system and various programs (for example, an image processing program). The ROM 23 and the HD 24 are read into the RAM 22 as necessary, and each program is executed by the CPU 21. The communication I / F 25 is an interface for communicating with other devices (for example, modalities, management servers, etc.). The input / output I / F 26 is an interface for receiving input from an input device such as a keyboard and a mouse and outputting data to a printer, a display, or the like. As this input / output I / F 26, USB, RS232C or the like is used. Further, if necessary, a drive corresponding to a removable disk such as a magneto-optical disk, a floppy disk (registered trademark), a CD-R, or a DVD-R can be connected.

  FIG. 3 is a functional block diagram of the image processing apparatus according to the present embodiment. The image processing apparatus 10 includes an image input unit 310, a preprocessing unit 320, an initial processing unit 330, an alignment unit 340, a processed image generation unit 350, and a display control unit 360. The initial processing unit 330 includes a centroid calculating unit 331, a parietal difference calculating unit 332, and an initial positioning unit 333. The positioning unit 340 includes a VOI setting unit 341, a feature point extracting unit 342, and a corresponding point determining unit. 343, a parameter determining unit 344, and an optimum positioning unit 345.

  The image input unit 310 uses a three-dimensional image group (hereinafter referred to as a CT image) captured by CT11 as input information and a three-dimensional image group (hereinafter referred to as an MRI image) captured by the MRI 12 as image information 301. Enter as. The preprocessing unit 320 performs preprocessing on the image information 301 input by the image input unit 310 as a preparation before alignment. The preprocessing includes, for example, isotropic voxel processing (unification of the voxel size of the CT image and the voxel size of the MRI image), removal of the examination table, smoothing of the image, and the like.

  Here, CT images and MRI images will be described. FIG. 4 is a diagram illustrating an example of a CT image and an MRI image processed by the image processing apparatus according to the present embodiment. 4A is a CT image, and FIG. 4B is an MRI image. The CT image indicates the degree to which X-rays are absorbed, and bone, bleeding, tissue edema, and the like can be observed. The MRI image uses nuclear magnetic resonance (NMR) and can observe soft tissues such as cartilage, muscle, and brain. These images are input to the image input unit 310 as image information 301.

  Returning to FIG. 3, the center-of-gravity calculation unit 331 generates a projection image for the axial plane of the three-dimensional image group, and calculates the center of gravity of the generated projection image. The parietal difference calculation unit 332 calculates the positional deviation of the parietal part with respect to the sagittal plane of the three-dimensional image group. The initial alignment unit 333 translates the projected image on the axial surface so that the centroids calculated by the centroid calculation unit 331 coincide with each other, and eliminates the deviation calculated by the parietal difference calculation unit 332 (the position of the top of the head). The sagittal plane image is translated so that the images coincide with each other, and temporary alignment is performed. By performing alignment on each of the axial plane and the sagittal plane, provisional alignment can be performed in all three-dimensional xyz directions.

  The VOI setting unit 341 sets a plurality of local regions of interest in the three-dimensional image group. In the case of a three-dimensional image of the head, in consideration of the symmetry of the head, at least three locations (for example, one of the front and rear heads and the left and right heads, one of the left and right heads and the front and back heads, etc.) ) Is set as a region of interest.

  The number and position for setting the region of interest may be arbitrarily set by the user, but five regions of the frontal head, the back of the head, the left and right temporal regions, and the top of the head are set as the region of interest. This is desirable from the viewpoint of alignment accuracy. It is further desirable to set a nose region that is a particularly characteristic region as a region of interest, and to exclude a tooth region having a large image disturbance in the CT image from the region of interest.

  The feature point extraction unit 342 extracts feature points in the VOI. Specifically, the contour portion is extracted as a feature point. The corresponding point determination unit 343 determines corresponding points corresponding to the CT image and the MRI image. That is, corresponding points are determined so that the evaluation function is minimized with respect to a point group of feature points of the CT image and MRI image extracted by the feature point extraction unit 342. The parameter determination unit 344 determines parameters for performing optimization so that the evaluation function is minimized for the corresponding points determined by the corresponding point determination unit 343. The optimal alignment unit 345 performs optimal alignment on the data of the 3D image group in the region other than the VOI based on the parameters determined by the parameter determination unit 344.

  The processed image generation unit 350 generates a fusion image by fusing the aligned CT image and MRI image. The display control unit 360 displays the fusion image generated by the processed image generation unit 350 on the display 302.

  Next, the operation of the image processing apparatus according to this embodiment will be described. FIG. 5 is a flowchart showing the operation of the image processing apparatus according to this embodiment. First, the preprocessing unit 320 performs preprocessing on the CT image and MRI image input by the image input unit 310 (S51). Here, pre-processing will be described. Specifically, the preprocessing includes isotropic voxel processing, removal of the examination table, and smoothing of the image.

The isotropic voxel process is a process for matching the pixels of the CT image and the MRI image. One pixel of the CT image is 0.683 × 0.683 mm, and the width between slices (height direction) is 2 mm. Therefore, the three-dimensional shape is a rectangular parallelepiped that is long in the height direction. This is converted into a cube (0.683 × 0.683 × 0.683 mm) by linear interpolation. On the other hand, since one pixel of the MRI image is 0.6836 × 0.6836 mm and the width (height direction) between slices is 2 mm, the cube (0.6836 × 0. 6836 × 0.6836 mm), and in order to match the pixels of the CT image, the MRI image is reduced to match the pixels of the CT image.
In the image processing apparatus according to the present embodiment, the MRI image is converted and aligned using the CT image as a reference, but either may be used as a reference image.

  As shown in FIG. 4A, the examination table is removed in order to prevent the examination table from being detected as an error when the examination table is captured in the image. Since the examination table is imaged at a fixed position every time, the examination table can be removed by removing the object imaged at that position. In the smoothing of an image, when noise that causes an extreme change in pixel value is included, the noise is removed by weighted averaging to smooth the image.

  Returning to FIG. 5, when the preprocessing of S51 is completed, the initial processing unit 330 performs initial alignment (S52). The initial alignment is a tentative rough alignment for limiting the processing range of strict alignment performed in later processing and shortening the processing time. First, the center-of-gravity calculation unit 331 generates a projection image for the axial plane, and calculates the center of gravity of the generated projection image. In addition, the parietal difference calculation unit 332 calculates the difference in the height of the parietal part with respect to the sagittal plane. Then, the initial alignment unit 333 performs parallel translation in the xy direction so that the calculated centroids coincide with each other, and performs parallel translation in the z direction so that the heights of the crowns coincide. By performing parallel movement in each direction, the centroids can be matched as three-dimensional image data, and temporary alignment can be performed.

  Note that S51 and S52 are preferably executed in parallel for speeding up the processing. For example, parallel processing may be performed using a plurality of image processing devices, or parallel processing may be realized by a single image processing device having a plurality of CPUs.

  When the initial alignment is completed, the alignment unit 340 performs a strict alignment process by the ICP method (S53). Here, the alignment process by the ICP method will be described in detail. FIG. 6 is a flowchart showing the alignment process by the ICP method in the image processing apparatus according to this embodiment.

  First, the VOI setting unit 341 sets a VOI in the three-dimensional image group (S61). As described above, the VOI has at least three locations (for example, any one of the front and rear heads and the left and right heads, or one of the left and right heads and the front and back heads) in consideration of the symmetry of the head. A local region is set as a region of interest.

  FIG. 7 is a diagram showing processing for setting a VOI in the image processing apparatus according to the present embodiment. Here, in order to improve the alignment accuracy, five areas of the frontal head, the back of the head, the left and right temporal regions, and the top of the head are set as the regions of interest. In addition, a nose region, which is a particularly characteristic region, is set as a region of interest, and a tooth region having a large image disturbance in the CT image is excluded from the region of interest.

  Returning to FIG. 6, when the VOI is set in S61, the feature point extraction unit 342 extracts the feature points in the set VOI (S62). FIG. 8 is a schematic diagram in which VOI feature points are extracted in the image processing apparatus according to the present embodiment. FIG. 8A shows VOI feature extraction from a CT image, and FIG. 8B shows VOI feature extraction from an MRI image. The upper two figures are feature extraction when viewed from the front, and the lower two figures are feature extraction when viewed from the side.

  Here, 18 × 18 pixels are extracted from the CT image, and 6 × 6 pixels are extracted from the MRI image. This is because, in the present embodiment, the CT image is fixed as reference image data and the MRI image is converted to perform alignment. As a result of repeated experiments by the inventors, this size is optimized. The size is set.

  Returning to FIG. 6, when feature points in each VOI are extracted, the corresponding point determination unit 343 determines corresponding points between the CT image and the MRI image by the ICP method (S63), and the parameter determination unit 344 optimizes the evaluation function. Perform (S64). It is determined whether or not the evaluation function is an optimum value (S65), and if it is not the optimum value, the processes of S63 and S64 are repeated until the optimum value is reached. When the optimum value is reached, the alignment process by the ICP method is terminated.

  Here, the processing algorithm of S63 and S64 by the ICP method will be described. The ICP method is one of the methods for performing conversion so that two image data most closely match. In the present embodiment, the MRI image is converted so that the point group of the CT image obtained from the VOI feature points and the point group of the MRI image most closely match. Each of the two point clouds

And corresponding points are determined so that the average distance d between the point groups represented by the following equation is minimized.

Next, using the Powell method, F is sequentially converted to obtain an optimum value that matches G. The converted point cloud in the mth step of sequential positioning calculation

And At this time, a point of the point group G corresponding to f _i ^(m) is defined as g _ki , and a residual vector e _i ^(m) is defined by the following equation.

A residual sum of squares function J is obtained by the following equation as an evaluation function in positioning.

As an end condition, it is assumed that the number of repetitions is a predetermined number or more, or the average distance is a predetermined value or less.

  Returning to FIG. 5, the optimum alignment unit 345 performs the MRI image conversion process for the region other than the VOI by using the parameter fluctuation amount (parallel movement amount, rotation amount) in the optimum solution obtained in S53 (S54). ). The processed image generation unit 350 generates a fusion image obtained by fusing the CT image and the MRI image (S55), and ends the process.

  As described above, according to the image processing apparatus according to the present embodiment, a plurality of local 3D regions of interest in a 3D image group are set, and between different 3D image groups in the set 3D region of interest. By matching the corresponding points and optimizing, it is possible to perform the alignment with high accuracy while greatly shortening the image alignment processing time between different three-dimensional image groups.

In addition, by calculating the center of gravity in the projected image of the axial surface, calculating the displacement in the height direction of the sagittal surface, aligning the calculated center of gravity, and aligning by eliminating the height displacement, it is possible to roughly Since initial alignment can be performed, the processing time can be shortened by limiting the processing range.
Furthermore, the processing time can be shortened by performing the processing for calculating the center of gravity by parallel processing for each different three-dimensional image group.

Furthermore, when the three-dimensional image group is an image of a human head, by setting at least three local regions that can identify the head direction as regions of interest, Even in the case of a three-dimensional image group, it is possible to accurately specify the direction and perform alignment with high accuracy.
Furthermore, since at least the nose region is included in the region of interest, the feature portion can be easily captured and accurate alignment can be performed.

(Second embodiment of the present invention)
The image processing apparatus according to this embodiment will be described with reference to FIGS. The image processing apparatus according to the present embodiment uses the property that CT can easily extract the skull, difficult to extract the brain region, MRI can easily extract the brain region, and difficult to extract the skull region. Highly accurate alignment is achieved.
In addition, in this embodiment, the description which overlaps with the said 1st Embodiment is abbreviate | omitted.

  FIG. 9 is a functional block diagram of the image processing apparatus according to the present embodiment. The difference from the first embodiment is that a brain region extraction unit 370, a skull region extraction unit 380, and a non-rigid body alignment unit 390 are newly provided. The brain region extraction unit 370 extracts a brain region from an MRI image that has already been aligned, and extracts a brain region from a CT image that has already been aligned based on the extracted brain region. The skull region extraction unit 380 extracts the skull region from the CT image that has already been aligned, and extracts the skull region from the MRI image that has already been aligned based on the extracted skull region.

  The non-rigid registration unit 390 aligns the brain region from the MRI image extracted by the brain region extraction unit 370 and the brain region from the CT image, calculates the amount of variation of the optimal parameter, and calculates the original input image Performs conversion processing for. Since the alignment of the brain region here is alignment of a non-rigid body, it is difficult to perform alignment by the ICP method. Therefore, when calculating the variation amount of the optimal parameter, for example, the mutual information amount is calculated, and the parameter when the mutual information amount is maximized is calculated as the optimal solution.

  On the other hand, since the positioning using the ICP method performed by the positioning unit 340 is possible for the positioning of the skull region, the positioning unit 340 detects the skull region from the CT image extracted by the skull region extracting unit 380. By aligning the skull region from the MRI image, the optimum parameters are calculated, and the original input image is converted.

  The processed image generation unit 350 generates a fusion image by fusing the image converted by the registration unit 340 using the skull region and the image converted by the non-rigid registration unit 390 using the brain region. To do.

  Next, the operation of the image processing apparatus according to this embodiment will be described. FIG. 10 is a flowchart showing the operation of the image processing apparatus according to this embodiment. Since the processing from S101 to S104 is the same as the processing from S51 to S54 shown in FIG. 5 in the first embodiment, description thereof will be omitted.

  When image conversion is performed in S104, processing related to the skull region and processing related to the brain region are performed in parallel (S105). In the process related to the skull region, first, the skull region extraction unit 380 extracts the skull region from the CT image (S106). When extracting the skull region, it is extracted by multiple threshold processing. That is, a range of pixel values indicating the skull is set as a threshold, and pixels within the range are extracted as a skull region.

  Based on the extracted skull region, a skull region is extracted from the MRI image (S107). At this time, the skull region can be extracted from the MRI image by extracting the same region as the skull region in the CT image from the MRI image that has already been aligned. The alignment unit 340 aligns the skull region extracted in S106 and S107 as a rigid body, and determines an optimal parameter fluctuation amount (S108). The process of S108 is performed by the same process as the alignment by the ICP method shown in FIG.

  On the other hand, in the process related to the brain region, first, the brain region extraction unit 370 extracts the brain region from the MRI image (S109). When the brain region is extracted, it is extracted by the region expansion method. That is, regions (pixels) having the same characteristics are integrated into one region and finally extracted as a brain region. At this time, brain regions are extracted from MRI images that have already been aligned.

  A brain region is extracted from the CT image based on the extracted brain region (S110). By extracting the same region as the brain region extracted from the MRI image that has already been aligned, the brain region can be extracted from the CT image. The non-rigid body alignment unit 390 aligns the brain regions extracted in S109 and S110 as non-rigid bodies, and determines an optimal parameter fluctuation amount (S111). Since the process of S111 is a process performed on a non-rigid body, alignment by the mutual information amount is performed without performing alignment by the ICP method. That is, an optimal parameter fluctuation amount is determined so that the mutual information amount is maximized.

  Retransformation is performed on the original image group using the optimum parameter variation determined in S108 and S111 (S112). The processed image generation unit 350 generates a fusion image by fusing the reconverted image group (S113), and ends the process.

  The result of the process shown in FIG. 10 will be described with reference to FIGS. 11 and 12. FIG. 11 is a first diagram illustrating an example of a processing result of the image processing apparatus according to the present embodiment, and FIG. 12 is a second diagram illustrating an example of a processing result of the image processing apparatus according to the present embodiment. FIG. 11A is a diagram illustrating a skull region extracted from a CT image, and FIG. 11B is a diagram illustrating a brain region extracted from an MRI image. FIG. 12 shows a fusion image obtained by aligning the CT image and the MRI image for each skull region and brain region extracted in FIG. In FIG. 12, a front view of the head, a top view of the head, and a side view of the head are shown in order from the left, and it can be seen from each image that the brain fits exactly on the skull and is accurately aligned.

  Thus, according to the image processing apparatus according to the present embodiment, based on the skull region extracted from the already-aligned CT image, the skull region is extracted from the MRI, and the respective regions are aligned, In addition, based on the brain region extracted from the MRI image that has already been aligned, the brain region is extracted from the CT image, the respective regions are aligned, and the original parameters are restored using the optimum parameters calculated for each alignment. Since the image groups are aligned, it is possible to generate a processed image that takes advantage of the advantages of each modality.

  In addition, by positioning the rigid skull region and the non-rigid brain region separately, the skull region can be accurately aligned without being affected by the deformation or movement of the brain region. In addition, the brain region can be accurately aligned regardless of the positional relationship with the skull region and the like, and finally, a more accurate alignment can be performed.

(Third embodiment of the present invention)
The image processing apparatus according to this embodiment will be described with reference to FIG. The image processing apparatus according to the present embodiment makes it easy to visually recognize the processed image subjected to the alignment process by devising the display control when visually determining whether the alignment is accurate. To do.

  Specifically, the display control unit 360 determines an arbitrary reference point in the processed image generated by the processed image generation unit 350, and the processed image is divided into a plurality of regions (for example, divided into two, four, etc.) around the reference point. Divided into 8 divisions, etc.). In each divided area, display control is performed so that the CT image and the MRI image are adjacent to each other.

  FIG. 13 is a diagram illustrating an example of a display mode of a processed image in the image processing apparatus according to the present embodiment. In FIG. 13, an original CT image 132 and an original MRI image 133 before alignment are displayed in an area 130, and a processed image 135 after alignment is displayed in an area 131. The reference point 134 is a position arbitrarily selected by the user, and the position of the cursor is the reference point 134. The user can change the position of the reference point 134 by moving the cursor using a mouse or a cross key as necessary.

Here, the processed image 135 is divided into four parts around the reference point 134, and the original CT image 132 and the original MRI image 133 are displayed adjacent to each other for each divided region. In the area 131, the upper left and lower right areas are original CT images, and the upper right and lower left areas are original MRI images. Thus, by displaying the original CT image 132 and the original MRI image 133 between the adjacent areas of the divided areas, it is possible to visually recognize the alignment accuracy in each boundary area. In addition, since the position of the cursor can be freely changed, only the movement of the cursor can be easily confirmed at a position that needs to be confirmed.
As described above, the number of divisions of the processed image 135 may be any number as long as it is an even number such as two divisions, four divisions, six divisions, and eight divisions.

  As described above, according to the image processing apparatus according to the present embodiment, an arbitrary reference point in the processed image that has been aligned is determined, and the processed image is divided into a plurality of regions around the reference point. By displaying adjacent areas in multiple areas as images of different 3D image groups, it is possible to compare and interpret images of different 3D image groups on one image, simplifying the work of checking images by a doctor In addition, the accuracy of alignment can be easily evaluated.

  Although the present invention has been described with the above embodiments, the technical scope of the present invention is not limited to the scope described in the embodiments, and various modifications or improvements can be added to these embodiments. . And embodiment which added such a change or improvement is also contained in the technical scope of the present invention. This is apparent from the claims and the means for solving the problems.

Using the image processing apparatus according to the first embodiment, an experiment for positioning a three-dimensional image group was performed.
(conditions)
Assume that five sets of CT images and MRI images are prepared as a three-dimensional image group, and the MRI images are moved and aligned with reference to the CT images. The CT image uses 120 images of 512 × 512 pixels, a pixel size of 0.683 mm, and a slice thickness of 2 mm, and the MRI image uses 120 images of 512 × 512 pixels, a pixel size of 0.6836 mm, and a slice thickness of 2 mm.

  In this experiment, processing time and mutual information are used as a measure for evaluating whether or not each image is correctly overlapped. Here, the mutual information amount will be described. The mutual information amount is an information amount regarding the other brought about by knowing one of the two matters. The mutual information I (A, B) is defined by the following equation.

p _A (a) and p _B (b) represent probability distributions of random variables A and B, and p _{A, B} (a, b) represent joint probability distributions.

  When obtaining mutual information in an image, a probability distribution function is obtained by dividing a histogram h (a, b) created using pixel values a and b of corresponding pixels of two images by the number N of pixels. Can think. Therefore,

Is used to find the mutual information.

(result)
The results of experiments conducted under the above conditions are shown in the following table. Table 1 shows the result of mutual information, and Table 2 shows the result of processing time (s).

  As can be seen from Table 1, the mutual information amount is almost the same as that of the non-patent documents 1 and 2 shown in the prior art document, and therefore, the positioning accuracy is performed in the same manner as the conventional technology. On the other hand, the processing time in Table 2 is shortened to about 1/460 compared to the technique shown in Non-Patent Document 1, and is about 1/4 or less compared to the technique shown in Non-Patent Document 2. It is getting shorter. Furthermore, when the parallel processing is adopted, it is about 3/5 shorter than the case where the parallel processing is not adopted.

  FIG. 14 shows a fusion image fused by the image processing apparatus according to the present invention. Although it is difficult to understand for convenience of drawing, in FIGS. 14B and 14E, since the two-dimensional image is aligned by two axes, there is a region where the brain and the skull partially overlap. In FIGS. 14C and 14F, since the alignment is performed using the three-dimensional data, the CT image and the MRI image are accurately aligned by improving the alignment.

  In other words, the image processing apparatus according to the present invention can greatly reduce the processing time while maintaining high accuracy, so that it can be practically used in the future, and can greatly contribute to medical development. is there.

1 System 2 Patient 3 Doctor 10 Image Processing Device 11 CT
12 MRI
21 CPU
22 RAM
23 ROM
24 HD
25 Communication I / F
26 I / O I / F
301 Image Information 302 Display 310 Image Input Unit 320 Pre-Processing Unit 330 Initial Processing Unit 331 Center of Gravity Calculation Unit 332 Parietal Difference Calculation Unit 333 Initial Positioning Unit 340 Positioning Unit 341 VOI Setting Unit 342 Feature Point Extracting Unit 343 Corresponding Point Determination Unit 344 Parameter determination unit 345 Optimal alignment unit 350 Processed image generation unit 360 Display control unit 370 Brain region extraction unit 380 Skull region extraction unit 390 Non-rigid body alignment unit

Claims (9)

In an image processing apparatus that performs alignment between three-dimensional image groups obtained by a medical device,
Region-of-interest setting means for setting a plurality of local three-dimensional regions of interest in the three-dimensional image group;
Feature extraction means for extracting a feature portion in the three-dimensional region of interest set by the region of interest setting means;
Corresponding point determination means for determining corresponding points between different three-dimensional image groups for the feature portion extracted by the feature extraction means;
Parameter determining means for determining the optimum parameters by specifying the positions of the corresponding points so that the corresponding points between the different three-dimensional image groups determined by the corresponding point determining means match;
An image processing apparatus comprising: an alignment unit configured to align the three-dimensional image group based on the parameter determined by the parameter determination unit. The image processing apparatus according to claim 1.
Centroid calculating means for generating a projection image for the axial plane of the three-dimensional image group and calculating the centroid of the generated projection image;
A difference calculating means for calculating a difference in height of the top of the sagittal plane of the three-dimensional image group,
And an initial alignment means for performing temporary alignment between the different three-dimensional image groups based on the difference between the center of gravity calculated by the center of gravity calculation means and the height of the crown calculated by the difference calculation means. An image processing apparatus. The image processing apparatus according to claim 2,
An image processing apparatus, wherein the processing performed by the centroid calculating means and the difference calculating means is performed by parallel processing for each different three-dimensional image group. The image processing apparatus according to any one of claims 1 to 3,
The three-dimensional image group is an image of a human head;
The image processing apparatus, wherein the region-of-interest setting means sets at least three local regions that can specify a head direction as a region of interest. The image processing apparatus according to claim 4.
The image processing apparatus characterized in that the set region of interest includes at least a nose region and does not include a tooth region. The image processing apparatus according to any one of claims 1 to 5,
The three-dimensional image group is an image group captured by CT (Computed Tomography) and MRI (Magnetic Resonance Imaging),
A skull region is extracted from the CT image group that has been aligned by the alignment unit, and the MRI image that has been aligned by the alignment unit based on the skull region extracted from the CT image group A skull region extracting means for extracting a skull region from the group;
Skull parameter determining means for determining an optimal parameter that the position of the skull area from the CT image group and the skull area from the MRI image group extracted by the skull area extracting means;
The brain image is extracted from the MRI image group that has been aligned by the alignment means, and the CT image that has been aligned by the alignment means based on the brain region extracted from the MRI image group. A brain region extracting means for extracting a brain region from the group;
A brain parameter determination unit that determines an optimal parameter that is extracted by the brain region extraction unit and that matches the position of the brain region from the MRI image group and the brain region from the CT image group;
An image processing apparatus, wherein the alignment unit performs alignment of the three-dimensional image group again based on the parameter determined by the skull parameter determination unit and the parameter determined by the brain parameter determination unit. The image processing apparatus according to any one of claims 1 to 6,
An arbitrary reference point in the processed image aligned by the alignment means is determined, the processed image is divided into a plurality of regions around the reference point, and adjacent regions in the divided plurality of regions are respectively determined. An image processing apparatus that displays images of different three-dimensional image groups. In an image processing method for aligning a three-dimensional image group obtained by a medical device,
A region-of-interest setting step of setting a plurality of local three-dimensional regions of interest in the three-dimensional image group;
A feature extraction step of extracting a feature in the three-dimensional region of interest set by the region of interest setting step;
Corresponding point determination step for determining corresponding points between the different three-dimensional image groups for the feature portion extracted by the feature extraction step;
A parameter determination step for determining an optimum parameter by specifying the position of the corresponding point so that the corresponding points between the different three-dimensional image groups determined by the corresponding point determination step match;
An image processing method comprising: an alignment step of aligning the three-dimensional image group based on the parameter determined by the parameter determination step. In an image processing program for causing a computer to function to align a three-dimensional image group obtained by a medical device,
Region-of-interest setting means for setting a plurality of local three-dimensional regions of interest in the three-dimensional image group;
Feature extraction means for extracting a feature in the three-dimensional region of interest set by the region of interest setting means;
Corresponding point determination means for determining corresponding points between the different three-dimensional image groups for the feature portion extracted by the feature extraction means;
Parameter determining means for determining the optimum parameters by specifying the positions of the corresponding points so that the corresponding points between different three-dimensional image groups determined by the corresponding point determining means match;
An image processing program for causing a computer to function as positioning means for positioning the three-dimensional image based on parameters determined by the parameter determination means.