JP5478832B2 - Medical image processing apparatus and medical image processing program - Google Patents

Description

  The present invention relates to a medical image processing apparatus for aligning positions of a plurality of medical images acquired by a medical image diagnostic apparatus, and a medical image processing program.

  Liver cancer accounts for about 10% of cancer diseases, and the number is increasing. Moreover, since the prognosis is relatively poor, there is a strong social demand for diagnosis and treatment. For diagnosis of a tumor, a medical image diagnostic apparatus such as an ultrasonic diagnostic apparatus, an MRI apparatus, or an X-ray CT apparatus is used. These medical image diagnostic apparatuses can acquire a three-dimensional image and a contrast image, and can detect tumors at an early stage.

  Examples of tumor treatment include radiofrequency ablation (Radio-Frequency Abbreviation; sometimes referred to as “RFA”), high-energy focused ultrasound (hereinafter sometimes referred to as “HIFU”), and the like. A low invasive treatment method is being implemented. The HIFU may also be referred to as focused ultrasound (hereinafter sometimes referred to as “FUS”).

  Then, after cauterizing a lesion such as a tumor by the low invasive treatment method, an image of the lesion acquired before the operation and an image of the lesion acquired after the operation are simultaneously displayed, and based on the images. To determine the therapeutic effect. For example, a two-dimensional B-mode image is acquired using an ultrasonic diagnostic apparatus, and a lesion is diagnosed based on the B-mode image. In addition, a tomographic image is acquired using an X-ray CT apparatus, and a lesion is diagnosed based on the tomographic image. In addition, when volume data is acquired by photographing a three-dimensional region, an operator such as a doctor positions the position of a cross section where an image is generated in order to match the position of the preoperative image and the position of the postoperative image. Both images were displayed in parallel by manually changing the image, enlarging / reducing the image, moving the image on the screen, and the like. In such a case, since the alignment is performed by the visual observation of the operator, the alignment error increases, and accurate postoperative diagnosis becomes difficult. In order to prevent recurrence caused by residual cautery or insufficient cauterization, it is required to more accurately determine the therapeutic effect. Conventionally, various methods for aligning a plurality of images have been proposed.

  As a method of aligning images, there are a method using information in the form represented in the image and a method not using information in the form. As a method using form information, there has been proposed a method for aligning images by matching the position of a three-dimensional form by pattern matching (for example, Patent Document 1). As another method, after aligning images representing specific structures such as bone and soft tissue by affine transformation, warping (nonlinear distortion transformation) the shift amount obtained by template matching, Again, a method of performing local alignment has been proposed (for example, Patent Document 2).

  Further, as a method that does not use form information, a method is proposed in which a mutual information method (hereinafter sometimes referred to as “MI method”) is applied to pixel values of an image to align the positions of the two images. (For example, Patent Document 3 and Patent Document 4).

JP 2002-17729 A JP 2006-6435 A JP 2004-509723 A JP 2007-54636 A

  When manually aligning the positions of a plurality of images, the work for searching for the position of the cross section for displaying the images is complicated and takes time. Furthermore, since the position of the cross section changes each time a new image is displayed, it is difficult to compare repeatedly under the same conditions. In addition, when the alignment is performed manually, errors in the position and angle of the cross section increase, and it is difficult to accurately compare the ablation plan area and the actually ablated area.

  Further, in the method of performing alignment using morphological information, alignment is performed using an image of a part whose shape does not change in the two images to be compared. Therefore, when there is no portion where the form does not change in the two images, it is difficult to accurately align the images.

  Further, in an ultrasonic image mainly used in the treatment using RFA, the outline of the tissue is relatively unclear, and the form of the ablation object changes significantly before and after the ablation. Therefore, it is difficult to improve the accuracy of image alignment by a method using morphological information.

  Further, in the registration method using the MI method, even if the outline of the tissue represented in the image is unclear, if the images are similar, the registration of the images can be performed with relatively high accuracy. . However, in the cauterization treatment, the luminance value of the image representing the affected part changes significantly before and after the cauterization. Therefore, even if the MI method is applied to match the position of the pre-operative image and the post-operative image, the images are not similar, so that it is difficult to perform alignment with high accuracy.

  The present invention solves the above-described problems and provides a medical image processing apparatus and a medical image processing program capable of more accurately aligning a plurality of images acquired by a medical image diagnostic apparatus. For the purpose.

The invention according to claim 1 is a storage means for storing a plurality of 4D contrast image data including volume data obtained by imaging a subject into which a contrast medium has been injected before and after medical treatment of the subject. And the previous pixel value curve and the subsequent pixel value curve representing temporal changes in pixel values in each voxel based on the previous and subsequent 4D contrast image data stored in the storage means, respectively. A curve model creating unit for obtaining a pixel value curve model representing a temporal change of a pixel value in each part of the subject for each of the previous and the following; and for, by comparing the pixel value curve model of each site and the pixel value curve of the voxels, each part of the pixel value curve of the voxels the subject By classifying, between the previous and subsequent 4D contrast image data, the first classification means for creating the previous first mask information and the subsequent first mask information indicating the region of each part, The previous second mask information and the subsequent second mask are excluded by excluding the region of the portion other than the portion where the shapes of the pixel value curve models substantially coincide with each other from the previous and subsequent first mask information. A second classification means for creating information, and each voxel included in each region indicated by each of the second mask information before and after the second is specified from each of the previous and subsequent pixel value curves. The feature amount image generating means for generating the previous feature amount image data and the subsequent feature amount image data by allocating the feature amount, and the positions of the previous and subsequent feature amount images are aligned. Display for generating, for each 4D contrast image data, image data for display that is aligned according to the position alignment based on image registration means and volume data included in each of the previous and subsequent 4D contrast image data A medical image processing apparatus comprising: an image generation unit; and a display control unit configured to display a display image based on the display image data for each of the 4D contrast image data on the display unit.
According to the eighth aspect of the present invention, a computer receives a plurality of 4D contrast image data including volume data obtained by imaging a subject into which a contrast medium has been injected before and after medical treatment of the subject. A curve creation function for creating the previous pixel value curve and the subsequent pixel value curve, each representing a temporal change of the pixel value in each voxel based on each of the previous and subsequent 4D contrast image data; Curve model creation means for obtaining a pixel value curve model representing a temporal change of pixel values in each part of the subject for each of the previous and the following, and a pixel value of the voxel for each of the previous and the subsequent wherein the curve is compared with the pixel value curve model of each site to classify the pixel value curve of the voxels for each region of the subject, The pixel value curve model between the first classification function for creating the previous first mask information and the subsequent first mask information indicating the region of the part, and the previous and subsequent 4D contrast image data The second mask information before and the second mask information after the second are created by excluding the regions of the parts other than the parts where the shapes of the first and second masks substantially coincide with each other from the previous and subsequent first mask information. A feature amount specified from each of the previous and subsequent pixel value curves is assigned to each voxel included in each region indicated by the classification function and each of the second mask information before and after the second mask information. A feature amount image generation function for generating the previous feature amount image data and the subsequent feature amount image data, and an image registration function for aligning the positions of the previous and subsequent feature amount images. A display image generation function for generating, for each 4D contrast image data, display image data that is aligned according to the alignment based on volume data included in each of the previous and subsequent 4D contrast image data; A medical image processing program for executing a display control function for arranging display images based on display image data for each 4D contrast image data and displaying the images on a display device.

  According to the present invention, by obtaining a pixel value curve representing a temporal change in pixel value in each voxel, it is possible to extract the region of each part based on the pixel value curve. Then, between each 4D contrast image data, the second mask information is created by excluding the region other than the region where the shape of the pixel value curve model substantially matches from the first mask information representing each region. To do. As a result, an area that is not suitable for image alignment processing can be excluded from the mask information. By obtaining a feature amount image based on the second mask information and performing image alignment using the feature amount image, it is possible to perform image alignment except for an area that is not suitable for alignment processing. . As described above, since it is possible to perform image alignment except for a region that is not suitable for the alignment processing, it is possible to perform image alignment more accurately.

  A medical image processing apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a medical image processing apparatus according to an embodiment of the present invention.

  The medical image diagnostic apparatus 100 includes an imaging apparatus such as an ultrasonic diagnostic apparatus, an X-ray CT apparatus, or an MRI apparatus, and acquires medical image data of a subject. The medical image data acquired by the medical image diagnostic apparatus 100 is output to the medical image processing apparatus 1 and stored in the image storage unit 9. For example, the medical image diagnostic apparatus 100 acquires a plurality of volume data with different imaging times by imaging a three-dimensional region of the subject, and the image storage unit 9 is acquired by the medical image diagnostic apparatus 100. A plurality of volume data is stored.

  In this embodiment, so-called contrast imaging is performed. The contrast agent injected into the subject by drip or blood vessel injection travels in the bloodstream and reaches the target organ. By observing the presence or absence of the contrast effect when the contrast agent permeates or the difference in the degree thereof, and observing the shape of the contrasted portion, it is possible to discover abnormalities in the lesion or organ. In this embodiment, a liver is taken as an imaging target, and ablation treatment using the RFA method will be described as an example. Specifically, before the ablation treatment (before surgery), the medical image diagnostic apparatus 100 performs contrast imaging to acquire a plurality of volume data with different imaging times. Furthermore, after the cauterization treatment (postoperative), the medical image diagnostic apparatus 100 performs contrast imaging to obtain a plurality of volume data with different imaging times. In pre-operative imaging and post-operative imaging, the contrast medium injected into the subject flows into the liver. In the meantime, since the medical image diagnostic apparatus 100 is photographing, it is possible to acquire a plurality of volume data representing how the contrast medium flows into the liver.

  Further, the medical image diagnostic apparatus 100 attaches the time when each volume data is captured to each volume data as supplementary information.

  The medical image processing apparatus 1 aligns the positions of a plurality of images acquired by the medical image diagnostic apparatus 100. In this embodiment, as an example, a case where the position of an image acquired before ablation treatment (before surgery) and the position of an image acquired after ablation treatment (after surgery) will be described. For example, in order to confirm the effect of the cauterization treatment, the position of the image acquired before the operation and the position of the image acquired after the operation are displayed together.

  Here, FIG. 2 shows an image representing tissue obtained by contrast imaging before surgery. FIG. 2 is a diagram showing images sequentially representing the state in which the contrast medium flows. The image shown in FIG. 2 is an image acquired by photographing the medical image diagnostic apparatus 100 using an ultrasonic diagnostic apparatus. An image 201 is an image taken before the contrast medium flows into the liver. Then, as time passes with the early phase, the portal vein phase, and the late phase, the contrast agent flows into the blood vessels, the tumor, and the surrounding tissues. For example, the image 202 acquired in the early phase shows a state in which the contrast medium flows into the blood vessel. In addition, the image 203 acquired in the portal vein phase shows that the contrast agent is flowing into the tumor. In addition, the image 204 acquired in the late phase shows a state in which the contrast agent flows into the surrounding tissue. Thus, it can be seen that the contrast dyed region shifts to blood vessels, tumors, and surrounding tissues from the early phase to the late phase. Here, the curve showing the time change of the contrast agent dark staining is referred to as Time-Density Curve (TDC). For example, a curve representing a temporal change in luminance value (pixel value) in each part is defined as TDC. In TDC, when a contrast medium flows into a tissue, the luminance value of the tissue increases. For this reason, the magnitude of the luminance value represents the state of dark staining of the contrast medium. Therefore, the TDC representing the time change of the luminance value in each part represents the time change of the contrast agent dark dyeing in each part (contrast dye deep dyeing process).

  In addition, after the ablation treatment (after surgery), the ablated tumor is not darkly stained by the contrast medium. Therefore, the temporal change in the brightness value (pixel value) of the tumor is significantly different before and after the operation. That is, in the tumor, the shape of the Time-Density Curve is remarkably different between before and after the operation. The medical image processing apparatus 1 according to this embodiment extracts a region of each part from a medical image using a temporal change of a contrast agent darkly dyed region, and uses a mask information representing each part to generate a plurality of medical images. Adjust the position of.

  It should be noted that the site of the subject in this embodiment includes a lesion site. For example, in addition to blood vessels and liver parenchyma, liver tumors and tumors after cauterization are also included in the site of this embodiment.

(Medical image processing apparatus 1)
The medical image processing apparatus 1 includes a data processing unit 2, an image processing unit 5, an image storage unit 9, a data storage unit 10, a display control unit 11, and a user interface (UI) 12. The medical image processing apparatus 1 aligns the positions of a plurality of images based on a plurality of volume data with different shooting times. In this embodiment, the position of the image acquired before the ablation treatment (before surgery) and the position of the image acquired after the ablation treatment (after surgery) are matched. The preoperative image and the postoperative image may be images acquired by the same type of medical image diagnostic apparatus, or images acquired by different types of medical image diagnostic apparatuses. Also good. Hereinafter, each part of the medical image processing apparatus 1 will be described.

(Image storage unit 9)
The image storage unit 9 stores a plurality of volume data acquired by contrast imaging by the medical image diagnostic apparatus 100 and having different imaging times. Here, a plurality of volume data acquired by contrast imaging and a plurality of volume data captured at different times are referred to as “4D contrast image data”. For example, before the ablation treatment (before surgery), the 4D contrast image data before the ablation treatment is acquired by performing contrast imaging with the medical image diagnostic apparatus 100. Similarly, 4D contrast image data after the cauterization treatment is acquired by performing contrast imaging by the medical image diagnostic apparatus 100 after the cauterization treatment (postoperative). The 4D contrast image data acquired before and after the operation is stored in the image storage unit 9. The 4D contrast image data acquired before the operation is accompanied by identification information indicating that the image is acquired before the operation. Similarly, identification information indicating that the image is acquired after the operation is attached to the 4D contrast image data acquired after the operation. Thereby, based on the identification information, 4D contrast image data acquired before or after surgery can be read from the image storage unit 9. The image storage unit 9 corresponds to an example of the “storage unit” of the present invention. The 4D contrast image data acquired before the ablation treatment (before surgery) corresponds to one example of the “first 4D contrast image data” of the present invention, and the 4D acquired after the ablation treatment (after surgery). The contrast image data corresponds to an example of “second 4D contrast image data” of the present invention.

(Data processing unit 2)
The data processing unit 2 includes a curve model creation unit 3 and a mask creation unit 4, specifies a time-density curve model (hereinafter referred to as “TDC model”) at each site, and the TDC model and 4D contrast Based on the image data, mask information representing the region of each part is generated. Hereinafter, the curve model creation unit 3 and the mask creation unit 4 will be described.

(Curve model creation part 3)
The curve model creation unit 3 includes a curve creation unit 31, a classification unit 32, an average curve creation unit 33, and a curve model identification unit 34, and creates a TDC model in each part.

  Here, a general example of the TDC model of the liver before surgery and a general example of the TDC model of the liver after surgery will be described with reference to FIG. FIG. 3 is a diagram showing a graph (TDC) showing a change in luminance of each part with respect to time. FIG. 3A shows a general example of the TDC model of the liver before the operation, and FIG. 3B shows a general example of the TDC model of the liver after the operation. 3A and 3B, the horizontal axis indicates time, and the vertical axis indicates image brightness. In FIG. 3A, a TDC model 301 represents a general time change of a luminance value (pixel value) in a blood vessel. That is, the TDC model 301 represents a general dark staining process of a contrast agent in a blood vessel. Further, the TDC model 302 represents a general temporal change in luminance value in normal liver parenchyma. Further, the TDC model 303 represents a general temporal change in the luminance value in the liver tumor. On the other hand, in FIG. 3B, a TDC model 401 represents a general time change of a luminance value in a blood vessel. Further, the TDC model 402 represents a general temporal change in luminance value in normal liver parenchyma. Further, the TDC model 403 represents a general temporal change in the luminance value in the liver tumor. The TDC models 301 to 303 and the TDC models 401 to 403 are graphs that are also obtained statistically, for example.

  As shown in FIGS. 3A and 3B, the TDC model 301 and the TDC model 401 are graphs having substantially the same shape, and the shapes are not significantly different. In this way, in the blood vessels, the process of dark staining does not differ significantly before and after the operation, and is almost the same. Similarly, the TDC model 302 and the TDC model 402 are graphs having substantially the same shape, and the shapes are not significantly different. Thus, in normal liver parenchyma, the process of dark staining does not differ significantly before and after surgery. In blood vessels, the luminance value is maximized in the early phase and gradually decreases with time. In normal liver parenchyma, the luminance value becomes maximum after the portal phase, and immediately after that, the luminance value decreases to the original size.

  On the other hand, when the TDC model 303 and the TDC model 403 are compared, it can be seen that the shapes are significantly different. That is, the process of dark staining of the contrast medium in the tumor is significantly different before and after surgery. Before the operation, the luminance value becomes maximum in the early phase, and immediately after that, the luminance value decreases rapidly. On the other hand, after the operation, since the tissue is destroyed by cauterization, dark staining of the contrast agent does not occur, and as a result, the luminance value does not change.

  The curve model creation unit 3 creates a blood vessel TDC model, a normal liver parenchyma TDC model, and a liver tumor TDC model before and after ablation treatment (before surgery) and after ablation treatment (after surgery). In this embodiment, the curve model creation unit 3 may use a generally estimated TDC model shown in FIGS. 3A and 3B, or actually captured 4D contrast image data. Based on the above, a TDC model of each part before and after surgery may be created. The generally estimated TDC models shown in FIG. 3 are stored in advance in the data storage unit 10. Identification information indicating each part is attached to the TDC model of each part. The curve model creation unit 3 corresponds to an example of the “curve model creation means” of the present invention, and the TDC model corresponds to an example of the “pixel value curve model” of the present invention. In addition, the TDC model before the ablation treatment (before surgery) corresponds to one example of the “first pixel value curve model” of the present invention, and the TDC model after the ablation treatment (after surgery) corresponds to “ This corresponds to an example of “2 pixel value curve model”. Hereinafter, a process for creating a TDC model of each part based on actually captured 4D contrast image data will be described.

(Curve creation unit 31)
The curve creation unit 31 reads the 4D contrast image data acquired before the operation and the 4D contrast image data acquired after the operation from the image storage unit 9. Then, the curve creation unit 31 obtains the TDC of each voxel constituting the volume data. That is, the curve creation unit 31 obtains the time change of the luminance value (pixel value) of each voxel constituting the volume data for each voxel based on the volume data at each time constituting the 4D contrast image data. This change in luminance value over time is defined as the voxel TDC. The curve creation unit 31 obtains the TDC of each voxel before the operation based on the 4D contrast image data before the operation, and further obtains the TDC of each voxel after the operation based on the 4D contrast image data after the operation. Then, the curve creation unit 31 outputs the TDC of each voxel before and after the operation to the classification unit 32 and the first classification unit 41 of the mask creation unit 4. Further, the curve creation unit 31 stores the TDC of each voxel in the data storage unit 10 before and after the operation. The curve creation unit 31 attaches identification information indicating that the data is based on an image acquired before the operation to the TDC of each voxel before the operation, and adds a post-operation to the TDC of each voxel after the operation. Identification information indicating that the data is based on the acquired image is attached. The curve creation unit 31 corresponds to an example of the “curve creation means” of the present invention, and the TDC of each voxel corresponds to an example of the “luminance curve” of the present invention. The TDC of each voxel before the ablation treatment (before surgery) corresponds to one example of the “first pixel value curve” of the present invention, and the TDC of each voxel after the ablation treatment (after surgery) is the present invention. Corresponds to an example of the “second pixel value curve”.

  Further, a curve representing the change over time of the change rate of the luminance value (luminance value) in each part may be TDC. In this case, the curve creation unit 31 obtains the time change of the change rate of the luminance value in each voxel and defines a curve representing the time change of the change rate as the TDC of each voxel. In this embodiment, as an example, each process will be described using a curve representing a temporal change in luminance value as TDC.

(Classification part 32)
The classification unit 32 classifies the TDC of each voxel before surgery for each similar curve by a known method, and further classifies the TDC of each voxel after surgery for each similar curve. For example, the classification unit 32 fits the TDCs of the voxels, and classifies the TDCs having differences in luminance values (curve shapes) within a predetermined range as similar curves. The classification unit 32 corresponds to an example of “classification means” of the present invention.

(Average curve creation unit 33)
Then, the average curve creation unit 33 obtains an average TDC luminance value (pixel value) for each classification, thereby obtaining an average TDC having an average luminance value for each classification. Thereby, the average curve creation part 33 calculates | requires the average TDC from which a luminance value becomes an average value about TDC before an operation and TDC after an operation, respectively. Then, the average curve creating unit 33 outputs each average TDC before the operation and each average TDC after the operation to the curve model specifying unit 34. The average curve creation unit 33 corresponds to an example of the “average curve creation unit” of the present invention, and the average TDC corresponds to an example of the “average pixel value curve” of the present invention.

(Curve model specifying unit 34)
The curve model specifying unit 34 reads a TDC model at each generally estimated portion from the data storage unit 10, compares each average TDC with a TDC model at each generally estimated portion, and calculates each average TDC. Specifies the TDC in which part. The curve model specifying unit 34 specifies, for each of the average TDC before the operation and the average TDC after the operation, which average TDC corresponds to the TDC in which part. That is, the curve model specifying unit 34 compares each average TDC before surgery with a general TDC model at each site before surgery, and which average TDC before surgery corresponds to the TDC at which site. Is identified. Similarly, the curve model specifying unit 34 compares each post-operative average TDC with a general TDC model at each postoperative site, and each postoperative average TDC corresponds to a TDC at any site. To identify.

  Specifically, in the curve model specifying unit 34, each average TDC before and after surgery corresponds to any TDC model among a TDC model in blood vessels, a TDC model in tumors, or a TDC model in normal liver parenchyma. Identify what to do. Thereby, each average TDC before and after operation can be classified for each region. And the curve model specific | specification part 34 defines each average TDC classified for every site | part as a TDC model in each site | part. Note that the curve model specifying unit 34 adds identification information indicating each part to the TDC model in each part. For example, when the average TDC corresponds to a TDC model in a blood vessel, the average TDC represents a change in luminance value over time in the blood vessel. In this case, the curve model specifying unit 34 adds identification information indicating the blood vessel to the average TDC defined as the TDC model in the blood vessel. In this way, the curve model specifying unit 34 classifies each average TDC before and after surgery into each part.

  For example, the curve model specifying unit 34 specifies the characteristics of the curve represented by each average TDC, and specifies the characteristics of the curve represented by the generally estimated TDC model. Then, the curve model specifying unit 34 compares the characteristic of the curve represented by each average TDC with the characteristic of the curve represented by the generally estimated TDC model, to which TDC model each average TDC corresponds. Is identified. As an example, the curve model specifying unit 34 obtains the amount of change in luminance value between predetermined time phases for each average TDC, and further obtains the amount of change in luminance value between predetermined time phases for each TDC model. Then, the curve model specifying unit 34 obtains a difference between the change amount of each average TDC and the change amount of the TDC model, and defines an average TDC in which the difference of the change amount is included within a predetermined value as the TDC model.

  The classified average TDC represents the TDC model of the part, and is defined as the TDC model of the part. As a result, a TDC model based on actually captured 4D contrast image data is created. The TDC model of each part before and after the operation is stored in the data storage unit 10. The curve model specifying unit 34 corresponds to an example of “curve model specifying means” of the present invention. A generally estimated TDC model corresponds to an example of the “estimated pixel value curve model” of the present invention.

(Mask creation unit 4)
The mask creation unit 4 includes a first classification unit 41 and a second classification unit 42, and creates mask information indicating the region of each part by classifying each voxel constituting the volume data into each part. Hereinafter, the processing content of the first classification unit 41 and the processing content of the second classification unit 42 will be described.

(First classification unit 41)
The first classification unit 41 receives the TDC of each voxel before and after the operation from the curve creation unit 31, and further reads the TDC model of each part before and after the operation from the data storage unit 10. Then, the first classification unit 41 compares the TDC of each voxel with the TDC model of each part, and identifies which part of the TDC model the TDC of each voxel corresponds to. TDC is classified into each site. For example, the first classification unit 41 compares the TDC of each voxel with the TDC model of each part and specifies a TDC model in which the TDC of each voxel is similar, so that the TDC of each voxel is in any part. It is specified whether it corresponds to the TDC model. As an example, the first classification unit 41 obtains the amount of change in luminance value between predetermined time phases for the TDC of each voxel, and further obtains the amount of change in luminance value between predetermined time phases for the TDC model of each part. Then, the first classification unit 41 obtains the difference between the change amount of each voxel and the change amount of the TDC model, and the TDC of the voxel in which the difference of the change amount is included within a predetermined value is similar to the TDC model of the part. TDC. Thus, the 1st classification | category part 41 specifies the TDC model similar to TDC of each voxel. The similarity analysis between the TDC of each voxel and the TDC model can be performed using a known similarity evaluation function.

  The first classification unit 41 classifies the TDC of each voxel before surgery into each site by comparing the TDC of each voxel before surgery and the TDC model of each site before surgery. Similarly, the 1st classification | category part 41 classifies TDC of each voxel after surgery into each site | part by comparing TDC of each voxel after surgery with the TDC model of each site | part after surgery. Accordingly, the first classification unit 41 creates first mask information in which each voxel is classified into each part. The first classification unit 41 creates first mask information before surgery and first mask information after surgery. At this time, the 1st classification | category part 41 may specify using the TDC model of each site | part defined by the curve model specific | specification part 34, and may specify using the TDC model generally estimated. .

  For example, the first classification unit 41 assigns identification information indicating a corresponding part to the position of each voxel. Specifically, the first classification unit 41 assigns identification information indicating a blood vessel to a voxel position where the TDC corresponds to a TDC model of the blood vessel. The first classification unit 41 assigns identification information indicating a liver tumor to the position of a voxel whose TDC corresponds to the TDC model of the liver tumor. Further, the first classification unit 41 assigns identification information indicating normal liver parenchyma to the position of a voxel whose TDC corresponds to a TDC model of normal liver parenchyma. Furthermore, the first classification unit 41 assigns identification information indicating a non-image region (non-processing region) to a voxel position where the TDC does not correspond to any of a blood vessel, a liver tumor, and normal liver parenchyma. The non-image area represents an image area that is not used for image alignment. As a result, the first mask information represents the region of each part. The first mask information is three-dimensional mask information because it is information in which identification information indicating a part is assigned to each voxel constituting three-dimensional volume data.

  For example, a numerical value is used as identification information. As an example, a numerical value (identification information) representing a blood vessel is “0”, a numerical value representing a liver tumor is “1”, a numerical value representing a normal liver parenchyma is “2”, and a numerical value representing a non-image area is “−”. 1 ”. Then, the first classification unit 41 assigns a numerical value “0” indicating the blood vessel to the position of the voxel whose TDC corresponds to the TDC model of the blood vessel. In addition, the first classification unit 41 assigns a numerical value “1” indicating a liver tumor to the position of a voxel whose TDC corresponds to a TDC model of a liver tumor. In addition, the first classification unit 41 assigns a numerical value “2” indicating the normal liver parenchyma to the position of the voxel whose TDC corresponds to the normal liver parenchyma. Furthermore, the first classification unit 41 assigns a numerical value “−1” indicating a non-image area to a voxel position where the TDC does not correspond to any of a blood vessel, a liver tumor, and normal liver parenchyma. Note that numerical values used as identification information may be referred to as “clustering values” below.

  As described above, the first classification unit 41 creates first mask information in which identification information indicating a part is assigned to the position of each voxel. The first classification unit 41 creates first mask information before surgery by comparing the TDC of each voxel before surgery with the TDC model of each site before surgery. Similarly, the first classification unit 41 creates first mask information after surgery by comparing the TDC of each voxel after surgery with the TDC model of each site after surgery.

  Here, the mask information will be described with reference to FIG. FIG. 4 is a diagram schematically showing mask information. The image 500 is an image based on volume data acquired before surgery. A mask 501 indicates a mask before the identification information of each part is assigned by the first classification unit 41, and schematically represents the position of each voxel in the volume data. The first mask information 510 is pre-operative first mask information created by the first classification unit 41. The image 600 is an image based on volume data acquired after surgery. A mask 510 shows a mask before the identification information of each part is assigned by the first classification unit 41, and schematically shows the position of each voxel in the volume data. The first mask information 610 is first post-operative mask information created by the first classification unit 41. In FIG. 4, for convenience of explanation, the first mask information 510 and the second mask information 610 are planarly represented. In this embodiment, the first mask information 510 and the second mask information 610 indicate a part because the first mask information is created by assigning identification information indicating the part to each voxel constituting the volume data. The identification information is information allocated and distributed in a three-dimensional area.

  In the first mask information 510 and the second mask information 610, the region to which the numerical value “0” is assigned indicates the blood vessel region, the region to which the numerical value “1” is assigned indicates the liver tumor region, and the numerical value The area to which “2” is assigned represents the “normal liver parenchyma” area. An area to which the numerical value “−1” is assigned represents a “non-image area”. In the tumor region, the luminance value (pixel value) changes before and after the operation, but in the region other than the tumor, the luminance value (pixel value) does not change before and after the operation.

  The first classification unit 41 outputs the first mask information before and after the operation to the second classification unit 42 and the data storage unit 10. The data storage unit 10 stores first mask information before and after surgery. The first classification unit 41 corresponds to an example of the “first classification unit” of the present invention.

(Second classification unit 42)
The second classifying unit 42 receives the first mask information output from the first classifying unit 41, and selects a region that is not suitable for image alignment among the regions represented by the first mask information. Reclassify to image area. The mask information created by this reclassification is set as second mask information. For example, the second classification unit 42 creates the second mask information by excluding, from the first mask information, the region of the region other than the region where the shape of the TDC model substantially matches before and after the operation. Since the luminance value (pixel value) changes remarkably in a part other than the part where the shape of the TDC model is substantially the same before and after the operation, it is not suitable for the image alignment target. Therefore, the region of the part where the luminance value changes remarkably is excluded from the first mask information.

  In this embodiment, in order to exclude the region classified as a liver tumor by the first classification unit 41 from the first mask information, the region classified as a liver tumor is reclassified as a non-image region. Specifically, the second classification unit 42 assigns a numerical value “−1” indicating a non-image area to the position of the voxel to which the numerical value “1” is assigned. In the tumor region, the luminance value (pixel value) changes significantly between before and after the operation, so that the tumor region is not suitable for image alignment. For example, as shown in FIG. 3, in the tumor region, the shape of the TDC model is significantly different before and after the operation, and the shapes do not almost coincide. Therefore, the voxels (regions) corresponding to the liver tumors are reclassified into the non-image regions in order to exclude the voxels classified as the liver tumors from the target for alignment. The second classification unit 42 outputs the second mask information to the feature amount image generation unit 6 of the image processing unit 5. Further, the second mask information is stored in the data storage unit 10. The second classification unit 42 corresponds to an example of the “second classification unit” of the present invention.

(Image processing unit 5)
The image processing unit 5 includes a feature amount image generation unit 6, a registration unit 51, and a display image generation unit 52, and generates feature amount image data used for image registration, image registration, and The image data for display is created. Hereinafter, the feature amount image generation unit 6, the alignment unit 51, and the display image generation unit 52 will be described.

(Feature image generator 6)
The feature amount image generation unit 6 includes a first image generation unit 7 and a second image generation unit 8. The feature amount image generation unit 6 receives the second mask information output from the mask creation unit 4, and generates feature amount image data used for image alignment based on the second mask information. At this time, the feature amount image generation unit 6 generates preoperative feature amount image data based on the preoperative second mask information, and the postoperative feature amount image data based on the postoperative second mask information. Generate. In this embodiment, the first image generation unit 7 and the second image generation unit 8 respectively generate feature amount image data. As the feature amount image data used for image alignment, the feature amount image data generated by the first image generation unit 7 or the feature amount image data generated by the second image generation unit 8 may be generated. You may do it. The medical image processing apparatus 1 may include both the first image generation unit 7 and the second image generation unit 8, or may include either one of the image generation units. The feature amount image generation unit 6 corresponds to an example of “feature amount image generation means” of the present invention.

(First image generation unit 7)
The first image generation unit 7 generates feature amount image data by assigning a feature amount specified from the TDC of each voxel to each voxel included in each region indicated by the second mask information. For example, the first image generation unit 7 reads the TDC of each voxel from the data storage unit 10 and obtains the amount of change in luminance value (pixel value) between arbitrary time phases for each voxel. Here, the amount of change in luminance value (pixel value) corresponds to an example of a feature amount. An arbitrary time phase can be designated by the operator using the operation unit 14. The operator designates a desired time phase using the operation unit 14. Specifically, two different time phases are specified. Information indicating the two time phases designated by the operation unit 14 is output from the user interface (UI) 12 to the image processing unit 5. The first image generation unit 7 receives information indicating two time phases designated by the operation unit 14, and obtains the amount of change in luminance value between the two time phases for each voxel. In this embodiment, the first image generation unit 7 reads the TDC of each voxel before operation from the data storage unit 10, obtains the amount of change in luminance value between arbitrary time phases for each voxel, and further, each voxel after operation. Are read from the data storage unit 10, and the amount of change in luminance value between arbitrary time phases is obtained for each voxel. And the 1st image generation part 7 receives the 2nd mask information output from the mask preparation part 4, It is the area | region of each site | part represented by the 2nd mask information, Comprising: The non-image area | region was remove | excluded The amount of change in luminance value is mapped to the position of each voxel in the region. Specifically, the first image generation unit 7 excludes a region (voxel) to which a value “−1” indicating a non-image region is assigned in the second mask information, and a value “0” indicating a blood vessel. The amount of change in luminance value is mapped to the area to which is assigned and the area to which the value “2” indicating normal liver parenchyma is assigned. As described above, since the value “−1” indicating the non-image area is assigned to the area specified as the liver tumor, the change amount of the luminance value is not mapped in the area of the liver tumor. The image data in which the change amount of the luminance value is mapped except for the non-image area is the feature amount image data used for image alignment. For convenience of explanation, the feature amount image data generated by the first image generation unit 7 is referred to as “first feature amount image data”. The first image generation unit 7 outputs the first feature amount image data to the alignment unit 51 and the image storage unit 9. The image storage unit 9 stores first feature amount image data.

  In addition, the first image generation unit 7 may use, as the first feature amount image data, image data represented by a region excluding the non-image region among regions of each part indicated by the second mask information. That is, the first image generation unit 7 may use the second mask information to which the clustering value (identification information) is assigned as it is as the first feature amount image data. Here, the clustering value (identification information) corresponds to an example of the feature amount. Specifically, the first image generation unit 7 uses, as the first feature amount, image data obtained by excluding an area to which a value “−1” indicating a non-image area is assigned among the areas indicated by the second mask information. Defined as image data.

(Second image generation unit 8)
The second image generation unit 8 includes a change amount calculation unit 81 and a pixel value conversion unit 82, and generates feature amount image data by a process different from that of the first image generation unit 7. The feature amount image data generated by the second image generation unit 8 is referred to as “second feature amount image data”. Alignment of medical images with different imaging conditions such as the type of medical image diagnostic device and contrast dyeing time, or different imaging targets such as the presence or absence of a systematically changing region and the presence or absence of a cauterized tumor region In some cases, the positions of medical images are aligned. In this embodiment, (1) adjustment of a luminance value (pixel value) in consideration of a luminance distribution unique to a medical image diagnostic apparatus, and (2) a luminance value (pixel) for aligning positions of images taken at different time phases Value) and (3) setting a luminance value (pixel value) for an area unsuitable for alignment, and generating feature image data for alignment. In this embodiment, in the second mask information, the region of the liver tumor is classified as a region that is not suitable for alignment. Therefore, the luminance value setting (3) is performed by the second mask information. The second image generation unit 8 performs setting of luminance values (1) and setting of luminance values (2), and generates feature amount image data for alignment.

  In this embodiment, as an example, before ablation (preoperative), 4D contrast image data (hereinafter referred to as “4DCT contrast image data”) is obtained by performing contrast imaging using the X-ray CT apparatus for the medical image diagnostic apparatus 100. Obtain). Further, after ablation (postoperative), 4D contrast image data (hereinafter referred to as “4D ultrasound contrast image data”) is acquired by performing contrast imaging on the medical image diagnostic apparatus 100 using an ultrasound diagnostic apparatus. To do. Here, as an example, a case where the position of the CT contrast image in the portal phase is matched with the position of the ultrasonic image in the late phase will be described. The second image generation unit 8 generates second feature amount image data used for aligning positions of images having different types of medical image diagnostic apparatuses and having different time phases.

  As a stage before the second image generation unit 8 generates the second feature amount image data, the curve model generation unit 3 obtains the TDC of each voxel based on the 4DCT contrast image data, similarly to the above-described processing, Classify voxel TDCs by similar curves. Then, the curve model creation unit 3 obtains an average TDC for each classification, and specifies the TDC model of each part before surgery. Similarly, the curve creation unit 3 obtains the TDC of each voxel based on the 4D ultrasound contrast image data, and classifies the TDC of each voxel for each similar curve. Then, the curve model creation unit 3 obtains an average TDC for each classification, and specifies the TDC model of each part after surgery. As described above, the TDC model of each part before surgery and the TDC model of each part after surgery are stored in the data storage unit 10.

  Moreover, the mask preparation part 4 classify | categorizes TDC of each voxel before and after operation into each site | part similarly to the process mentioned above, The 1st mask information before an operation and the 1st mask information after an operation And create. Further, the mask creation unit 4 creates the pre-operative second mask information and the post-operative second mask information by classifying the region classified as a liver tumor into a non-image region. The second mask information before surgery is mask information created based on 4DCT contrast image data, and the second mask information after surgery is mask information created based on 4D ultrasound contrast image data. As described above, the preoperative second mask information and the postoperative second mask information are stored in the data storage unit 10.

  Here, a TDC model of each part before surgery and a TDC model of each part after surgery will be described with reference to FIG. FIG. 5 is a diagram showing a graph (TDC) showing a change in luminance of each part with respect to time. In FIG. 5, the horizontal axis indicates time, and the vertical axis indicates the luminance of the image. TDC models 701, 702, and 703 are TDC models created based on pre-operative 4DCT contrast image data. TDC models 711, 712, and 713 are TDC models created based on post-operative 4D ultrasound contrast image data. The TDC model 701 represents a temporal change in the luminance value (pixel value) of the blood vessel before the operation. The TDC model 702 represents a temporal change in the luminance value of normal liver parenchyma before operation. Further, the TDC model 703 represents a temporal change in the luminance value of the liver tumor before the operation. On the other hand, the TDC model 711 represents a temporal change in the luminance value of the blood vessel after the operation. The TDC model 712 represents a temporal change in the luminance value of normal liver parenchyma after the operation. A TDC model 713 represents a temporal change in the luminance value of the liver tumor after the operation.

(Change amount calculation unit 81)
The change amount calculation unit 81 obtains a change amount between the pre-operative TDC model and the post-operative TDC model for each part. As an example, a case where the position of the CT contrast image in the portal vein phase (time phase t1) before surgery and the position of the ultrasound contrast image in the later phase (time phase t2) after surgery will be described. For example, when the operator specifies a desired time phase before and after the operation using the operation unit 14, the specified time phase is output to the change amount calculation unit 81. The change amount calculation unit 81 specifies the luminance value (pixel value) of each part in the designated time phase based on the TDC model in each part before surgery. Similarly, the change amount calculation unit 81 specifies the luminance value (pixel value) of each part in the designated time phase based on the TDC model in each part after surgery. The time phase t1 corresponds to an example of the “first time phase” of the present invention, and the time phase t2 corresponds to an example of the “second time phase” of the present invention.

  For example, when the operator designates the preoperative portal vein phase (time phase t1) using the operation unit 14, the change amount calculating unit 81 determines each of the portal vein phases based on the TDC model of each site before surgery. The luminance value (pixel value) of the part is specified. Specifically, the change amount calculation unit 81 specifies the luminance value (pixel value) of the blood vessel in the portal vein phase (time phase t1) based on the TDC model 701 of the blood vessel before surgery. Further, the change amount calculation unit 81 specifies the luminance value (pixel value) of the normal liver parenchyma in the portal phase (time phase t1) based on the TDC model 702 of the normal liver parenchyma before operation.

  Further, when the operator designates the later phase (time phase t2) after the operation using the operation unit 14, the change amount calculation unit 81 determines the luminance of each part in the later phase based on the TDC model of each part after the operation. Specify the value (pixel value). Specifically, the change amount calculation unit 81 identifies the luminance value (pixel value) of the blood vessel in the late phase (time phase t2) based on the TDC model 711 of the blood vessel after surgery. Further, the change amount calculation unit 81 specifies the luminance value (pixel value) of the normal liver parenchyma in the late phase (time phase t2) based on the TDC model 712 of the normal liver parenchyma after the operation.

  Then, the change amount calculation unit 81 obtains the difference between the preoperative luminance value (pixel value) and the postoperative luminance value (pixel value) for each part. For example, when calculating the difference between the luminance values in the blood vessel, as shown in FIG. 5, the change amount calculation unit 81 calculates the luminance value of the blood vessel in the portal vein phase (time phase t1) before surgery and the later phase (time) after surgery. The difference b1 with the blood vessel luminance value in phase t2) is obtained. Similarly, when calculating the difference between the luminance values in the normal liver parenchyma, the change amount calculation unit 81 calculates the luminance value of the normal liver parenchyma in the portal vein phase (time phase t1) before surgery and the late phase (temporal phase after surgery). A difference b2 from the luminance value of normal liver parenchyma at t2) is obtained.

  Further, the change amount calculation unit 81 obtains a time difference Δt between the late phase (time phase t2) and the portal phase (time phase t1). Then, the change amount calculation unit 81 calculates the change amount (b1 / Δt) for the blood vessel by dividing the difference b1 of the luminance value (pixel value) of the blood vessel by the time difference Δt. Similarly, the change amount calculation unit 81 calculates the change amount (b2 / Δt) for the normal liver parenchyma by dividing the difference b2 for the normal liver parenchyma by the time difference Δt. The change amount calculation unit 81 outputs the change amount (b1 / Δt) of the blood vessel and the change amount (b2 / Δt) of the normal liver parenchyma to the pixel value conversion unit 82. The change amount calculation unit 81 corresponds to an example of the “change amount calculation unit” of the present invention.

(Pixel value converter 82)
The pixel value conversion unit 82 uses the above-described change amount for the luminance value (pixel value) of one of the two pieces of image data to be aligned, so that the other image data that is the counterpart of the alignment is used. The luminance value (pixel value) corresponding to the luminance value (pixel value) is converted. By this conversion, the adjustment of the luminance value due to the difference in the medical image diagnostic apparatus and the adjustment of the luminance value due to the difference in time phase are performed. Note that the luminance value converted using the change amount corresponds to an example of the “feature amount” of the present invention.

  For example, when the position of the CT contrast image in the portal vein phase (time phase t1) before surgery and the position of the ultrasound contrast image in the later phase after surgery (time phase t2) are matched, the pixel value conversion unit 82 CT contrast image data (volume data) in the portal vein phase (time phase t1) before operation is read from the image storage unit 9, and ultrasound contrast image data (volume data) in the later phase (time phase t2) after surgery. Are read from the image storage unit 9. Furthermore, the pixel value conversion unit 82 reads the preoperative second mask information and the postoperative second mask information from the data storage unit 10.

  Then, the pixel value conversion unit 82 uses the second mask information before surgery to obtain the region of the blood vessel and the region of normal liver parenchyma in the CT contrast image data of the portal vein phase (time phase t1) before surgery. Identify. The pixel value conversion unit 82 uses the luminance value (pixel value) of each voxel included in the blood vessel region in the CT contrast image data of the portal vein phase before operation by using the change amount (b1 / Δt) of the blood vessel. The luminance value (pixel value) corresponding to the luminance value (pixel value) of the ultrasound contrast image data in the later phase after surgery is converted. Similarly, the pixel value conversion unit 82 converts the luminance value (pixel value) of each voxel included in the normal liver parenchymal region in the CT contrast image data of the portal phase before surgery to the normal liver parenchymal change amount (b2 / By using (Δt), the luminance value (pixel value) corresponding to the luminance value (pixel value) of the ultrasound contrast image data in the later phase after the operation is converted. The image data converted by the pixel value conversion unit 82 will be referred to as “converted image data”.

  For example, the pixel value conversion unit 82 multiplies the luminance value (pixel value) of each voxel included in the blood vessel region in the CT contrast image data of the portal vein phase before surgery by the blood vessel change amount (b1 / Δt). By the multiplication, the luminance value (pixel value) of each voxel contained in the blood vessel region in the CT contrast image data of the portal vein phase before surgery is calculated, and the brightness value (pixel value) of the ultrasound contrast image data in the later phase after surgery. Is converted into a luminance value (pixel value) corresponding to. Similarly, the pixel value conversion unit 82 converts the normal liver parenchyma change amount (b2 //) into the luminance value (pixel value) of each voxel included in the normal liver parenchyma region in the CT contrast image data of the portal phase before surgery. Multiply [Delta] t). By the multiplication, the luminance value (pixel value) of each voxel included in the normal liver parenchyma in the CT contrast image data of the portal vein phase before surgery is obtained, and the brightness value (pixel value) of the ultrasound contrast image data in the later phase after surgery. ) Is converted into a luminance value (pixel value) corresponding to.

  This conversion process adjusts the difference in luminance value (pixel value) due to the time phase difference, and further adjusts the difference in luminance value (pixel value) due to the difference in the medical image diagnostic apparatus, so that the CT contrast image It becomes possible to match the position with the position of the ultrasound contrast image.

  Then, the pixel value conversion unit 82 defines the pre-operative converted image data (volume data) that has been subjected to the conversion process as second pre-operative feature image data, and the contrast-enhanced image data in the later phase after the operation. (Volume data) is defined as post-operation second feature image data. The pixel value conversion unit 82 outputs the second feature amount image data before and after the operation to the alignment unit 51.

  In addition, the pixel value conversion unit 82 converts the luminance value (pixel value) of the ultrasound contrast image data in the later phase after surgery, instead of converting the brightness value (pixel value) of the CT contrast image data in the portal vein phase before surgery. ) May be converted. In this case, the pixel value conversion unit 82 uses the blood vessel change amount (b1 / Δt) as the luminance value of each voxel included in the blood vessel region in the later-phase ultrasound contrast image data after the operation, so that Are converted into luminance values corresponding to the luminance values of the CT contrast image data in the portal vein phase. Similarly, the pixel value conversion unit 82 uses the amount of change (b2 / Δt) of the normal liver parenchyma as the luminance value of each voxel included in the normal liver parenchyma region in the later-phase ultrasound contrast image data after the operation. Thus, the luminance value corresponding to the luminance value of the CT contrast image data in the portal vein phase before operation is converted. Thus, even when the luminance value of post-operative ultrasound contrast image data is converted, the luminance value difference due to the time phase difference is adjusted, and further the luminance value difference due to the difference in the medical image diagnostic apparatus is adjusted. It is possible to adjust the position of the CT contrast image and the position of the ultrasonic contrast image. Then, the pixel value conversion unit 82 defines the CT contrast image data in the portal vein phase before surgery as the second feature image data before surgery, and converts the post-operative converted image data subjected to the conversion processing to It defines as 2nd feature-value image data. The pixel value conversion unit 82 outputs the second feature amount image data before and after the operation to the alignment unit 51. The pixel value conversion unit 82 corresponds to an example of the “pixel value conversion unit” of the present invention.

  In this embodiment, the second image generation unit 8 targets the pre-operative image and the post-operative image acquired by different medical image diagnostic apparatuses, and the luminance values of the images acquired at different time phases. However, the present invention is not limited to the processing. For example, even if the second image generation unit 8 adjusts the luminance values of images acquired at different time phases for a preoperative image and a postoperative image acquired by the same medical image diagnostic apparatus. good.

(Positioning part 51)
The alignment unit 51 accepts preoperative feature amount image data and postoperative feature amount image data from the feature amount image generation unit 6, and positions of the preoperative feature amount image and postoperative feature amount image. And match.

  When the first feature amount image data is generated by the first image generation unit 7, the registration unit 51 and the first post-operative first feature amount image data generated by the first image generation unit 7. The feature amount image data is received, and the position of the first feature amount image before the operation and the position of the first feature amount image after the operation are matched. Further, when the second feature amount image data is generated by the second image generation unit 8, the alignment unit 51 and the pre-operation second feature amount image data generated by the second image generation unit 7 and the post-operation amount. The second feature amount image data is received, and the position of the second feature amount image before the operation and the position of the second feature amount image after the operation are matched.

  In this embodiment, the alignment unit 51 performs image alignment by applying the MI method (mutual information method) to the feature amount image. For example, the alignment unit 51 performs image alignment by applying the MI method to the first feature value image before surgery and the first feature value image after surgery. Further, the registration unit 51 performs image registration by applying the MI method to the second feature value image before surgery and the second feature value image after surgery. The MI method may be a known method described in the document: Mutuality Image Registration by Maximum of Mutual Information (1997).

  In addition, the alignment unit 51 reads pre-operative second mask information and post-operative second mask information from the data storage unit 10, extracts morphological information of each part from the second mask information, and performs template matching, etc. Image alignment may be performed by a known method using morphological information. The alignment unit 51 corresponds to an example of the “alignment unit” of the present invention.

  Then, the alignment unit 51 outputs position information indicating the relative positional relationship between the preoperative feature amount image and the postoperative feature amount image after the alignment to the display image generation unit 52.

(Display Image Generation Unit 52)
The display image generation unit 52 generates display image data by performing predetermined image processing on the volume data stored in the image storage unit 9. For example, the display image generation unit 52 generates three-dimensional image data in which the tissue is three-dimensionally represented by performing volume rendering on the volume data. The display image generation unit 52 may generate image data (MPR image data) in an arbitrary cross section by performing MPR (Multi Planar Reconstruction) processing on the captured image. Then, the display image generation unit 52 outputs medical image data such as 3D image data and MPR image data to the display control unit 11. The display control unit 11 causes the display unit 13 to display a three-dimensional image based on the three-dimensional image data and an MPR image based on the MPR image data. The display image generation unit 52 corresponds to an example of the “display image generation unit” of the present invention, and the display control unit 11 corresponds to an example of the “display control unit” of the present invention.

  For example, the display image generation unit 52 generates three-dimensional image data of each time phase by performing volume rendering on each volume data of 4D contrast image data acquired before and after surgery. The display image generation unit 52 may generate MPR image data in an arbitrary cross section of each time phase by performing MPR processing on 4D contrast image data acquired before and after surgery. At this time, the display image generation unit 52 performs rendering by combining the position of the preoperative volume data and the position of the postoperative volume data in accordance with the relative positional relationship indicated by the positional information output from the alignment unit 51. By performing this processing, three-dimensional image data in which the positions are the same before and after the operation and MPR image data in which the positions of the cross sections are the same before and after the operation are generated. Then, the display control unit 11 causes the display unit 13 to display the three-dimensional image and the MPR image that are aligned. The image data for display before the ablation treatment (before surgery) corresponds to one example of the “first display image data” of the present invention, and the image data for display after the ablation treatment (after surgery) This corresponds to an example of “second display image data” according to the present invention.

  Here, an example of the aligned image is shown in FIG. FIG. 6 is a diagram showing a registered image. An image 800 is an MPR image acquired by the ultrasonic diagnostic apparatus before the operation, and an image 810 is an MPR image acquired by the ultrasonic diagnostic apparatus after the operation. The image 800 shows a tumor 801 contrasted before cauterization. On the other hand, a tumor 811 after cauterization is shown in the image 810. The display control unit 11 displays the aligned image 800 and the image 810 side by side on the display unit 13. Since the positions of the pre-operative image 800 and the post-operative image 810 are aligned, it is possible to easily compare the shape of the tumor and the surrounding tissues.

  Further, the operator gives an instruction to change display conditions such as an instruction to change a cross section, an instruction to enlarge or reduce an image, an instruction to move in a plane, an instruction to rotate, etc. to one image using the operation unit 13. The other image may be subjected to each process in conjunction with the change instruction. For example, when an instruction to change the cross section is given by the operation unit 13, the display image generation unit 52 performs MPR processing by changing the position of the cross section according to the instruction, thereby performing MPR in a new cross section before and after the operation. Image data is generated before and after surgery. Then, the display control unit 11 causes the display unit 13 to display the MPR image in the new cross section. In addition, the display control unit 11 causes the display unit 13 to display a pre-operative image and a post-operative image on the display unit 13 by changing display conditions according to an image enlargement or reduction instruction, a movement instruction, or a rotation instruction. Further, when the operator designates a desired position on the image using the operation unit 13, the coordinate information of the designated position is output to the display control unit 11, and the display control unit 11 performs the designated coordinate (voxel). ) May be read from the data storage unit 10 and the TDC may be displayed on the display unit 13.

  The user interface (UI) 12 includes a display unit 13 and an operation unit 14. The display unit 13 includes a monitor such as a CRT or a liquid crystal display, and displays a three-dimensional image or an MPR image on the screen. The operation unit 14 includes a pointing device such as a joystick or a trackball, a switch, various buttons, or a keyboard.

  The curve model creation unit 3, the mask creation unit 4, the feature amount image generation unit 6, the alignment unit 51, the display image generation unit 52, and the display control unit 11 each have a CPU (not shown) and a storage such as a ROM and a RAM. Device. In the storage device, a curve model creation program for executing the function of the curve model creation unit 3, a mask creation program for executing the function of the mask creation unit 4, and a function of the feature quantity image generation unit 6 are executed. In order to execute the feature amount image generation program, the alignment program for executing the function of the alignment unit 51, the display image generation program for executing the function of the display image generation unit 52, and the function of the display control unit 11 The display control program is stored.

  The curve model creation program includes a curve creation program for executing the function of the curve creation unit 31, a classification program for executing the function of the classification unit 32, and an average for executing the function of the average curve creation unit 33. A curve creation program and a curve model specifying program for executing the function of the curve model specifying unit 34 are included. In addition, the mask creation program includes a first classification program for executing the function of the first classification unit 41 and a second classification program for executing the function of the second classification unit 42. The feature image generation program includes a first image generation program for executing the function of the first image generation unit 7 and a second image generation program for executing the function of the second image generation unit 8. include. The second image generation program includes a change amount calculation program for executing the function of the change amount calculation unit 81 and a pixel value conversion program for executing the function of the pixel value conversion unit 82. .

  Then, the CPU executes the curve creation program to create a TDC for each voxel. Further, the CPU executes a classification program to classify the TDC of each voxel into a similar curve. Further, the CPU executes an average curve creation program to create an average TDC that represents the average value of each classified TDC. Then, the CPU executes the curve model specifying program to define the TDC model of each part.

  Further, the CPU executes the first classification program to create first mask information in which the clustering value is assigned to the position of each voxel. Further, the CPU executes the second classification program, so that the second mask information is created by reassigning the clustering value assigned as the tumor to the clustering value of the non-image region.

  In addition, the CPU executes the first image generation program to generate first feature amount image data. In addition, the CPU executes the change amount calculation program to obtain the change amount between the pre-operative TDC model and the post-operative TDC model. In addition, the CPU executes a pixel value conversion program to convert the pixel value of one of the image data to be aligned using the change amount.

  Further, the CPU executes the alignment program to align the positions of the two feature amount images. In addition, the CPU executes a display image generation program to generate three-dimensional image data for display and MPR image data based on volume data. Then, the CPU executes a display control program to display a three-dimensional image based on the three-dimensional image data on the display unit 13.

(Operation)
Next, a series of operations by the medical image processing apparatus 1 according to this embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing a series of operations by the medical image processing apparatus according to the embodiment of the present invention.

(Step S01)
First, preoperative 4D contrast image data and postoperative 4D contrast image data are acquired by performing contrast imaging by the medical image diagnostic apparatus 100 before and after surgery. Pre-operative 4D contrast image data and post-operative 4D contrast image data are output from the medical image diagnostic apparatus 100 to the medical image processing apparatus 1 and stored in the image storage unit 9 of the medical image processing apparatus 1. In addition, preoperative 4D contrast image data and postoperative 4D contrast image data may be acquired by the same type of medical image diagnostic apparatus, or preoperative 4D contrast image data may be acquired by different types of medical image diagnostic apparatuses. You may acquire postoperative 4D contrast image data.

(Step S02)
The curve model creation unit 3 creates a TDC model for each part before surgery. In this embodiment, the curve model creation unit 3 creates a pre-operative blood vessel TDC model, a normal liver parenchyma TDC model, and a liver tumor TDC model. For example, the curve model creation unit 3 reads the generally estimated TDC model of each part from the data processing unit 2, and defines the generally estimated TDC model of each part as the TDC model of each part. . Alternatively, the curve model creation unit 3 may create a TDC model for each part based on actually captured 4D contrast image data. In this case, the curve model creation unit 3 creates a TDC for each voxel, classifies each TDC for each similar curve, and creates an average TDC that is an average of the classified TDCs. Then, the curve model creation unit 3 compares each average TDC with a generally estimated TDC model of each part, and specifies which part of the average TDC corresponds to the TDC. The classified average TDC represents the TDC model of the part, and the curve model creation unit 3 defines the classified average TDC as a TDC model. The TDC model of each part is stored in the data storage unit 10.

(Step S03)
Next, the curve creation unit 31 creates a TDC for each voxel before surgery. In step S02, when a TDC model is created from actually acquired 4D contrast image data, the TDC of each voxel is created in the creation process, and therefore the processing in step S03 may be omitted. . The TDC of each voxel is stored in the data storage unit 10.

(Step S04)
The first classification unit 41 compares the TDC of each voxel with the TDC model of each part, and identifies the TDC model of which part the TDC of each voxel corresponds to, thereby determining the TDC of each voxel. Classify into each part. By this classification, the first classification unit 41 generates first mask information before surgery. For example, the first classification unit 41 assigns a numerical value “0” indicating a blood vessel to the position of a voxel whose TDC corresponds to the TDC model of the blood vessel. Thus, the 1st classification | category part 41 produces the 1st mask information before a surgery by assigning the numerical value which shows a site | part to the position of each voxel. The first mask information is stored in the data storage unit 10.

(Step S05)
Then, the second classification unit 42 creates second mask information by reclassifying a region that is not suitable for image alignment among the regions represented by the first mask information to a non-image region. To do. In this embodiment, the second classification unit 42 assigns a numerical value “−1” indicating a non-image area to the position of a voxel to which a numerical value “1” indicating a liver tumor is assigned. As a result, the region classified as the liver tumor is excluded from the alignment target. The second mask information is stored in the data storage unit 10.

(Step S06)
Then, the data processing unit 2 determines whether or not mask information has been created for all images to be aligned. For example, when the pre-operative second mask information is created and the post-operative second mask information is not created (No at Step S06), the post-operative 4D contrast image data is targeted and the steps from Step S02 are performed. By executing the process of S05, post-operative second mask information is created. If mask information has been created for all images (step S06, Yes), the processing from step S07 is executed.

(Step S07, Step S08)
Next, the feature amount image generation unit 6 generates feature amount image data for alignment using the second mask information before and after the operation. In the medical image processing apparatus 1 according to this embodiment, the first feature amount image data may be generated by the first image generation unit 7 (step S07), and the second image is used instead of the first feature amount image data. Second feature amount image data may be generated by the generation unit 8 (step S08). The feature amount image generation unit 6 includes first feature amount image data generated by the first image generation unit 7 before and after operation, or pre-operation and post-operation generated by the second image generation unit 8. 2 Outputs the feature amount image data to the alignment unit 51.

(Step S09)
The alignment unit 51 performs image alignment by applying the MI method to the pre-operative feature amount image and the post-operative feature amount image. When the first feature amount image data is generated by the first image generation unit 7, the alignment unit 51 uses the MI method to calculate the position of the first feature amount image before the operation and the position of the first feature amount image after the operation. Adjust according to. When the second feature amount image data is generated by the second image generation unit 8, the alignment unit 51 determines the position of the second feature amount image before the operation and the position of the second feature amount image after the operation. Match by MI method. Then, the alignment unit 51 outputs position information indicating the relative positional relationship between the position of the preoperative feature amount image and the postoperative feature amount image to the display image generating unit 52.

(Step S10)
In accordance with the relative positional relationship indicated by the positional information output from the alignment unit 51, the display image generation unit 52 matches the position of volume data before surgery and the position of volume data after surgery to perform volume rendering or MPR. Image processing such as processing is performed. As a result, three-dimensional image data whose positions match before and after surgery and MPR image data whose cross-sectional positions match before and after surgery are generated. For example, the display control unit 11 causes the display unit 13 to display a pre-operative MPR image and a post-operative MPR image in which the positions of the cross sections coincide.

  As described above, according to the medical image processing apparatus 1 according to this embodiment, the TDC of each voxel is created, and the region of each part and its form are specified by using the features of contrast-enhanced dyeing of each part. It becomes possible. Then, by excluding the areas that are not suitable for image alignment from the alignment process, even if a part of the image has changed between before and after surgery, the position of multiple images can be determined more accurately. It is possible to perform matching. Specifically, the second mask information excluding the ablated tumor region is created, and the position of the preoperative image and the position of the postoperative image are determined based on the feature amount image based on the second mask information. By combining them, it is possible to more accurately match the position of the pre-operative image and the post-operative image even when the brightness value of the tumor changes before and after the operation. . In other words, according to the medical image processing apparatus 1 according to this embodiment, since the second mask information excluding the region where the luminance value changes is used, image alignment using the MI method that does not require morphological information is performed. As a result, image alignment can be performed more accurately.

  In addition, by creating a TDC for each voxel and using the features of contrast-enhanced dyeing at each site, it is possible to specify the region of each site and its form. Therefore, it is possible to perform alignment using form information as in a known alignment method.

  Further, since the mask information created based on the TDC model can be used in common for all time phases, it is possible to extract the region of each part from the image data in each time phase. Further, it is possible to align images in different time phases, such as an early phase and a late phase. Furthermore, since mask information can be created based on the contrast image acquired by each medical image diagnostic apparatus, it is possible to align the images acquired by different medical image diagnostic apparatuses.

  Furthermore, the alignment can be performed more accurately by performing the alignment in consideration of the characteristics of each medical image diagnostic apparatus and the characteristics in each time phase. Specifically, by adjusting the position of the image based on the second feature amount image generated by the second image generation unit 8, the difference in the luminance value caused by the difference in the medical image diagnostic apparatus is adjusted, The difference in luminance value due to the time phase difference can be adjusted. Then, by performing alignment using the second feature image that has been subjected to brightness adjustment, it is possible to more accurately align the positions of a plurality of images acquired at different time phases by different medical image diagnostic apparatuses. It becomes.

  As described above, according to the medical image processing apparatus 1 according to this embodiment, it is possible to compare the tumor area before cauterization and the tumor area after cauterization at the same position without performing complicated operations. A more accurate postoperative diagnosis becomes possible. For example, since the tumor area before ablation and the tumor area after ablation can be compared in the same cross section, more accurate postoperative diagnosis becomes possible.

1 is a block diagram illustrating a medical image processing apparatus according to an embodiment of the present invention. It is a figure which shows the image showing a mode that a contrast agent flows in in order. It is a figure which shows the graph (TDC) showing the change of the brightness | luminance of each site | part with respect to time. It is a figure which shows mask information typically. It is a figure which shows the graph showing the change of the brightness | luminance of each site | part with respect to time. It is a figure which shows the image aligned. It is a flowchart which shows a series of operation | movement by the medical image processing apparatus which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Medical image processing apparatus 2 Data processing part 3 Curve model creation part 4 Mask creation part 5 Image processing part 6 Feature-value image generation part 7 1st image generation part 8 2nd image generation part 9 Image storage part 10 Data storage part 11 Display Control unit 12 User interface (UI)
DESCRIPTION OF SYMBOLS 13 Display part 14 Operation part 31 Curve creation part 32 Classification part 33 Average curve creation part 34 Curve model specification part 41 1st classification part 42 2nd classification part 51 Position alignment part 52 Display image generation part 81 Change amount calculation part 82 Pixel value Conversion unit 100 medical image diagnostic apparatus

Claims (8)

Storage means for storing a plurality of 4D contrast image data including volume data obtained by imaging a subject into which a contrast medium is injected before and after medical treatment of the subject;
Based on each of the previous and subsequent 4D contrast image data stored in the storage means, the previous pixel value curve and the subsequent pixel value curve representing the temporal change of the pixel value in each voxel are created. Curve creation means to
A curve model creating means for obtaining a pixel value curve model representing a temporal change of a pixel value in each part of the subject for each of the before and after,
By comparing the voxel pixel value curve and the pixel value curve model of each part for each of the previous and the subsequent, and classifying the voxel pixel value curve for each part of the subject, First classification means for creating the previous first mask information and the subsequent first mask information indicating a region of a part;
By excluding a region other than the region where the shape of the pixel value curve model substantially matches between the previous and subsequent 4D contrast image data from the first and subsequent first mask information, Second classification means for creating the previous second mask information and the subsequent second mask information;
By assigning a feature amount specified from each of the previous and subsequent pixel value curves to each voxel included in each region indicated by the second mask information before and after the previous, Feature amount image generation means for generating feature amount image data and the subsequent feature amount image data; and
Image alignment means for aligning the positions of the feature image before and after the image;
Display image generation means for generating, for each 4D contrast image data, display image data aligned according to the alignment based on volume data included in each of the previous and subsequent 4D contrast image data;
Display control means for displaying display images side by side on the display means based on display image data for each of the 4D contrast image data;
A medical image processing apparatus comprising:   The first classification unit compares the pixel value curve of each voxel with the pixel value curve model of each part, and determines a part of the pixel value curve similar to the shape of the pixel value curve model for each voxel. The medical image processing apparatus according to claim 1, wherein the first mask information is created by specifying the first mask information.   The second classifying means classifies a region classified as a tumor as a region of a region other than a region where the shape of the pixel value curve model substantially matches between the previous and subsequent 4D contrast image data. 3. The medical image processing apparatus according to claim 1, wherein the second mask information before and after the first mask information is created by excluding the first mask information from the subsequent first mask information. 4. The curve model creating means includes
Classification means for classifying the pixel value curve of each voxel for each similar curve;
Average curve creating means for creating an average pixel value curve having an average value of pixel values by obtaining an average of the pixel value curves for each of the classifications,
The estimated pixel value curve model representing the temporal change of the pixel value in each part estimated in advance and the average pixel value curve for each classification are compared, and the shape of the average pixel value curve is that of the estimated pixel value curve model. A curve model specifying unit that defines each average pixel value curve as a pixel value curve model of each part by specifying a part similar to a shape for each of the classifications;
The medical image processing apparatus according to any one of claims 1 to 3, further comprising:   The feature amount image generation means obtains a change amount of a pixel value of each voxel between arbitrarily designated time phases as the feature amount based on a pixel value curve of each voxel, and the change amount is determined by the second mask. 5. The medical image processing apparatus according to claim 1, wherein the feature image data is generated by assigning to each voxel included in each region indicated by information. 6. The feature image generation unit includes:
Based on the pixel value curve model of each part created based on the first 4D contrast image data of one of the previous or subsequent ones, the pixel value of each part in the arbitrarily specified first time phase is obtained. Further, the pixel value of each part in the second time phase arbitrarily specified based on the pixel value curve model of each part created based on the other second 4D contrast image data before or after the previous or subsequent Change amount calculation means for obtaining a change amount of the pixel value of each part by obtaining a difference between the pixel value in the first time phase and the pixel value in the second time phase for the same part,
Volume data included in the first 4D contrast image data, and included in the region of each part indicated by the corresponding second mask information among the regions included in the volume data acquired in the first time phase. A pixel value obtained by converting a pixel value of each voxel based on the change amount is obtained as the feature amount, volume data obtained by converting the pixel value is defined as the one feature amount image data, and the second Pixel value conversion means for defining the volume data included in the 4D contrast image data and the volume data acquired in the corresponding second time phase as the other feature amount image data;
The medical image processing apparatus according to claim 4, further comprising: The medical treatment is cauterization treatment, the previous is before the cauterization treatment, and the subsequent is after the cauterization treatment,
The storage means stores first 4D contrast image data photographed before cauterization treatment, and second contrast image data photographed after cauterization treatment,
The curve creation means includes a first pixel value curve representing a temporal change of a pixel value in each voxel of the first 4D contrast image data, and a temporal change of a pixel value in each voxel of the second 4D contrast image data. And a second pixel value curve representing
The curve model creation means includes a first pixel value curve model that represents a temporal change in the pixel value of each part before the ablation treatment, and a second that represents a temporal change in the pixel value of the part after the ablation treatment. And a pixel value curve model of
The first classifying means classifies each voxel of the first 4D contrast image data for each part by comparing the first pixel value curve and the first pixel value curve model of each part. Then, the first mask information before the ablation treatment is created, the second pixel value curve is compared with the second pixel value curve model of each part, and each voxel of the second 4D contrast image data is compared. By classifying by region, create the first mask information after ablation treatment,
The second classifying means excludes the region classified as the ablation target region from the first mask information before the ablation treatment and the first mask information after the ablation treatment. Creating the second mask information before the ablation treatment and the second mask information after the ablation treatment from which the classified region is removed;
The feature amount image generation unit is characterized in that each voxel included in each region indicated by the second mask information before the ablation treatment is specified from a pixel value curve of each voxel of the first 4D contrast image data. By assigning an amount, feature image data before the ablation treatment is generated, and further, the second 4D contrast image is applied to each voxel included in each region indicated by the second mask information after the ablation treatment. By assigning a feature amount specified from the pixel value curve of each voxel of data, the feature amount image data after the ablation treatment is generated,
The image alignment means aligns the position of the feature amount image before the ablation treatment and the position of the feature amount image after the ablation treatment,
The display image generation unit generates first display image data based on volume data included in the first 4D contrast image data, and based on volume data included in the second 4D contrast image data, Generating second display image data aligned with the first display image data according to the alignment;
The display control means displays a first display image based on the first display image data and a second display image based on the second display image data side by side on the display means. The medical image processing apparatus according to claim 1. On the computer,
A plurality of 4D contrast image data including volume data obtained by imaging a subject into which a contrast medium is injected before and after medical treatment for the subject are received, and before and after the 4D contrast image data. A curve creation function for creating the previous pixel value curve and the subsequent pixel value curve, each representing a temporal change in pixel value in each voxel,
Curve model creating means for obtaining a pixel value curve model representing a temporal change of a pixel value in each part of the subject for each of the before and after,
By comparing the voxel pixel value curve and the pixel value curve model of each part for each of the previous and the subsequent, and classifying the voxel pixel value curve for each part of the subject, A first classification function for creating the previous first mask information and the subsequent first mask information indicating a region of a part;
By excluding a region other than the region where the shape of the pixel value curve model substantially matches between the previous and subsequent 4D contrast image data from the first and subsequent first mask information, A second classification function for creating the previous second mask information and the subsequent second mask information;
By assigning a feature amount specified from each of the previous and subsequent pixel value curves to each voxel included in each region indicated by the second mask information before and after the previous, A feature amount image generation function for generating feature amount image data and the subsequent feature amount image data;
An image alignment function for aligning the positions of the feature image before and after the image;
A display image generation function for generating, for each 4D contrast image data, display image data that is aligned according to the alignment based on volume data included in each of the previous and subsequent 4D contrast image data;
A display control function for displaying on the display device a display image based on the display image data for each of the 4D contrast image data;
A medical image processing program characterized in that

Images