JP6527614B2 - Image processing apparatus and operation method of image processing apparatus - Google Patents

Description

  The present invention relates to technology for processing and displaying three-dimensional medical image data.

  Medical image data in recent years are mainly three-dimensional volume data. When the doctor makes an image diagnosis, the tomograms are continuously switched and displayed (scroll display) to check whether there is an abnormal shadow or not. When the doctor finds an abnormal shadow by image diagnosis, the user can grasp the characteristics of the abnormal shadow from detailed observation while repeatedly scrolling and displaying display parameters in the vicinity of a tomographic image including the abnormal shadow. Then, the features of the abnormal shadow are input as image findings, and a tomographic image representative of the abnormal shadow (for example, the size of the abnormal shadow becomes maximum) is selected and put together in a report or the like.

  In image diagnosis, three-dimensionalization of image data has enabled detailed diagnosis. However, when displaying as a two-dimensional tomogram or putting it together in a report, it is necessary to select a tomogram from among three-dimensional image data, which increases the burden on the doctor. In this series of work, inventions have been proposed so far to reduce the burden on doctors. Patent Document 1 proposes a method of automatically adjusting display parameters based on image feature amounts in a region of interest. Further, as a general method, a method of obtaining a center of gravity of a region of interest in three-dimensional image data and displaying a tomographic image of a cross section passing through the center of gravity may be considered.

Japanese Patent Application Publication No. 2007-143982

  However, the prior art has the following problems. Patent Document 1 is an invention of adjusting display parameters in accordance with the conditions at the time of examination, at the time of a detailed examination, etc., and it is impossible to select a tomogram from three-dimensional image data. Further, even if a method of selecting a tomogram passing through the center of gravity of a region of interest in three-dimensional image data is used as a selection means of tomograms to be displayed, such tomograms in which the tomograms necessarily represent abnormal shadows It does not. Therefore, a doctor's task of selecting a tomogram occurs after all, and the burden on the doctor can not be reduced.

The present invention has been made in view of the above problems, with observations about the three-dimensional medical image, an inference means for inferring the diagnosis, of the findings previously used Symbol inference, inference of the diagnosis An image processing apparatus comprising: selection means for selecting a finding that has contributed to ; and identification means for identifying at least one tomogram representing the feature of the selected observation from the three-dimensional medical image. provide.

  According to the present invention, it is possible to automatically select and display a tomogram representing the feature of the imaging findings to be focused on. As a result, the burden on the doctor can be reduced.

FIG. 1 is a block diagram showing an example of the arrangement of an image processing apparatus according to the first embodiment; FIG. 1 is a diagram showing a basic configuration of a computer that implements each unit of an image processing apparatus by software. FIG. 2 is a diagram showing an outline of processing of the image processing apparatus in the first embodiment. 3 is a flowchart showing the processing procedure of the first embodiment of the image processing apparatus. 6 is a flowchart showing the processing procedure of the second embodiment of the image processing apparatus. The flowchart which shows the process sequence of 3rd Example of an image processing apparatus.

  Hereinafter, preferred embodiments of an image processing apparatus and a control method thereof according to the present invention will be described with reference to the attached drawings. However, the scope of the invention is not limited to the illustrated example.

First Embodiment
The configuration of the image processing apparatus 1 according to the present embodiment will be described according to FIG. In the image processing apparatus 1, the three-dimensional medical image acquisition unit 1001 acquires an image from the database 2. Next, the notable finding acquisition unit 1002 acquires an image finding focused as a result of interpretation of this image by the user. In the present embodiment, the imaging findings acquired by the notable observation acquisition unit 1002 are regarded as the attention findings. Next, the feature amount type identification unit 1003 identifies an image feature amount type to be calculated based on the acquired noted findings, and the feature amount calculation unit 1004 calculates the identified image feature amount. Then, the tomogram identification unit 1005 calculates a tomogram selection index in each tomogram, and determines a tomogram based on the tomogram selection index. Finally, the display control unit 1006 displays the identified tomogram.

  FIG. 2 is a view showing an example of the configuration when the image processing apparatus according to the first embodiment is realized by a computer. The image processing apparatus 1 includes a control unit 10, a monitor 104, a mouse 105, and a keyboard 106. The control unit 10 includes a central processing unit (CPU) 100, a main memory 101, a magnetic disk 102, and a display memory 103. Then, the CPU 100 executes a program stored in the main memory 101 to execute various controls such as communication with the database 2 and overall control of the image processing apparatus 1. The CPU 100 mainly controls the operation of each component of the image processing apparatus 1. The main memory 101 stores a control program executed by the CPU 100, and provides a work area when the CPU 100 executes a program. The magnetic disk 102 stores an operating system (OS), device drives of peripheral devices, various application software including a program for performing a diagnostic support process described later, and the like. The display memory 103 temporarily stores display data according to the display control unit 1006. The monitor 104 is, for example, a CRT monitor or a liquid crystal monitor, and displays an image based on data from the display memory 103. The mouse 105 and the keyboard 106 perform pointing input and input of characters and the like by the user (doctor). The above components are communicably connected to each other by a shared bus 107.

  In the present embodiment, the image processing apparatus 1 can read medical image data from the database 2 via a LAN (Local Area Network) 3. Here, an existing Picture Archiving and Communicating System (PACS) can be used as the database 2. The types of three-dimensional medical image data include an X-ray CT image, an MRI image, a PET image, and a SPECT image.

  In addition, the above-mentioned apparatus structure can be comprised using a general computer and its peripheral device. Further, the control procedure of the image processing apparatus according to the present invention described later with reference to FIG. 4 can be realized as a computer program executed on a computer.

  FIG. 3 shows an outline of the image processing of this embodiment. In FIG. 3, MI3 is a three-dimensional medical image to be diagnosed, which is input from a database. In the present embodiment, a chest X-ray CT image is taken as an example, and a processing apparatus used when making an image diagnosis of the presence or absence of an abnormal shadow or a feature will be described. IF 3 is an imaging findings input area, and imaging findings can be input from the pull-down menu. When the image finding is input, the type of the image feature amount associated with the image finding is identified in advance. And the said image feature-value is calculated about each tomogram, and a tomogram selection index is further calculated from there. Then, according to the tomogram selection index, the tomogram representing the features of the input image findings is selected. In FIG. 3, “Squamous process” is input as the image finding, and a tomogram selection index is calculated from the image feature amount previously associated with “Squamous process”. Then, based on the calculation result, the tomographic image T3 is selected.

  FIG. 4 is a flowchart of this embodiment. A specific processing procedure executed by the image processing apparatus 1 will be described along this flowchart.

<Step S410>
In step S410, the three-dimensional medical image acquisition unit 1001 acquires a three-dimensional medical image from the database 2. In the present embodiment, the acquired image may be used as it is in the subsequent processing, or the user may designate a region of interest (ROI, Region of Interest) and limit the region of the image. In the case of limiting the area, for example, the area may be designated from the three-dimensional medical image through a GUI (not shown), and the area may be replaced as a three-dimensional medical image to be handled in the subsequent processing.

<Step S420>
In step S420, the notable finding acquisition unit 1002 acquires, as a notable finding, an imaging finding input by the user via a GUI (not shown) for a three-dimensional medical image. In the present embodiment, the user diagnoses the acquired three-dimensional medical image, and inputs an image finding using an image finding input form as shown at IF 3 in FIG. 3. Here, the imaging findings that can be input are, for example, "spheroid", "spinoid process", "border boundary unclear", and the like. The user inputs at least one of these. In addition, the selection method of an imaging finding is not limited to this. For example, when past diagnosis results of the same patient are stored in the database, the imaging findings input in the diagnosis results may be used as the attention findings.

<Step S430>
In step S430, the feature amount type identification unit 1003 identifies the type of the image feature amount calculated based on the notable finding obtained by the notable finding acquisition unit 1002. In the present embodiment, the type of the image feature amount is associated in advance with each image finding, and the type of the image feature amount to be automatically calculated from the selected notable finding is determined. An example in which the image findings and the image feature amount type are associated is as follows.
"Spherical": Roundness C, aspect ratio A
"Squamous process": contour line length L, filling factor F
"Boundary unclear": Sum of boundary edge components E

<Step S440>
In step S440, the feature amount calculation unit 1004 calculates an image feature amount whose type is identified in step S430 with respect to each tomogram of the three-dimensional medical image based on the noted findings obtained by the noted finding acquisition unit 1002. Do.
Here, the degree of circularity C, the aspect ratio A, the outline length L, and the filling rate F are calculated from a binary image generated from a three-dimensional medical image. With regard to the method of generating a binary image, in this embodiment, a threshold value is determined by the discriminant analysis method from the histogram of a three-dimensional medical image, and a binarization process is performed. Using the generated binary image, the degree of circularity C, the aspect ratio A, and the filling factor F are calculated by the following equations.

  Here, Area is the area of each area in the binary image. Further, Fereth is a horizontal Feret diameter, and Feretv is a vertical Feret diameter, both of which are calculated from circumscribed rectangles of the respective regions in the binary image. When a plurality of regions exist in the binary image, in the present embodiment, the region with the largest area is taken as a target region, and the image feature amount is calculated. The outline length L is the length of the outline of the target area. The sum E of edge components is the sum of edge components on the outline of the target area. Edge components are components obtained by applying a Sobel filter to a three-dimensional medical image.

  By these calculation methods, the image feature amount associated with the notable observation is calculated in each tomogram. In the present embodiment, image feature quantities are calculated for tomographic image groups (for example, tomographic image groups generated at intervals of 1 mm) in respective axial, coronal, sagital cutting directions. Note that how to select a tomographic image for calculating an image feature amount is not limited to this. For example, the cutting direction of tomograms is not limited to axial, coronal, sagital, and a tomogram group may be created with free cutting directions. For example, an image feature is generated from a tomogram group in a cutting direction which is formed by setting an x-axis, a y-axis, and a z-axis passing through the center of the image and rotating 180 degrees by 10 ° about one or two axes. The amount may be calculated.

<Step S450>
In step S450, based on the image feature amount calculated by the feature amount calculation unit 1004, the tomogram identification unit 1005 determines a tomogram representing the feature of the notable observation. In the present embodiment, a value indicating “the degree to which the tomogram represents the notable finding” is calculated (digitized) from the image feature amount calculated in step S440, and this is “tomographic image selection index” based on the notable finding Do. As an example, an example of a mathematical expression for calculating a tomogram selection index corresponding to the three image findings is shown below.

ここで，wは予め定めた重みである。本実施例ではw1〜w5 = 1.0としている。

Here, w is a predetermined weight. In the present embodiment, w1 to w5 = 1.0.

  In addition, you may incorporate other selection criteria other than "the degree which represents an attention finding" in the index calculation formula. For example, criteria other than the notable findings may be taken into consideration by adding to the above equation the terms representing the size of the region in the binary image and the proximity of the region of interest to the center of gravity.

  A tomogram selection index corresponding to a notable observation is calculated for each tomogram using these formulas. Then, the tomogram for which the tomogram selection index is maximum is determined as the tomogram to be displayed. In the case where a plurality of findings are selected in step S420, in the present embodiment, a tomographic image selection index is calculated for all selected target findings, and a tomographic image for displaying a tomographic image having the largest total value of the indices is displayed. Determined as an image. And save this as data for the report.

<Step S460>
In step S460, the display control unit 1006 displays the tomogram specified by the tomogram specifying unit 1005.

  Thus, by selecting a tomogram to be displayed based on the image feature amount associated with the image finding, it is possible to automatically select a tomogram representing the feature of the input image finding.

<Modification 1>
In the above embodiment, in the case where a plurality of notable findings are present in step S450, each tomogram selection index is calculated, and a tomogram is determined from the total value thereof. However, tomograms that maximize each tomogram selection index may be respectively selected and displayed in association with each notable observation. Also, in this case, the user may be able to select a tomographic image to be used for the report.

Second Embodiment
In this embodiment, an example will be described in which an image feature amount is calculated from a three-dimensional medical image and an attention finding is automatically selected without using a user's instruction in inputting an image finding. Based on the image feature quantity calculated from the acquired image, it is possible to identify an image finding which will be included in the image and automatically select it as a notable finding.

  The configuration of the apparatus is the same as that of the first embodiment, so the description will be omitted. However, this embodiment differs from the first embodiment in that the focused finding acquisition unit 1002 automatically selects a focused finding from a three-dimensional medical image. Further, when the image processing apparatus 1 operates based on a command of software, the program stored in the ROM or the HDD executes the processing of FIG. 5 and is a program for realizing the function therefor. It is different.

  Details of steps S520 and S530 according to the second embodiment will be described with reference to FIG. The processes in steps S510 and S540 to S570 are the same as the processes in steps S410 and S430 to S460 in the flowchart of FIG. 4 of the first embodiment, and thus the description thereof is omitted.

<Step S520>
In step S520, the notable finding acquisition unit 1002 calculates a feature amount for focused observation selection from the three-dimensional medical image acquired by the three-dimensional image acquisition unit 1001.

  In the present embodiment, first, tomogram selection indexes of all types are calculated in each tomogram using the equations (4) to (6) in step S450. Next, the tomogram selection indices calculated from the respective tomograms are summed up by the number of tomograms, and are used as feature amounts for selecting a notable finding. That is, in this embodiment, in each three-dimensional medical image, all kinds of prepared indexes such as spheroid index, leaf shape index, irregular index, spinous process index, boundary irregular index etc are calculated, and each index A value obtained by totaling the number of images is calculated as a feature amount. Note that the method of calculating the feature amount for selecting the notable finding is not limited to this. For example, the maximum value of the index calculated in each tomogram may be used as the feature amount for selecting the notable observation.

<Step S530>
In step S530, the notable finding acquisition unit 1002 identifies an image finding that will be included in the image based on the feature amount for the notable finding selection calculated in step S520, and selects this as a notable finding.

  In the present embodiment, the image finding of the three-dimensional medical image is selected by performing threshold processing on the feature amount for the notable finding selection calculated in step S520. A threshold value is set in advance for each feature amount, and an image finding corresponding to the feature amount equal to or more than the threshold is selected as an attention finding. When there are a plurality of feature amounts equal to or greater than the threshold value, all of them are treated as attention findings and passed to the next step. However, when a plurality of image findings belonging to the same category (for example, image findings belonging to the category of “shape” are “aspheric,” “lobular,” “irregular”) are selected, the feature amount is largest among them Choose one and make it a notable finding. The method of selecting the imaging findings is not limited to this. For example, an image finding may be selected from feature amounts by constructing a classifier using Support Vector Machine, AdaBoost, or the like by learning using a combination of feature amounts and image findings obtained with respect to past diagnostic results. .

  As described above, the imaging findings input operation by the doctor can be automated by selecting the attention finding from the feature amount. That is, all the processes from the input of the imaging findings to the display of the tomogram representing the features of the imaging findings become automatic.

Third Embodiment
In the present embodiment, after obtaining a finding of interest using the method of the first embodiment or the second embodiment, a differential diagnosis is performed using an inference device, and a fault representing the feature of the finding of interest contributing to the diagnosis Display the image. In the present embodiment, it is assumed that the doctor inputs image findings other than the notable findings.

  The configuration of the apparatus is the same as that of the first embodiment, so the description will be omitted. However, this embodiment differs from the first embodiment in that the focused finding acquisition unit 1002 automatically selects a focused finding from a large number of image findings. Further, when the image processing apparatus 1 operates based on a command of software, the program stored in the ROM or the HDD executes the processing of FIG. 6 and is a program for realizing the function therefor. It is different.

  Details of steps S630 and S640 according to the third embodiment will be described with reference to FIG. The processes of steps S610 and S620 and steps S650 to S680 are the same as the processes of steps S410 and S420 and steps S430 to S460 in the flowchart of FIG. 4 of the first embodiment, and thus the description thereof is omitted. .

<Step S630>
In step S630, the notable finding acquisition unit 1002 inputs all the acquired image findings to the inference unit. As the inference means, existing inference methods such as Bayesian networks and neural networks can be used. In this embodiment, a Bayesian network is used as an inference means. The Bayesian network is an inference model using conditional probability, and when applied to the present embodiment, it is possible to obtain an inference probability of each diagnosis using image findings as an input. The diagnostic names used in this example are "primary lung cancer", "lung metastasis of cancer", and "others".

In this embodiment, assuming that each diagnosis name is Di and each finding is Sj, only the prior probability P (Di) (probability of each diagnosis name when there is no input) and Sj are input. It calculates using the difference with the probability P (Di | Sj) of a case. For example, the degree of contribution C (Di | Sj) of the finding Sj to the diagnostic name Di is expressed by the following equation.

  The degree of contribution in each diagnosis is calculated for all the imaging findings prepared as options. In the present embodiment, the degree of contribution of each image finding in the diagnosis with the highest probability (estimated diagnosis) in the inference result by the Bayesian network is sent to the next step.

<Step S640>
In step S640, the notable finding acquisition unit 1002 selects a notable finding in accordance with the degree of contribution calculated in step S630. In the present embodiment, a notable observation is selected by performing threshold processing on the degree of contribution calculated in step S630. A threshold is set in advance for the degree of contribution, and a notable finding for which the degree of contribution equal to or greater than the threshold is obtained is selected. If there are multiple notable findings whose contribution degree is equal to or higher than the threshold, all of them are selected. If there is no notable finding whose contribution degree is equal to or more than the threshold value, the attention finding with the largest contribution degree is selected. In addition to the selection method, a notable finding having the largest contribution may be selected without using a threshold.

  As described above, by selecting the notable observation on the basis of the degree of contribution to the diagnostic name obtained from the inference unit, it is possible to display a tomogram representing the feature of the image findings supporting the inference result.

<Modification 2>
In the above embodiment, in step S630, the degree of contribution is calculated for the diagnosis with the highest probability in the inference result by the Bayesian network, and is used as the target of the notable finding selection. However, attention findings may be selected from the degree of contribution of the user (doctor) in the input diagnostic name. Specifically, when the diagnosis input by the doctor as an impression is different from the inferred diagnosis by the Bayesian network, the attention findings may be selected based on the degree of contribution to the diagnosis selected by the doctor. According to this, it is possible to obtain an effect that a tomogram representing an imaging finding that explains the basis of a doctor's diagnosis is selected.

<Other Embodiments>
The present invention is also realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiment is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU or the like) of the system or apparatus reads the program. It is a process to execute.

1 Image processing apparatus 2 Database 3 LAN (Local Area Network)
1001 Three-dimensional medical image acquisition unit 1002 notable observation acquisition unit 1003 feature amount type identification unit 1004 feature amount calculation unit 1005 tomographic image identification unit 1006 display control unit

Claims (11)

Using the findings concerning the three-dimensional medical image, an inference means for inferring the diagnosis,
Of the findings previously used Symbol reasoning, selection means for selecting the findings contributed to the inference of the diagnosis,
Specifying means for specifying at least one tomogram representing the feature of the selected finding from the three-dimensional medical image ;
An image processing apparatus comprising: The image according to claim 1, wherein the specifying unit specifies the at least one tomogram from the three-dimensional medical image based on a feature amount that is a value representing a feature of the selected finding. Processing unit. Calculation means for calculating a contribution degree indicating the degree of contribution to the inference of the diagnosis name for each finding used for the inference
And have
It said selection means, using said contribution, the image processing apparatus according to claim 1 or 2, characterized in that selecting the findings contributed to the inference of the diagnosis. The image processing apparatus according to claim 3 , wherein the selection unit selects a finding whose contribution degree is equal to or more than a threshold as a finding contributing to the inference of the diagnosis name . The image processing apparatus according to claim 3 , wherein the selection unit selects a finding with the highest degree of contribution as a finding that has contributed to inference of the diagnosis name . The calculation means is characterized in that the degree of contribution is calculated using a difference between an inference probability of the diagnosis when using the finding and an inference probability of the diagnosis when not using the finding. The image processing apparatus according to any one of claims 3 to 5 . Acceptance means for accepting input of diagnostic name from user
And have
It said selection means includes a diagnosis accepted the input, if the inference diagnostic names are different, according to claim 1, characterized in that selecting the findings contributed to infer the diagnosis accepted the input The image processing apparatus according to any one of items 1 to 6 . The image processing apparatus according to any one of claims 1 to 7 , further comprising display control means for causing the display unit to display the identified tomographic image . The image processing apparatus according to any one of claims 1 to 8, wherein a cutting direction of the tomogram is any of axial, coronal, and sagittal cutting directions, or a free cutting direction. Inferring a diagnostic name using findings on a three- dimensional medical image ;
Of the findings previously used Symbol reasoning, and selecting the findings contributed to the inference of the diagnosis,
At least one tomographic image indicating a feature of the selected findings, and as engineering that identifies from the three-dimensional medical image,
A method of operating an image processing apparatus, comprising: A program for causing a computer to execute the method according to claim 10 .

Images