JP2020074967A - Image retrieval program, image retrieval device and image retrieval method - Google Patents

Abstract

To provide an image retrieval program, image retrieval device and image retrieval method which can correctly distinguish between a bronchus and a honeycomb lung, and can improve the accuracy of the case retrieval.SOLUTION: An image retrieval program causes a computer of an image retrieval device for retrieving a second image similar to a first image from the first image to execute: processing of extracting an image of a candidate region from the first image being a cross section image of a lung field region of a subject; processing of generating a binarized image obtained by binarizing the pixel value of the extracted image of the candidate region; processing of extracting the center line of the image of the candidate region from the binarized image; processing of dividing the center line at a branch point where the center line branches; processing of dividing the image of the candidate region into the region of the honeycomb lung and the region of the bronchus on the basis of the angle of the divided center line; and processing of retrieving the second image from the first image on the basis of the image of the lung field region other than the divided bronchus or the image of the region of the divided bronchus.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image search program, an image search device, and an image search method.

  2. Description of the Related Art Conventionally, in the medical field, there is an image search apparatus that searches a similar case in the past and refers to the search result to support a doctor's diagnosis. For example, an image representing a past case is stored in a database, an image search apparatus searches the database for images having similar image characteristics, and the searched images are displayed in order. In the image search device, a case is linked to the searched image. Therefore, the image retrieval device can retrieve a case similar to the case of the patient from the database, for example. In such an image search device, for example, a CT (Computed Tomography) image is used.

  One of the abnormal shadows taken on CT images is the cell lung. Honeycomb lung refers to, for example, the formation of irregularly expanded air spaces found in fibrosis (or hardening) in which the alveolar structure of the lung has been destroyed. Honeycomb lung is, for example, a lung in a state where fibrosis due to inflammation or the like has advanced.

  In addition, the bronchi are connected to the lungs, and the honeycomb lung is also connected to the bronchi. In this case, the bronchus may expand due to contraction due to fibrosis of the lung due to the cell lung. A bronchus that has become deformed due to fibrosis of the lung may be referred to as tractive bronchiectasis, for example.

  Although the bronchus that have become tractive bronchiectasis are not shadows, the image retrieval apparatus may mistakenly identify them as honeycomb lungs. That is, it is difficult for the image retrieval apparatus to distinguish between the beehive lung and the bronchus that are connected to the beehive lung and have become tractive bronchiectasis, and the bronchus may be mistakenly recognized as the beehive lung. Not only a bronchus that has become a tractive bronchiole dilation, but even a normal bronchus has individual differences. Therefore, the image retrieval apparatus may identify such a bronchus as a honeycomb.

  As a cause thereof, for example, as described above, it is considered that the honeycomb lung and the bronchus are physically connected. Since both the bronchus and the honeycomb are hollow structures, if the pixel value corresponding to the CT value (-1000) of air is used as a threshold value and the region is extracted, regions having similar concentrations are connected and extracted.

  In the image search apparatus, the accuracy of the case search may be reduced due to an identification error in distinguishing such a bronchus from a beehive lung.

  On the other hand, for example, a CT image of the lung is divided into two regions, a central region and a peripheral region, and the position and number of abnormal shadows existing in each region are represented by a histogram, and the histograms of past cases similarly represented are collated. Then, there is also an image search device that searches for similar cases. The histogram is, for example, a bar graph. By displaying the position and number of abnormal shadows as a bar graph, for example, it becomes possible to quantitatively compare the distribution of abnormal shadows between cases. With such an image search device, for example, a similar case can be searched with high accuracy, and the diagnosis time can be shortened.

  Examples of techniques related to image retrieval include the following. That is, there is an image processing apparatus that extracts a region related to a target region from a plurality of cross-sectional images, detects a peripheral point of a bronchial tree such as a bronchus from the extracted regions, and extracts a ring structure of the bronchial tree from the peripheral point.

  According to this technique, a tree-shaped tubular structure with a high branching level can be obtained.

  In addition, based on the feature amount acquired from the lung marginal area extracted from the chest tomographic image, one or more lesion areas including the normal part and the honeycomb lung in the lung marginal area are identified and classified, and the lesion identification result is obtained. There is a diagnosis support device that outputs a classification result.

  According to this technology, it has excellent diagnostic performance for lung field lesions such as interstitial pneumonia, and IPF (idiopathic pulmonary fibrosis) and NSIP (nonspecific interstitial pneumonia), which have been extremely difficult in the past, have been diagnosed. Specific interstitial pneumonia) can be distinguished.

JP, 2014-223311, A International Publication No. 2017/150497

  However, as described above, it may be difficult to distinguish between the bronchus and the beehive lung in the image search device because the bronchus and the beehive lung are physically connected. Even in the image search apparatus using the histogram, the bronchus may be distinguished from the honeycomb lung to extract a larger area of the honeycomb lung in the central direction than in the actual case. In this case, the image search device will search for a case from the database using an incorrect histogram, and the case to be searched originally may not be searched, and the accuracy of the case search may decrease.

  Further, in the above-described image processing apparatus that detects the peripheral points of the bronchial tree and extracts the annular structure of the bronchial tree from the peripheral points, there is no discussion about the discrimination between the bronchi and the lungs of the honeycomb. In addition, the above-described diagnosis support device that identifies one or more lesions including a normal part and a honeycomb lung in the lung peripheral region based on the feature amount acquired from the lung peripheral region also identifies the honeycomb lung. However, there is no discussion of how to distinguish between the bronchi and the honeycomb.

  Therefore, one disclosure is to provide an image search program, an image search apparatus, and an image search method capable of accurately dividing the bronchus and the honeycomb lung.

  Further, one disclosure is to provide an image search program, an image search device, and an image search method capable of improving the accuracy of case search.

  One disclosure is an image search program executed by a computer of an image search device that searches a first image for a second image similar to the first image, and is a cross-sectional image of a lung field region of a subject. An image of the candidate area is extracted from the first image, a binarized image is generated by binarizing the pixel values of the extracted image of the candidate area, and the center of the image of the candidate area is generated from the binarized image. Extract a line, divide the centerline at a branch point where the centerline is branched, and based on the angle of the divided centerline, divide the image of the candidate region into a honeycomb lung region and a bronchial region. The program causes the computer to execute a process of searching the first image for the second image based on the divided image of the lung field other than the bronchus or the divided image of the bronchus.

  According to one disclosure, it is possible to accurately divide the regions of the honeycomb and bronchus. Further, according to one disclosure, it is possible to improve the accuracy of case search.

FIG. 1 is a diagram showing a configuration example of an image search system. 2A is a diagram showing a configuration example of a dictionary registration module, and FIG. 2B is a diagram showing a configuration example of a search module. 3A is a diagram showing an example of a CT image, and FIG. 3B is a diagram showing an example of a histogram. FIG. 4 is a diagram illustrating a configuration example of the bronchial region dividing unit. 5 (A) and 5 (B) are diagrams showing an example of a CT image in which the bronchus and the honeycomb are shown. FIG. 6A and FIG. 6B are schematic diagrams showing an example of distribution of bronchi and honeycomb lungs. 7A to 7D are diagrams showing examples of images and the like generated by each process. FIG. 8A to FIG. 8D are diagrams showing an example of each processing. FIG. 9 is a flowchart showing an example of the feature extraction processing. FIG. 10 is a flowchart showing an example of bronchial region extraction processing. FIG. 11 is a flowchart showing an example of the centerline extraction processing. 12A to 12C are diagrams showing an example of distance conversion. 13A and 13B are diagrams showing an example of distance conversion. 14A and 14B are diagrams showing an example of distance conversion. 15A and 15B are diagrams showing an example of distance conversion. 16A and 16B are diagrams showing an example of distance conversion. 17A and 17B are diagrams showing an example of distance conversion in the case of three dimensions. 18A and 18B are diagrams showing an example of distance conversion in the case of three dimensions. 19A to 19C are diagrams showing an example of the center line extraction processing. 20A and 20B are diagrams showing an example of centerline extraction processing in the case of three dimensions. 21 (A) and 21 (B) are diagrams showing an example of labeling. FIG. 22 is a diagram showing an example of area division. FIG. 23 is a flowchart showing an example of search processing. FIG. 24 is a diagram showing an example of a honeycomb lung region and a bronchial region. FIG. 25 is a diagram illustrating a hardware configuration example of the image search device. FIG. 26 is a diagram showing a configuration example of the image search system. FIG. 27 is a diagram illustrating a hardware configuration example of the image search device.

  Hereinafter, modes for carrying out the present invention will be described. The following embodiments do not limit the disclosed technology. Then, the respective embodiments can be appropriately combined within a range in which the processing content is not inconsistent.

[First Embodiment]
<Example of image search system configuration>
FIG. 1 is a diagram showing a configuration example of an image search system 10.

  The image search system 10 includes an image search device 100, an image database (hereinafter sometimes referred to as “image DB” (DB: Data Base)) 200, and a CT image capturing device 300.

  The image DB 200 stores a cross-sectional image of the lung field region of the patient, which has been photographed in the past, together with cases and the like. The CT image capturing apparatus 300 captures a cross-sectional image of a patient's lung field region and outputs the captured cross-sectional image to the image retrieval apparatus 100. Both the image DB 200 and the CT image capturing apparatus 300 use CT images as cross-sectional images.

  The image search device 100 searches the CT images captured by the CT image capturing device 300 for CT images similar to CT images representing past cases registered in the image DB 200. Then, the image search device 100 displays the searched CT image on the UI (User Interface) 140. The doctor diagnoses the patient based on the displayed CT image. The image search device 100 is also a similar case search device, for example.

  Hereinafter, the CT image read from the image DB 200 may be referred to as a “CT image”, and the CT image output from the CT image capturing apparatus 300 may be referred to as a “query image”. Further, in the first embodiment, a CT image will be described as an example of an image, but a cross-sectional image may be, for example, an MRI (Magnetic Resonance Imaging) image.

  The image search device 100 includes a dictionary registration module (or dictionary registration unit) 110, a feature amount dictionary database (or feature amount dictionary DB (Data Base)) 120, a search module (or search unit) 130, and a UI 140.

  The "module" represents, for example, a subprogram that realizes a certain function in the program. However, in the following, a processing block that realizes such a function may be represented as a “module”. Therefore, the “dictionary registration module” 110 is, for example, a processing block having a function of registering the feature amount of the CT image in the feature amount dictionary DB 120, and the “search module” 130 is, for example, a CT image similar to the query image. It is a processing block having a function of searching an image.

  The dictionary registration module 110 reads out a CT image representing a past case from the image DB 200, and extracts a feature amount of the CT image based on the CT value (or image data) of the read CT image. At that time, the dictionary registration module 110 extracts a candidate region from the CT image, divides the bronchus region and the honeycomb lung region from the extracted candidate region, and based on the divided bronchus region or honeycomb lung region, The shadow is identified and the feature amount is extracted. The dictionary registration module 110 outputs the feature amount to the feature amount dictionary DB 120. Details of the dictionary registration module 110 will be described later.

  The feature amount dictionary DB 120 is, for example, a memory or a storage device, and stores the feature amount.

  The search module 130 performs a feature amount calculation process based on the CT value (or image data) of the query image output from the CT image capturing apparatus 300, and calculates the feature amount of the query image. At that time, the search module 130 extracts a candidate region from the CT image, divides the bronchus region and the honeycomb lung region from the extracted candidate region, and determines the shadow based on the divided bronchus region or the honeycomb lung region. Identification is performed and feature quantities are extracted. The search module 130 reads the feature amount of the CT image from the feature amount dictionary DB 120, matches the read feature amount with the feature amount of the query image, and searches for a CT image having a feature amount similar to the query image. The search module 130 outputs the search result to the UI 140. Details of the search module 130 will be described later.

  The UI 140 is, for example, a monitor 141, a keyboard 142, or the like, and is an interface for exchanging information between a user (for example, a doctor) and the image search device 100. The UI 140 displays the search result via, for example, the monitor 141. In this case, the UI 140 may read out the CT image corresponding to the search result from the image DB 200 and display it. In addition, the UI 140 can output a search key to the search module 130, for example, to browse a CT image captured by the CT image capturing apparatus 300 and to select a query image that is a search processing target. .. The UI 140 can also display, for example, the case associated with the searched CT image as the search result.

  The image DB 200 is, for example, a memory or a storage device, and stores CT images representing past cases. The image DB 200 may also store the file name of each CT image, the ID (Identification) of each CT image, the corresponding case, and the like. The dictionary registration module 110 may read the CT image and these pieces of information from the image DB 200, and store these pieces of information of the corresponding CT image in the characteristic quantity dictionary DB 120 together with the characteristic quantity.

  The CT image capturing apparatus 300 rotates the irradiator to irradiate the human body with radiation such as X-rays, and detects the radiation radiated by the detector to detect the CT value (or pixel value. Then, a CT image having a CT value and a pixel value may be used without distinction). The CT image capturing device 300 outputs the captured CT image as a query image to the image search device 100.

<Each configuration example of dictionary registration module and search module>
FIG. 2A is a diagram showing a configuration example of the dictionary registration module 110.

  The dictionary registration module 110 includes a feature quantity extraction unit 111. The feature amount extraction unit 111 extracts the feature amount from the CT image output from the image DB 200 and stores the extracted feature amount in the feature amount dictionary DB 120. The feature amount is, for example, the number of lattice regions determined to be an abnormal shadow in each lattice region obtained by dividing the CT image in a lattice shape. The feature amount extraction unit 111 may store a histogram in the feature amount dictionary DB 120 as a feature amount. In the following, the feature amount and the histogram may be used without distinction.

  3A is a diagram showing an example of a CT image (or a query image), and FIG. 3B is a diagram showing an example of a histogram.

  As shown in FIG. 3A, the CT image is, for example, a cross-sectional image (or tomographic image) obtained by cross-sectioning (or slicing) the lung field region of the subject along the axial plane with the trunk as the center. The dictionary registration module 110 divides each CT image into grid-shaped lattice areas, and identifies whether each of the lattice areas is an abnormal shadow or a normal shadow.

  Further, the dictionary registration module 110 identifies each CT image as a central part based on a three-dimensional model of a region related to the center of the lung (for example, the central part of the body) and the periphery (for example, a part other than the central part of the body). The number of lattice regions identified as abnormal shadows is counted for each central portion and each peripheral portion.

  In the example of FIG. 3B, the dictionary registration module 110 divides into a central portion and a peripheral portion of the left lung and a central portion and a peripheral portion of the right lung based on such a three-dimensional model, and divides each portion. 4 shows an example of a histogram generated by counting the number of lattice areas identified as abnormal shadows. The dictionary registration module 110 creates one histogram shown in FIG. 3B for a plurality of CT images of one person, for example.

  In the histogram shown in FIG. 3B, the upper side is the “head” direction of the subject and the lower side is the “foot” direction in the drawing. Further, in FIG. 3B, the horizontal direction represents, for example, the number of lattice areas identified as abnormal shadows at that position.

  Returning to FIG. 2A, the feature amount extraction unit 111 may be referred to as a candidate region extraction unit 112, a bronchial region division unit 113, and a shadow identification and distribution feature extraction unit (hereinafter referred to as “distribution feature extraction unit”). ) 114.

  The candidate area extraction unit 112 extracts candidate areas from the CT image. For example, the candidate area extraction unit 112 extracts an area in which the CT value of each pixel of the CT image is “−1000” or less (CT value ≦ −1000) as a candidate area (or a candidate area image). A CT value of "-1000" or less indicates that the pixel is air, for example. FIG. 7A is a diagram showing an example of candidate regions for a CT image, for example.

  Returning to FIG. 2A, the bronchial region dividing unit 113 divides the image of the candidate region in the CT image into, for example, a bronchial region and a lung field region other than the bronchus, and an image of the lung field region other than the bronchus. Is output to the distribution feature extraction unit 114. In this case, the bronchial region dividing unit 113 may output the divided image of the bronchial region to the distribution feature extracting unit 114. Details of the bronchial region dividing unit 113 will be described later.

  FIG. 2B is a diagram showing a configuration example of the search module 130.

  The search module 130 includes a feature amount extraction unit 131 and a matching processing unit 135. The feature amount extraction unit 131 includes a candidate region extraction unit 132, a bronchial region division unit 133, and a distribution feature extraction unit 134. The feature amount extraction unit 131 is, for example, the same as the feature amount extraction unit 111 of the dictionary registration module 110.

  However, in the feature amount extraction unit 131, the candidate region extraction unit 132 extracts candidate regions from the query image acquired from the CT image capturing apparatus 300, and the bronchus region division unit 133 selects bronchus and other lung field regions. Then, the distribution feature extraction unit 134 divides the feature amount and extracts the feature amount from the lung field region other than the bronchus. The bronchial area dividing unit 133 may output an image of the divided bronchial area, similar to the bronchial area dividing unit 113, for example. The feature quantity extraction unit 131 can also create the histogram shown in FIG. 3B for the query image, for example.

  Returning to FIG. 2B, the matching processing unit 135 matches the feature amount output from the feature amount extraction unit 131 with the feature amount read from the feature amount dictionary DB 120, and a feature similar to the feature amount of the query image. Retrieve CT images with quantity. The matching processing unit 135 outputs, for example, a CT image having such similar characteristics to the UI 140 as the search result. In this case, the matching processing unit 135 may also read the case information associated with the searched CT image from the feature amount dictionary DB 120 and output the case information together with the CT image to the UI 140 as the search result.

<Structural example of bronchial region dividing section>
FIG. 4 is a diagram showing a configuration example of the bronchial region dividing unit 113. The bronchial region dividing unit 133 of the search module 130 also has the same configuration as the configuration example shown in FIG. 4, for example. As a representative, the bronchial region dividing unit 113 of the dictionary registration module 110 will be described.

  The bronchial region dividing unit 113 according to the first embodiment focuses on the fact that the distribution properties of the bronchus and the honeycomb lung in the lung are different, and the respective regions of the bronchus and the honeycomb lung are divided by utilizing this property.

  FIG. 5A shows an example of a distribution of bronchi, and FIG. 5B shows an example of a CT image showing an example of distribution of honeycombs. Further, FIGS. 6A and 6B are schematic diagrams showing the difference in distribution between the bronchus and the honeycomb lung.

  As shown in FIG. 5 (A), the bronchi are distributed from the center toward the periphery or from the periphery toward the center. On the other hand, as shown in FIG. 5B, the honeycomb lungs are distributed along the periphery (or the contour of the lung). As shown in FIG. 6 (B), in the bronchial region dividing unit 113 according to the first embodiment, focusing on such a difference in the distribution of the bronchi and the honeycomb lung, not only from the central direction but also from the peripheral direction. Also, by tracing that area, each area of the bronchus and the honeycomb is divided.

  Returning to FIG. 4, the bronchial region dividing unit 113 includes a binarized image generating unit 1130, a centerline extracting unit 1131, a centerline dividing unit 1134, an angle calculating unit 1135, a labeling unit 1136, and a region dividing unit 1137.

  The binarized image generation unit 1130 binarizes the image of the candidate area using a predetermined threshold value. For example, the binarized image generation unit 1130 sets the pixel value of the pixel to “255” when the CT value of each pixel of the image of the candidate region is equal to or larger than the threshold value, and sets the pixel value of the pixel to the pixel value when the CT value is smaller than the threshold value. Set the value to "0". The binarized image generation unit 1130 binarizes the CT value of the image of the candidate region to generate a binarized image including the background image and the foreground image.

  FIG. 7B is a diagram showing an example of a binarized image. For example, the binarized image generation unit 1130 sets the area of the pixel whose pixel value is set to “255” to the foreground area and the area of the pixel whose pixel value is set to “0” to the background area, thereby setting the background image. And a binary image of the foreground image is generated. The binarized image generation unit 1130 outputs the binarized image and the like to the centerline extraction unit 1131.

  Returning to FIG. 4, the centerline extraction unit 1131 includes a distance conversion unit 1132 and a centerline extraction processing unit 1133.

  The distance conversion unit 1132 converts each pixel value of the foreground image into a distance value representing a distance from the foreground image in the binarized image, thereby converting the binarized image into a distance converted image. In the distance conversion image, for example, each pixel value of the foreground image is converted into a distance value corresponding to the shortest distance (or the shortest number of pixels) from the boundary surface with the background image.

  FIG. 7C is a diagram showing an example of the distance conversion image. As shown in FIG. 7C, the vicinity of the center of the foreground image has the largest distance value because the distance from the background image is the longest in the foreground image. Details of distance conversion in the distance conversion unit 1132 will be described in an operation example. The distance conversion unit 1132 outputs the distance conversion image and the like to the centerline extraction processing unit 1133.

  Returning to FIG. 4, the centerline extraction processing unit 1133 extracts the centerline based on the distance value of the distance conversion image.

  FIG. 7D is a diagram showing an example of the center line extracted by the center line extraction processing unit 1133. Details of the centerline extraction processing will be described with an operation example. The centerline extraction processing unit 1133 outputs information about the extracted centerline to the centerline dividing unit 1134.

  Returning to FIG. 4, the centerline dividing unit 1134 divides the centerline into branches at branch points where the centerline is branched. FIG. 8A is a diagram showing an example of each branch. In the example of FIG. 8A, the center line is divided into the branch a1b1, the branch b1c, and the branch b1d1 around the branch point b1. Details of the center line division processing will be described in an operation example. The center line dividing unit 1134 outputs information about each divided branch to the angle calculating unit 1135.

  Returning to FIG. 4, the angle calculation unit 1135 approximates each branched branch with a straight line, and calculates the angle of each branch (or each straight line) using the approximated straight line. For example, the angle calculation unit 1135 approximates a straight line from each point of the center line by using the least squares method or the like, and calculates the angles of the two straight lines by using the approximated straight line. Details of the linear approximation and the angle calculation will be described in an operation example. FIG. 8B is a diagram showing an example of angle calculation. FIG. 8B shows an example in which the angle calculation unit 1135 calculates the angle between the branch a1b1 and the branch b1d1. The angle calculation unit 1135 outputs information regarding the calculated angle and information regarding each branch to the labeling unit 1136.

  Returning to FIG. 4, the labeling unit 1136 performs bronchial or honeycomb lung labeling on each branch based on the angle calculated by the angle calculation unit 1135.

  FIG. 8C is a diagram showing an example of labeling. In the example of FIG. 8C, the labeling unit 1136 labels the branch a1b1 and the branch b1c with the honeycomb lung, and the branch b1d1 with the bronchus. The labeling unit 1136 outputs information about each center line, information about each center line, and the like to the region dividing unit 1137.

  Returning to FIG. 4, the region dividing unit 1137 divides the candidate region into a bronchial region and a honeycomb lung region, for example, based on the information labeled on each branch. FIG. 8D is a diagram illustrating an example of area division. Details will be described in an operation example. The region dividing unit 1137 outputs, for example, an image of the lung field region obtained by removing the divided bronchus region from the image of the candidate region.

<Operation example>
Next, an operation example will be described. Regarding the operation example, first, the feature amount extraction processing (or the feature extraction processing performed by the feature amount extraction unit 111 of the dictionary registration module 110 or the feature amount extraction unit 131 of the search module 130. May be referred to). Next, the collation processing performed by the collation processing unit 135 of the search module 130 will be described.

<1. Feature extraction process>
FIG. 9 is a flowchart showing an example of the feature extraction processing in the image search device 100.

  The image search device 100 calculates the feature amount of the image based on the image data by the feature extraction processing. For example, the dictionary registration module 110 performs feature extraction processing on the CT image read from the image DB 200 to extract a feature amount. Further, for example, the search module 130 performs feature extraction processing on the query image output from the CT image capturing apparatus 300 to extract a feature amount. Here, FIG. 9 will be described by taking a case where the dictionary registration module 110 is used as an example.

  When the feature extraction unit 111 of the dictionary registration module 110 starts the feature extraction process (S10), it reads the CT image and extracts a candidate region from the CT image (S11). For example, the candidate area extraction unit 112 compares the CT value of each pixel of the CT image with a threshold value (for example, −1000) and extracts an area of the CT image having a CT value smaller than the threshold value.

  Next, the feature amount extraction unit 111 performs a bronchial region extraction process (S12). For example, the bronchial region dividing unit 113 performs a bronchial region extraction process.

  FIG. 10 is a flowchart showing an example of bronchial region extraction processing.

  When the bronchial region dividing unit 113 starts the bronchial region extraction process (S120), it acquires a binarized image (S121). For example, the binarized image generation unit 1130 sets the pixel value of the pixel to “255” when the CT value of the candidate region image is equal to or larger than the threshold value, and sets the pixel value of the pixel to “0” when the CT value is smaller than the threshold value. To obtain a binarized image. In this case, for example, the background image is an image in which the pixel value of the pixel is “0”, and the foreground image is an image in which the pixel value of the pixel is “255”.

  Next, the bronchial region dividing unit 113 performs centerline extraction processing (S122).

  FIG. 11 is a flowchart showing an example of the centerline extraction processing.

  When the centerline extraction unit 1131 starts the centerline extraction processing (S1220), the centerline extraction unit 1131 performs distance conversion (S1221).

  FIG. 12A is a diagram illustrating an example of a binarized image of 8 × 9 pixels. The distance conversion process (S1221) will be described with reference to an example in which the distance conversion image is generated from the binarized image shown in FIG. For example, the distance conversion unit 1132 performs the following processing.

  That is, the distance conversion unit 1132 scans the forward raster scanning mask shown in FIG. 12B in the forward direction (forward raster scanning) with respect to the binarized image shown in FIG. 12A. At this time, in the distance conversion unit 1132, when the pixel P (x, y) is a pixel of the foreground image other than “0”, four pixels (P (x−1, y−) adjacent to the pixel P (x, y) are used. 1), P (x, y−1), P (x + 1, y−1), P (x−1, y)) each pixel value (or distance value) is checked, and “1” is added to the minimum value. This value is set as the distance value of the pixel P (x, y).

  FIG. 13A is a diagram showing a state in which the pixel (2, 2) is set to P (x, y) and is sequentially scanned from the pixel (2, 2) using the forward raster scanning mask. .. The distance conversion unit 1132 scans the forward raster scanning mask from the upper left to the lower right on the drawing.

  Further, FIG. 13B is a diagram showing a state in which the distance value is set for the pixel (4, 2). As shown in FIG. 13B, when the pixel (4, 2) is P (x, y), P (x, y) is a pixel of the foreground image other than “0”. Therefore, the distance conversion unit 1132 has the minimum pixel value “0” of the adjacent four pixels (four pixels of (3,1), (3,2), (3,3), and (4,1)). “1” obtained by adding “1” to “” is set as the distance value of the pixel (4, 2).

  Further, FIG. 14A is also a diagram showing how the distance value is set. In FIG. 14B, the distance conversion unit 1132 sets the pixel (5, 3) to P (x, y) and adds “1” to the minimum value “1” of the pixel values of four adjacent pixels to “2”. Is set to the distance value.

  FIG. 14B is a diagram illustrating an example of the distance-converted image when the forward scan of the forward raster scan mask is completed. As shown in FIG. 14B, the distance value of each pixel of the foreground image is, for example, the distance from the boundary surface with the background image represented by the number of pixels. For example, pixel (4, 2) has a distance of “1” pixels from the boundary surface of the background image, and thus “1” is set as the distance value, and pixel (5, 3) is separated from the boundary surface of the background image. Since there is a distance of "2" pixels, "2" is set as the distance value.

  However, the distance value from the pixel (6, 3) to the pixel (6, 6) does not correspond to the distance from the boundary surface.

  For example, in the forward raster scanning mask shown in FIG. 12B, this is because pixels (P (x-1, y-1) and P (x-) on the left side of the pixel P (x, y). , Y)) and the pixel values (or distance values) of the upper pixels (P (x-1, y-1), P (x, y-1), etc.) are set, and the distance value is set. This is because that.

  Therefore, the distance conversion unit 1132 uses the reverse raster scanning mask shown in FIG. 12C, and as shown in FIG. On the drawing, scanning is performed in the reverse direction from the lower right to the upper left, and conversion into the final range image is performed.

  In this case, when the pixel P (x, y) is a pixel of the foreground image other than “0”, the distance conversion unit 1132 determines that four pixels (P (x + 1, y), P adjacent to the pixel P (x, y) are included. Each distance of (x-1, y + 1), P (x, y + 1), P (x + 1, y + 1)) is checked. Then, the distance conversion unit 1132 compares the value obtained by adding “1” to the minimum distance value with the distance value of the set pixel P (x, y), and determines a smaller value as the pixel P (x, y). ) Is adopted as the distance value.

  FIG. 15B is a diagram illustrating an example of setting the distance value for the pixel (6, 7). In this case, the distance conversion unit 1132 compares the value "1" obtained by adding "1" to the minimum distance value "0" of the adjacent pixel and the distance value "1" of the already set pixel (6, 7). Then, the smaller value “1” is adopted. Therefore, the distance conversion unit 1132 sets “1” as the distance value for the pixel (6, 7).

  FIG. 16A is a diagram illustrating an example of setting the distance value for the pixel (6, 6). In this case, the distance conversion unit 1132 compares the value “1”, which is obtained by adding “1” to the minimum distance value “0” of the adjacent pixel, with the preset value “2” of the pixel (6, 6), The small value "1" is set to the distance value of the pixel (6, 6).

  FIG. 16B is a diagram showing an example of the distance-converted image when the backward scan of the backward raster scan mask is completed. As shown in FIG. 16B, the distance value from pixel pixel (6,3) to pixel (6,6) is a value corresponding to the distance from the boundary surface.

  For example, as shown in FIG. 12C, this is because the pixel on the right side (P (x + 1, y) and P (x + 1, y + 1), etc.) and the pixel on the lower side with respect to the pixel P (x, y). This is because the distance value of the pixel P (x, y) is set by focusing on the distance value of (P (x-1, y + 1) and P (x, y + 1), etc.).

  The above is an example of the distance conversion processing (S1221). The examples of FIGS. 12A to 16B are examples of distance conversion in the case of two dimensions. For example, in the first embodiment, the three-dimensional case and the distance conversion can be performed.

  FIG. 17A is a diagram showing an example of forward scanning in a three-dimensional case. Further, FIG. 17B is a diagram showing an example of a forward raster scanning mask used in a three-dimensional case.

In this case, focusing on the pixel B ₂₀ of the forward raster scanning mask, for each CT image, as shown in FIG. 17A, the distance conversion unit 1132 changes the distance from the upper left to the lower right (lower left in FIG. 17A). While performing raster scanning (forward raster scanning) toward the upper right), the distance value of the pixel B ₂₀ is set. When the pixel B ₂₀ is a pixel of the foreground image other than “0”, the distance conversion unit 1132 determines the pixel value (or distance value) of 13 pixels (B _{10 to} B ₁₈ , B _{24 to} B ₂₇ ) adjacent to the pixel B _20. ), And a value obtained by adding “1” to the minimum value is set as the distance value of the pixel B ₂₀ . The distance conversion unit 1132 performs reverse raster scanning after the completion of forward raster scanning.

FIG. 18A is a diagram showing an example of a reverse raster scan, and FIG. 18B is a diagram showing an example of a reverse raster scan mask. The distance conversion unit 1132 checks the minimum distance value of 13 pixels (B _{21 to} B ₂₃ , B _{30 to} B ₃₈ ) adjacent to the pixel B ₂₀ and adds a value of “1” to the distance value and the set value. The distance value of the pixel B ₂₀ is compared, and the smaller distance value is set as the distance value of the pixel B ₂₀ .

  Returning to FIG. 11, when the distance conversion unit 1132 finishes the distance conversion processing (S1221), the centerline extraction processing unit 1133 then performs centerline extraction processing (S1222 to S1226).

  In other words, the centerline extraction processing unit 1133 performs point raster scanning on the image after distance conversion and has a distance value larger than “1” (or a pixel; hereinafter, it may be referred to as a “point”). For i, a circle having a radius of the distance R of the point i is defined. Then, the center line extraction processing unit 1133 defines the point having the distance value “1” as a point forming the contour line (S1222). A set of points having a distance value of “1” is, for example, a contour line.

  FIG. 19A is a diagram showing an example of a circle having a distance value R of the point A as a radius when the point A is the point i. As described above, the distance value of each pixel is, for example, a value in which the distance from the boundary line (or contour line; hereinafter, sometimes referred to as “contour line”) between the foreground image and the background image is represented by the number of pixels. Has become. Therefore, the distance value of the point A (or the pixel A at the position of this point A) represents the distance from the contour line, and it is possible to define a circle having the distance as a radius. When the point B is the i point, a circle having the radius r of the point B can be defined, and when the point C is the i point, a circle having the radius r of the point C can also be defined. It is possible.

  The point with the distance of "1" (or the pixel at the position of this point) is defined as the point forming the contour line.

  Returning to FIG. 11, next, the center line extraction processing unit 1133 determines whether or not the following formula (1) always holds for a circle of a certain point j other than the point i. Yes (S1223).

R _i > R _j , and the distance between points i and j <R _i (1)
In Expression (1), R _i represents the radius (or distance value) of the circle at the point i, and R _j represents the radius (or distance value) of the circle at the arbitrary point j.

  In the example of FIG. 19A, when the point A is the point i and the arbitrary point j is the point B, the radius R of the circle centered on the point A is smaller than the radius r of the circle centered on the point B. And the distance d between the two points is smaller than the radius R of the circle centered on the point A. In this case, the centerline extraction processing unit 1133 determines that the expression (1) holds.

  On the other hand, when the point A is the point i and the arbitrary point j is the point C, the radius R of the circle centered on the point A is larger than the radius r of the circle centered on the point C, but between the two points AC. The distance is larger than the radius R of the circle centered on the point A. In this case, the centerline extraction processing unit 1133 determines that the formula (1) is not established.

  In this processing (S1223), for example, the centerline extraction processing unit 1133 tries to find a circle whose radius is the shortest distance to the contour line of the point i and which is not surrounded by circles at other arbitrary points. .

  For example, in the example of FIG. 19A, when point A is point i and point B is point j, equation (1) is satisfied, and when point A is point i and point C is point j, equation (1) is satisfied. Does not meet. In this case, the circle centered on the point A encloses the circle centered on the point B, but does not enclose the circle centered on the point C. Further, the circle centered on the point A is not surrounded by other circles that satisfy the formula (1). Therefore, the circle centered on the point A surrounds other circles, but is not surrounded by other circles.

  The center line extraction processing unit 1133 has the distance value of the point A larger than the distance value of another point (for example, the point B) and the distance between the point A and the other point smaller than the distance value of the point A. At this time, it is determined that the circle centered on the point A is a circle that is not surrounded by circles centered on other points (Yes in S1223).

  A point i having a circle that is not surrounded by other circles may be referred to as a “skeleton”, for example. The points A and C are skeletons, but the point B is not.

  Returning to FIG. 11, when the center line extraction processing unit 1133 determines that the formula (1) is satisfied (Yes in S1223), the point i is set as a point forming the center line, and the radius Ri around the point i is the center. Is defined as the maximum circle (S1224). In the example of FIG. 19A, the point A is a point forming the center line, and a circle centered on the point A and having a radius R is a maximum circle. The point C is also the same.

  Returning to FIG. 11, next, the center line extraction processing unit 1133 determines whether or not the scanning of the forward raster is completed (all the points whose distance values are larger than “1” have been completed) (S1225).

  When the center line extraction processing unit 1133 determines that the scanning of the forward raster is completed (Yes in S1225), the center line is formed by connecting the points forming the center line (S1226). That is, the centerline extraction processing unit 1133 determines that a circle centered on the point A having a distance value larger than “1” is a circle centered on another point having a distance value larger than “1” (for example, point B). When the circle is not enclosed, the set of points A is extracted as the center line of the candidate area image.

  Then, the centerline extraction processing unit 1133 classifies each point forming the centerline into four types based on the following rules (S1226).

1) End point: the center of a maximal circle where all the points that coincide with the contour line are connected 2) Branch point: The center of the maximal circle where three or more separate connecting parts are in contact with the contour line 3) Branch : A line segment consisting of a set of points, both ends of which are end points or bifurcation points.

  FIG. 19B is a diagram illustrating an example of an end point T and a branch point B in a certain area. The end point is defined as in 1) above, but in other words, it is, for example, the center point of the maximum circle in contact with the contour line. As shown in FIG. 19B, each of the points T is the center point of the maximum circle that is in contact with the contour line.

  The branch point is defined as in 2) above, but in other words, it is the center point of a maximal circle having three or more points in contact with the contour line, for example. As shown in FIG. 19B, the maximum circles at the point B are in contact with the contour line at all three points.

  Further, in the example of FIG. 19B, the line segment TB and the line segment BB are both branches because both ends are end points or branch points.

  Further, FIG. 19C is a diagram showing an example of the center C of the circle. As shown in FIG. 19C, the maximum circle at the point C coincides with a part of the contour line, and its center C can be the center of the circle.

  The centerline can be considered as a set of points classified in this way, for example.

  Returning to FIG. 11, the centerline extraction processing unit 1133 classifies the points forming the centerline into four types (S1226), and ends the centerline extraction processing (S1227).

  On the other hand, when the forward raster scan is not finished (No in S1225), the centerline extraction processing unit 1133 moves to S1222 and performs the processes of S1222 and subsequent steps until the forward raster scan is finished.

  On the other hand, when the circle of a certain point i does not hold the formula (1) for any point j other than the point i (No in S1223), the circle centered on such a point i is another arbitrary point. It is surrounded by a circle centered on j. Therefore, the centerline extraction processing unit 1133 does not define such a point i as a centerline, and does not define a circle centered on the point i as a maximum circle, and proceeds to S1225. Transition.

  The three-dimensional example has been described in the distance conversion (S1221). The centerline extraction processing (S1222 to S1226) can also be performed in a three-dimensional example.

  In this case, in step S1222, the center line extraction processing unit 1133 has a radius of the distance value R of the point i with respect to the point i whose distance value is larger than “1” in the forward raster scan with respect to the image after the distance conversion. And a distance value of "1" is defined as a point forming the contour surface.

  Further, in S1223, the centerline extraction processing unit 1133 determines whether or not the sphere having the radius R centered on the point i satisfies the formula (1) for the sphere at any point j other than the point i. judge.

  Then, when the expression (1) is satisfied (Yes in S1223), the centerline extraction processing unit 1133 defines a sphere having a radius Ri centered on the point i as a maximum sphere (S1225). Further, in S1226, the centerline extraction processing unit 1133 forms a centerline by connecting the points forming the centerline (S1224), and classifies the points forming the centerline into four types (S1226).

  In this case, the centerline extraction processing unit 1133 may classify the contour line and the maximum circle in the two-dimensional example into four types as the contour surface and the maximum sphere in the three-dimensional case.

  The centerline extraction processing unit 1133 outputs, for example, information on each point forming the centerline and information on each classified point to the centerline dividing unit 1134.

  The above is the operation example (S1220 to S1227) of the center line extraction processing.

  Returning to FIG. 10, the bronchus region dividing unit 113 divides the center line (S123) after finishing the center line extraction processing (S122). For example, the centerline dividing unit 1134 divides the centerline into branches with the branch point as the center, based on information on each point forming the centerline, information on each classified point, and the like. In the example of FIG. 8A, the center line dividing unit 1134 divides the center line into a branch a1b1, a branch b1c, and a branch b1d1 around the branch point b1. The branch a1b1, the branch b1c, and the branch b1d1 are, for example, the respective divided center lines. The center line division unit 1134 outputs information on each divided branch, information on each point forming each branch, and the like to the angle calculation unit 1135.

  Returning to FIG. 10, next, the bronchial region dividing unit 113 performs linear approximation on each of the divided branches and detects an angle with respect to the approximated straight line (S124). For example, the angle calculation unit 1135 performs the following process for linear approximation.

  That is, the angle calculation unit 1135 applies the least-squares method to each point constituting each branch divided by the center line division unit 1134 to generate a straight line approximate to each branch. Specifically, the angle calculation unit 1135 applies the least squares method to each point (x, y) forming the branch, and calculates the slope a and the y intercept b of the quadratic equation y = ax + b as follows. A straight line that approximates a branch is generated by using a formula and calculating a quadratic equation.

  In equations (2) and (3), xi and yi are the positions of the points (x, y) that make up the center line, and x (-) and y (-) are the averages of the points (x, y). Represent each value.

  The examples of the formulas (2) and (3) are examples in which each point (x, y) is represented by two-dimensional coordinates. In some cases, the points forming the center line are represented by three-dimensional coordinates such as (x, y, z).

  FIG. 20A shows an example in which each of the points A to E forming the center line is represented by three-dimensional coordinates. In the case of three-dimensional coordinates, the angle calculation unit 1135 converts each point (x, y, z) into a polar coordinate system (r, θ, φ) and calculates the mode value of each branch in the (θ, φ) space. By doing so, the linear approximation in the (θ, φ) space is performed. The angle calculation unit 1135 performs conversion into a polar coordinate system using the following formula.

  FIG. 20B is a diagram showing an example of polar coordinate points of a branch calculated in this way. In the case of FIG. 20B, the angle calculation unit 1135 calculates (θ, φ) = (45 degrees, 30 degrees) as the mode value. Then, the angle calculation unit 1135 performs, in the (θ, φ) space, a linear approximation with a straight line that is (45 degrees, 30 degrees).

  As for the three-dimensional coordinates (x, y, z), the linear approximation may be performed using the least squares method as in the case of the two-dimensional coordinates. In this case, the angle calculation unit 1135 calculates the straight line that minimizes the vertical distance from each three-dimensional coordinate point by using a known method using principal component analysis (PCA). It is possible to obtain an approximate straight line.

  The above is an example of linear approximation. An example of the linear approximation may be performed by a known method other than the method of least squares or the method of determining the mode value described above. Note that the angle calculation unit 1135 stores, for example, the equations (2) to (6) in the internal memory, reads them at the time of processing, and substitutes the position of each point to obtain an approximated straight line.

  Next, the angle calculation unit 1135 calculates the angle of the straight line.

  FIG. 21A is a diagram showing an example of an approximate straight line. In FIG. 21A, the branch a1b1 is approximated by the straight line ab, the branch b1c is approximated by the straight line bc, and the branch b1d1 is approximated by the straight line bd. The angle calculation unit 1135 calculates the angle as follows, for example.

  That is, the angle calculation unit 1135 calculates the angle based on the slope a of each approximate straight line calculated by the equation (2) when each point forming the branch is represented by two-dimensional coordinates. Specifically, in the example of FIG. 21A, if the inclination of the straight line ab is a2, the inclination of the straight line bd is a3, and the angle formed by the two straight lines is θ1 (0 ≦ θ1 ≦ π / 2), the angle calculation is performed. The unit 1135 calculates the angle θ1 using the following formula.

  As shown in the example of FIG. 21A, the angle calculation unit 1135 sets the slope of the straight line bc as a2 and the slope of the straight line bd as a3 with respect to the other straight lines bc and bd, and uses Expression 7. Then, the angle θ2 is calculated with the angle θ1 as the angle θ2.

  Note that the angle calculation unit 1135 may use the calculated most frequent value of (θ, φ) as the angle of the straight line when each point forming the branch is represented by three-dimensional coordinates. In addition, the angle calculation unit 1135 calculates the angle using Expression (7) when each point forming the branch is represented by three-dimensional coordinates and the approximate straight line is calculated by the least squares method.

  The angle calculation unit 1135 obtains the angle θ by, for example, storing the formula (7) in the internal memory, reading it at the time of processing, and substituting the slope of the straight line into the formula (7). The angle calculation unit 1135 outputs information regarding the calculated angle, information regarding each straight line, and the like to the labeling unit 1136.

  Returning to FIG. 10, next, the bronchial region dividing unit 113 performs labeling (S125). For example, the labeling unit 1136 labels each straight line (or each branch) on the bronchus or honeycomb based on the calculated angle (S124).

  FIG. 21B is a diagram showing an example of labeling. For example, the labeling unit 1136 labels the straight line with the bronchus when the angle θ calculated in S124 is less than or equal to the threshold θth, and labels the straight line with the honeycomb lung when the angle θ is greater than the threshold θth.

  This is because, for example, as described in FIG. 5 (B) and FIG. 6 (B), the bronchi are distributed from the central direction to the peripheral direction or from the peripheral direction to the central direction. Is based on being distributed along the contour of the lung. For example, when there are two bronchi, the angles of the two straight lines that approximate the distribution of the bronchi are almost the same. On the other hand, when one is a bronchus and the other is a cell lung, for example, the angles of the two approximated straight lines may be greatly different and may be close to 90 degrees. The threshold value θth may be set, for example, in consideration of such a difference in distribution between the bronchi and the honeycomb lung.

  In the example of FIG. 21A, the labeling unit 1136 labels the straight line ab with the honeycomb because the angle θ1 formed by the straight line bd and the straight line bd is equal to or greater than the threshold value θth. In this case, the labeling unit 1136 may label the other straight line bd with the bronchus when determining the one straight line ab as the honeycomb lung. Regarding the relationship between the straight line bc and the straight line bd, the labeling unit 1136 labels the straight line bc with the honeycomb lung and the straight line bd with the bronchus because the angle θ2 with the straight line bd is greater than or equal to the threshold value θth.

  In the example of FIG. 21A, for example, when the angle θ1 formed by the straight line ab and the straight line bd is less than or equal to the threshold value θ, the labeling unit 1136 determines that the straight line ab is a bronchus, and the straight line ab and the straight line bd. Label the bronchus. In this case, the area dividing unit 1137 described later does not need to perform area division on the area including the two straight lines ab and bd determined to be the bronchus.

  When each point forming the branch is represented by three-dimensional coordinates, when the labeling unit 1136 approximates a straight line by the method of least squares, each straight line is determined by the threshold value θth as in the case of the two-dimensional coordinate. Label. In addition, for example, when the mode value of (θ, φ) is calculated, the labeling unit 1136 performs labeling as follows. That is, regarding the mode (θ, φ), the labeling unit 1136 defines the straight line represented by (θ, φ) as the bronchus when θ is smaller than the threshold θth and φ is smaller than the threshold φth. Label, otherwise label the lungs.

  Returning to FIG. 10, the bronchial region dividing unit 113 then divides the region (S126).

  FIG. 22 is a diagram illustrating an example of area division. Explaining using the example of FIG. 22, the area division unit 1137 performs area division as follows, for example.

  That is, the area division unit 1137 receives the binarized image of the candidate area from the binarized image generation unit 1130, and in the binarized image, the boundary area (pixels changing from “0” to “255”) and the straight line bc. The intersection e with is calculated. The area dividing unit 1137 may set the pixel (x, y) satisfying the quadratic equation representing the straight line bc among the pixels in the boundary area of the binarized image as the intersection point e. Further, the area dividing unit 1137 similarly calculates the intersection f of the straight line ab and the boundary area in the binarized image. Then, the region dividing unit 1137 divides the candidate region into a honeycomb lung region and a bronchial region by the straight line ge and the straight line gf with respect to the intersection point g of the straight line ab and the straight line bc.

  Then, the region dividing unit 1137 divides the candidate region with respect to the candidate region image received from the candidate region extracting unit 112 by the straight line ge and the straight line gf, and divides the region including the honeycomb lung, the labeled straight line ab, and the straight line bc. , And extract as a honeycomb lung region. On the other hand, the region dividing unit 1137 divides the candidate region by the straight line ge and the straight line gf, and extracts a region including the bronchus and the labeled straight line bd as the bronchus region.

  Alternatively, the area dividing unit 1137 expands the area of the candidate area image received from the candidate area extracting unit 112 to a straight line ge and a straight line gf using a known area expanding method with the end point a1 as a start point. Then, the enlarged area may be used as the honeycomb area. In addition, the area dividing unit 1137 expands the area up to the straight line ge and the straight line gf using the known area expansion method with the end point d1 (or the branching point d1) as a start point, and the expanded area as the bronchial area. Good. The area expansion method is, for example, a method of starting from a start point and including a point or the like satisfying a certain fixed criterion in the area to enlarge the area satisfying the certain criterion.

  The region division unit 1137 outputs, for example, an image of the lung field region excluding the bronchial region from the candidate region to the distribution feature extraction unit 114. Alternatively, the region division unit 1137 may output the divided image of the bronchial region to the distribution feature extraction unit 114, for example.

  Returning to FIG. 10, when the bronchial region dividing unit 113 divides the region (S126), the bronchial region extracting process ends (S127).

  Returning to FIG. 9, upon completion of the bronchial region extraction processing (S12), the dictionary registration module 110 performs shadow identification (S13) and extracts distribution characteristics (S14).

  For example, the distribution feature extraction unit 114 divides the lung field image output from the bronchial region division unit 113 into lattice regions for each predetermined pixel range, and identifies each lattice region as a normal image or an abnormal image. To do. The distribution feature extraction unit 114 may identify whether it is normal or abnormal by using, for example, SVM (Support Vector Machine).

  SVM is, for example, one of pattern recognition models using supervised learning. For example, the dictionary registration module 110 holds an image of a normal lattice and an image of an abnormal lattice in the internal memory as teacher data, and uses the teacher data to determine whether the input lattice-shaped lung field image is normal. To determine.

  Then, for example, the distribution feature extraction unit 114 creates (or extracts) a histogram as the distribution feature. The distribution feature extraction unit 114 creates a histogram in which the number of grid regions identified as abnormal is a feature amount. FIG. 3B is an example of the histogram.

  Returning to FIG. 9, the dictionary registration module 110 ends the feature extraction processing (S15).

  The dictionary registration module 110 stores the created histogram in the feature dictionary DB 120.

  The processing illustrated in FIG. 9 is also performed by the search module 130 as described above. The feature amount extraction unit 131 of the search module 130 also creates a histogram as a distribution feature and outputs it to the matching processing unit 135.

<2. Search process>
FIG. 23 is a flowchart showing an example of the search process.

  Upon starting the processing (S20), the search module 130 inputs a query image from the CT image capturing apparatus 300 (S21).

  Next, the search module 130 performs a feature extraction process (S10). The search module 130 performs the same processing as the feature extraction processing (S10 to S15 in FIG. 9) in the dictionary registration module 110, and creates a histogram for the query image.

  Next, the search module 130 performs collation processing (S22). For example, the matching processing unit 135 extracts a histogram of CT images in a certain range from the histogram of the query image. As the fixed range, for example, at each position in the histogram, the number of lattice areas identified as abnormal is the same or within a fixed range (± 1, ± 2, etc.). For example, the matching processing unit 135 may use a known method to determine the similarity with respect to the histogram of the query image, and extract the histogram of the CT image that is within a certain range with the similarity. .. The matching processing unit 135 outputs the matching result to the UI 140.

  Next, the image search device 100 displays the search result (S23). For example, the UI 140 displays, on the monitor 141, the query image and the CT image having the histogram of the CLIE image and the histogram within a certain range as the matching result.

  Then, the image search device 100 ends the search process (S24).

  For example, a doctor diagnoses a shadow and judges a case while looking at the monitor 141 on which the search result is displayed.

  As described above, in the image search device 100 according to the first embodiment, attention is paid to the fact that the distribution characteristics of the bronchus and the honeycomb lung are different, and the respective regions of the bronchus and the honeycomb lung are divided by utilizing the characteristics. Specifically, the image search device 100 generates a binarized image from the candidate region image, extracts a centerline from the binarized image, divides the centerline around the branch point, and divides each divided line. Calculate the angle of the centerline (or each branch). Then, in the image search apparatus 100, when the angle is equal to or smaller than the threshold value θth, the divided center line is determined to be bronchus, and when the angle is larger than the threshold value θth, the divided center line is determined to be the honeycomb structure (S125 in FIG. 10, FIG. 21 (B)).

  FIG. 24 is a diagram illustrating an example of dividing a bronchial region using the region expansion method. The area expansion method is, for example, a method of expanding the area by adding pixels or small areas within a certain reference from the start point to the area as described above. In the example of FIG. 24, the starting point is the throat, and the constant reference is within a range of 30 cm from the throat, and pixels or small areas within this range are the bronchial areas. In this case, as shown in FIG. 24, a region within 30 cm from the throat is determined as the bronchial region by the region expansion method.

  However, the honeycomb may cause the bronchi to become tractive bronchiectasis. In addition, the bronchial region may have individual differences or deformation. In such a case, the bronchus may not extend from the throat to 30 cm, but may extend to 34 cm as shown in FIG. In this case, in the area expansion method, the area from 30 cm to 34 cm from the throat may be erroneously determined to be the honeycomb lung area instead of the bronchial area.

  Therefore, in the first embodiment, the bronchus region and the honeycomb lung region are determined based on the angles of the respective divided center lines in consideration of the difference in the distribution characteristics of the bronchus and the honeycomb lung. Therefore, since the image search apparatus 100 divides each region of the bronchus and the honeycomb lung based on such a difference in distribution, each region of the bronchus and the honeycomb lung is compared with the case where the region expansion method is used. It is possible to divide the area accurately.

  Therefore, for example, the image search device 100 can determine that 30 cm to 34 cm from the throat is the bronchial region instead of the honeycomb lung region, and can search for similar cases in which this part is the bronchial region. Therefore, the image search device 100 can improve the search accuracy of similar cases. Further, since the search accuracy of similar cases is improved, it is possible to improve the efficiency of doctors' image diagnosis work, for example.

[Other Embodiments]
FIG. 25 is a diagram illustrating a hardware configuration example of the image search device 100.

  The image search apparatus 100 includes a CPU (Central Processing Unit) 160, an IF (Interface) 161, a ROM (Read Only Memory) 162, a RAM (Random Access Memory) 163, a memory 164, a monitor 141, and a keyboard 142.

  The CPU 160 realizes the functions of the dictionary registration module 110 and the search module 130 by, for example, reading a program stored in the ROM 162, loading the program into the RAM 163, and executing the loaded program. The CPU 160 corresponds to, for example, the dictionary registration module 110 and the search module 130.

  Instead of the CPU 160, a processor such as an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), a controller, or the like may be used.

  The IF 161 inputs the image data of the CT image from the image DB 200 and outputs the input image data to the CPU 160. Further, the IF 161 inputs the image data of the query image from the CT image capturing apparatus 300 and outputs the input image data to the CPU 160. The IF 161 corresponds to, for example, the dictionary registration module 110 and the search module 130.

  The memory 164 corresponds to the feature amount dictionary DB 120, for example.

  The image search device 100 according to the first embodiment described above may be a cloud server device, for example, and may transmit a search result to another device (for example, a client device). FIG. 26 is a diagram illustrating a configuration example of the image search device 100 in this case. The image search device 100 further includes an IF 170. The IF 170 transmits the search result output from the search module 130 to another device. The IF 170 may transmit the search result not only by a wired method but also by a wireless method. FIG. 27 is a diagram illustrating a hardware configuration example of the image search device 100 in this case. The IF 161 corresponds to the IF 170, for example, and can transmit the search result to another device.

  In the above-described first embodiment, the example in which the dictionary registration module 110 and the search module 130 perform the matching process after extracting the feature amount of the CT image or the query image has been described. For example, the dictionary registration module 110 and the search module 130 do not have to extract the feature amount. In this case, the search module 130 may perform the matching process using the CT image or the query image as they are. In this case, the feature amount extraction unit 111 of the dictionary registration module 110 does not have the distribution feature extraction unit 114, and the image of the lung field output from the bronchial region division unit 113 is stored in the feature amount dictionary DB 120. Also in the search module 130, the feature amount extraction unit 131 does not have the distribution feature extraction unit 134, and the image of the lung field output from the bronchial region division unit 133 is output to the matching processing unit 135 as it is. The matching processing unit 135 performs a straight line matching between the lung field image of the CT image and the lung field image of the query image, and determines the pixel value (or CT value) of each pixel of each image or the pixel value of a pixel included in a certain fixed area. Based on, the CT image similar to the query image may be searched.

  The above is summarized as follows.

(Appendix 1)
An image search program to be executed by a computer of an image search device for searching a first image for a second image similar to the first image,
An image of the candidate region is extracted from the first image, which is a cross-sectional image of the lung field region of the subject,
Generating a binarized image in which the pixel values of the image of the extracted candidate area are binarized,
Extracting the center line of the image of the candidate region from the binarized image,
Divide the center line at a branch point where the center line branches,
Based on the angle of the divided center line, the image of the candidate region is divided into a honeycomb region and a bronchus region,
An image retrieval program that causes the computer to execute a process of retrieving the second image from the first image based on an image of a lung field region other than the divided bronchus or an image of the divided region of the bronchus.

(Appendix 2)
The image search according to Note 1, wherein when the angle is equal to or less than a threshold value, the bronchial area is determined, and when the angle is greater than the threshold value, the candidate lung area is determined and the image of the candidate area is divided. program.

(Appendix 3)
Dividing the center line into a first center line and a second center line at the branch point,
When the angle between the first centerline and the second centerline is less than or equal to a threshold value, the first and second centerlines label the bronchi, and the first centerline is labeled with respect to the first centerline. When the angle formed by the second center line is larger than a threshold value, the first center line is labeled with the honeycomb lung, the second center line is labeled with the bronchus, respectively, and the first center line labeled with the honeycomb lung is The image including the candidate region is divided into a bronchial region and a honeycomb lung region, with the region including the honeycomb lung region and the region including the second centerline labeled with the bronchus as the bronchus region. The image search program according to attachment 2.

(Appendix 4)
The image search program according to Note 1, wherein an area including pixels having a pixel value of −1000 or less in the first image is extracted as an image of the candidate area.

(Appendix 5)
Generate a binarized image including a background image and a foreground image by binarizing the pixel values of the extracted image of the candidate region,
From the binarized image, the pixel value of each pixel of the background image is converted into a distance value that represents the shortest distance from the background image, and the center line of the image of the candidate region is extracted based on the distance value. An image retrieval program according to appendix 1, characterized in that.

(Appendix 6)
The first circle or sphere centered on the first pixel having a distance value larger than “1” is compared to the second circle or sphere centered on the second pixel having a distance value larger than “1”. 6. The image retrieval program according to appendix 5, wherein when the circles or spheres are not enclosed, the first pixel set is extracted as the center line of the image of the candidate area.

(Appendix 7)
The distance value of the first pixel is larger than the distance value of the second pixel, and the distance between the first pixel and the second pixel is smaller than the distance value of the first pixel. At this time, the image search program according to note 6, wherein the first circle or sphere is determined to be a circle or a sphere that is not surrounded by the second circle or sphere, respectively.

(Appendix 8)
The first pixel in a first circle or sphere centered on a first pixel having a distance value larger than "1", which is in contact with an outline that is a set of pixels having a distance value of "1" at three or more points. The image retrieval program according to appendix 1, wherein the center line is branched around the branch point as a branch point.

(Appendix 9)
The image retrieval program according to appendix 1, wherein a straight line that is approximated to the divided center line is generated, and the angle is calculated using the straight line.

(Appendix 10)
In an image search device that searches for a second image similar to the first image from the first image,
A candidate area extraction unit that extracts an image of the candidate area from the first image that is a cross-sectional image of the lung field area of the subject;
A binarized image generation unit that generates a binarized image by binarizing the pixel values of the image of the extracted candidate region,
A centerline extraction unit that extracts the centerline of the image of the candidate region from the binarized image,
A centerline dividing unit that divides the centerline at a branch point where the centerline is branched,
Based on the angle of the divided center line, a region dividing unit that divides the image of the candidate region into a honeycomb lung region and a bronchus region,
An image of a lung field region other than the divided bronchus or an image of a divided region of the bronchus, and a collation processing unit that retrieves the second image from the first image. Search device.

(Appendix 11)
An image retrieval method in an image retrieval device for retrieving a second image similar to the first image from a first image,
An image of the candidate region is extracted from the first image, which is a cross-sectional image of the lung field region of the subject,
Generating a binarized image in which the pixel values of the image of the extracted candidate area are binarized,
Extracting the center line of the image of the candidate region from the binarized image,
Divide the center line at a branch point where the center line branches,
Based on the angle of the divided center line, the image of the candidate region is divided into a honeycomb region and a bronchus region,
An image search method, wherein the second image is searched from the first image based on an image of a lung field region other than the divided bronchus or an image of a divided region of the bronchus.

10: Image Retrieval System 100: Image Retrieval Device 110: Dictionary Registration Module 111, 131: Feature Extraction Unit 112, 132: Candidate Region Extraction Unit 113, 133: Bronchus Region Division Unit 114, 134: Shadow Identification and Distribution Feature Extraction Unit (Distribution feature extraction unit)
1130: Binary image generation unit 1131: Center line extraction unit 1132: Distance conversion unit 1133: Center line extraction processing unit 1134: Center line division unit 1135: Angle calculation unit 1136: Labeling unit 1137: Region division unit 120: Feature amount Dictionary DB 130: Search module 135: Collation processing unit 140: UI
160: CPU 170: IF
200: Image DB 300: CT image pickup device

Claims (6)

An image search program to be executed by a computer of an image search device for searching a first image for a second image similar to the first image,
An image of the candidate region is extracted from the first image, which is a cross-sectional image of the lung field region of the subject,
Generating a binarized image in which the pixel values of the image of the extracted candidate area are binarized,
Extracting the center line of the image of the candidate region from the binarized image,
Divide the center line at a branch point where the center line branches,
Based on the angle of the divided center line, the image of the candidate region is divided into a honeycomb region and a bronchus region,
An image retrieval program that causes the computer to execute a process of retrieving the second image from the first image based on an image of a lung field region other than the divided bronchus or an image of the divided region of the bronchus.   The image according to claim 1, wherein when the angle is less than or equal to a threshold value, the bronchus area is determined, and when the angle is greater than the threshold value, the honeycomb area is determined and the image of the candidate area is divided. Search program. Generate a binarized image including a background image and a foreground image by binarizing the pixel values of the extracted image of the candidate region,
From the binarized image, the pixel value of each pixel of the background image is converted into a distance value that represents the shortest distance from the background image, and the center line of the image of the candidate region is extracted based on the distance value. The image search program according to claim 1, wherein:   The image search program according to claim 1, wherein a straight line that is close to the divided center line is generated, and the angle is calculated using the straight line. In an image search device that searches for a second image similar to the first image from the first image,
A candidate area extraction unit that extracts an image of the candidate area from the first image that is a cross-sectional image of the lung field area of the subject;
A binarized image generation unit that generates a binarized image by binarizing the pixel values of the image of the extracted candidate region,
A centerline extraction unit that extracts the centerline of the image of the candidate region from the binarized image,
A centerline dividing unit that divides the centerline at a branch point where the centerline is branched,
Based on the angle of the divided center line, a region dividing unit that divides the image of the candidate region into a honeycomb lung region and a bronchus region,
An image of a lung field region other than the divided bronchus or an image of a divided region of the bronchus, and a collation processing unit that retrieves the second image from the first image. Search device. An image retrieval method in an image retrieval device for retrieving a second image similar to the first image from a first image,
An image of the candidate region is extracted from the first image, which is a cross-sectional image of the lung field region of the subject,
Generating a binarized image in which the pixel values of the image of the extracted candidate area are binarized,
Extracting the center line of the image of the candidate region from the binarized image,
Divide the center line at a branch point where the center line branches,
Based on the angle of the divided center line, the image of the candidate region is divided into a honeycomb region and a bronchus region,
An image search method, wherein the second image is searched from the first image based on an image of a lung field region other than the divided bronchus or an image of a divided region of the bronchus.