JP2009142552A - Image processor, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, an image processing method and an image processing program capable of reducing a load on an observer when observing the inner side of a body through the use of a capsule type endoscope. <P>SOLUTION: The image processor 1 includes: an image acquiring part 11 for acquiring a group of a series of internal images constituted of a plurality of internal images acquired by repetitively imaging while moving inside the body; and a calculating part 50 for performing image processing with respect to the internal images acquired by the image acquiring part 11, and calculating an index value which expresses probability where the internal region of interest is caught in the group of the series of internal images. The probability where the internal region of interest is caught in the group of the series of internal images is, for example, whether or not the inward organ region of interest being the internal region of interest, or the internal image obtained by imaging the region of interest exists in the group of the series of internal images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program for processing a series of in-vivo images formed by a plurality of in-vivo images obtained by repeatedly capturing images while moving in the body.

  In recent years, in the field of endoscopes, swallowable capsule endoscopes have been developed. Capsule endoscopes are peristaltic movements after being swallowed from the subject's mouth for observation (examination), and then passed through the esophagus, stomach, small intestine, large intestine, and other bodies. The inside of the body is sequentially imaged while moving with other contents. The number of images to be picked up is generally indicated by an imaging rate (2 to 4 frames / sec) × dwelling time in the body (8 hours = 8 × 60 × 60 sec), and is a large number of tens of thousands. An observer such as a doctor examines the inside of the subject by observing a series of captured in-vivo images.

JP 2000-23980 A

  By the way, since the capsule endoscope moves in the body by a peristaltic motion, for example, if the peristaltic motion is weakened due to diabetes or the like, a phenomenon that the capsule endoscope stagnates in a specific organ site (passage abnormality) occurs. There is a case. For example, when a passage abnormality occurs in the stomach, the battery runs out when the capsule endoscope reaches the small intestine or the large intestine, and the small intestine or the large intestine is not photographed. In addition, when taking an image of the inside of a body using a capsule endoscope, if the inside of the body is washed and the contents are aspirated before taking the image, the bleeding site may become indeterminate, so the inside of the body may not be washed. is there. In this case, the capsule endoscope may not be able to take an image of the mucosal surface due to a large amount of contents remaining in the body.

  Because the observer had to observe all of the in-vivo images to see if there was a test failure such as when the organ site of interest was not imaged or when the mucosal surface was not imaged, The burden was great.

  The present invention has been made in view of the above, and provides an image processing device, an image processing method, and an image processing program that reduce the burden on the observer when observing the inside of a body using a capsule endoscope. The purpose is to do.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention includes a series of in-vivo image groups configured by a plurality of in-vivo images obtained by repeatedly capturing images while moving in the body. An image acquisition unit to acquire, and a calculation unit that calculates an index value representing a probability that the in-vivo region of interest is captured in the series of in-vivo image groups using the in-vivo image acquired by the image acquisition unit And. Here, the body region refers to a region desired by the observer such as the stomach, small intestine, esophagus, blood vessel, and other organ sites that have unique functions in the body, as well as the surface of the mucous membrane in the organ site (hereinafter referred to as the body region). “A region of interest”). The in-vivo image is an image showing the above-described in-vivo region and contents remaining in the body. Note that the in-vivo image may be a part of which the in-vivo region of interest is copied, or the in-vivo region of interest of interest may be copied over the entire in-vivo image.

  Further, an image processing apparatus according to the present invention includes an image acquisition unit that acquires a series of in-vivo image groups configured by a plurality of in-vivo images obtained by repeatedly capturing images while moving in the body, and the image acquisition unit. A display unit that displays an index value that indicates a probability that the in-vivo region of interest is copied in the series of in-vivo images calculated using the acquired in-vivo images. .

  The image processing method according to the present invention includes an image acquisition step of acquiring a series of in-vivo images composed of a plurality of in-vivo images obtained by repeatedly capturing images while moving in the body, and the image acquisition step. And a calculation step of calculating an index value representing a probability that the in-vivo region of interest is captured in the series of in-vivo images using the acquired in-vivo images.

  Further, an image processing program according to the present invention includes an image acquisition step of acquiring a series of in-vivo images composed of a plurality of in-vivo images obtained by repeatedly capturing images while moving in the body, and the image acquisition step. Using the acquired in-vivo images to cause a computer to perform a calculation step of calculating an index value indicating the probability that the in-vivo region of interest is captured in the series of in-vivo image groups. .

  According to the image processing apparatus, the image processing method, and the image processing program of the present invention, it is possible to reduce the burden on the observer when observing the inside of the body using a capsule endoscope.

  Hereinafter, an image processing apparatus, an image processing method, and an image processing program that are the best mode for carrying out the present invention will be described with reference to the accompanying drawings. The image processing apparatus according to the embodiment of the present invention is described as processing a series of in-vivo images composed of in-vivo images in which the inside of the esophagus, stomach, small intestine, and large intestine is sequentially captured by a capsule endoscope. To do. The series of in-vivo images does not include images outside the body taken before being swallowed by the body. In addition, this invention is not limited by each embodiment. In the description of the drawings, the same or corresponding parts are denoted by the same reference numerals.

(Embodiment)
FIG. 1 is a diagram showing a schematic configuration of an image processing apparatus 1 according to an embodiment of the present invention. As shown in FIG. 1, the image processing apparatus 1 includes an input unit 10 that inputs various information, an output unit 20 that outputs various information, a storage unit that stores in-vivo images, programs related to processing operations of the image processing apparatus 1, and the like. 30 and a control unit 40 that controls the operation of each unit of the image processing apparatus 1. Furthermore, the image processing apparatus 1 is electrically connected to the control unit 40, processes in-vivo images, and displays an index value indicating the probability that the in-vivo region of interest is captured in a series of in-vivo image groups. A calculating unit 50 for calculating is provided. The image processing apparatus 1 is mounted on an electronic device such as a portable terminal or a personal computer, and performs image processing of an in-vivo image captured by an imaging apparatus such as a capsule endoscope described later.

  The input unit 10 is realized by a keyboard, a mouse, a data communication interface, and the like, and includes an image acquisition unit 11. The image acquisition unit 11 is realized by an interface corresponding to a portable storage medium such as various memory cards, CDs, and DVDs, and receives input of image information of in-vivo images constituting a series of in-vivo images from the portable recording medium. .

  The output unit 20 is realized by a liquid crystal display or the like. In particular, the output unit 20 includes an index value display unit 21 that displays an index value, and a warning notification unit 22 that notifies an observer of a warning corresponding to the index value. The output unit 20 also displays a GUI (Graphical User Interface) screen that requests the operator to input various processing information.

  The storage unit 30 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and stored, a hard disk connected by a built-in or data communication terminal, an information storage medium such as a CD-ROM, and a reading device thereof. Stores processing programs and the like. The storage unit 30 includes an in-vivo image storage unit 31 that stores a series of in-vivo image groups.

  The control unit 40 is realized by a CPU and controls processing and operation of each unit of the image processing apparatus 1 by executing various processing programs stored in the recording unit 40.

  The calculation unit 50 is realized by the CPU, performs image processing on the in-vivo image based on various processing programs executed by the control unit 40, and calculates a series of index values of the in-vivo image group. As shown in FIG. 1, the calculation unit 50 includes a section setting unit 51 and a presence / absence determination unit 52, and calculates an index value indicating the probability that an organ site of interest is imaged in a series of in-vivo image groups. . In addition, the calculation unit 50 includes a region-of-interest setting unit 56, a division unit 57, a selection unit 58, and an in-vivo image specifying unit 59, and the region of interest of the target organ site is captured in a series of in-vivo image groups. An index value indicating the probability is calculated. In the present embodiment, the calculation unit 50 calculates the index value in the range of 0.0 to 1.0.

  The section setting unit 51 sets a target section that is a section including the target organ part based on information input by the observer through the input unit 10 or the GUI screen or information stored in the storage unit 30 in advance. For example, when the observer designates a specific organ part as the target organ part, the section setting unit 51 sets the start part to the end part of the designated organ part as the attention part.

  The presence / absence determination unit 52 specifies an organ part that is captured in each in-vivo image for each in-vivo image, an organ part that is specified by the specifying unit 53 for each in-vivo image, and an in-vivo image that captures the organ part. Are associated with each other, and a measurement unit 55 that measures the time taken to capture in-vivo images associated with the same organ site when a series of in-vivo image groups is captured. In the present embodiment, the presence / absence determination unit 52 includes a section of interest in the series of in-vivo images based on the section of interest in the series of in-vivo images and the imaging time of the organ sites located above and below the section of interest. It is determined whether there is an in-vivo image obtained by imaging the entire area, that is, the start point portion to the end point portion of the attention section. The measuring unit 55 calculates the difference between the time when the first in-vivo image is captured and the time when the last in-vivo image is captured among the plurality of in-vivo images associated with the same organ site by the associating unit 54. This is the imaging time of the organ site.

  The region-of-interest setting unit 56 sets the region of interest based on information input through the input unit 10 or the GUI screen by the observer or information stored in the storage unit 30 in advance. In the present embodiment, the region of interest is a mucous membrane region, and the region other than the region of interest is a content or a stool. When there is an abnormality in each organ site, abnormalities such as bleeding and polyps are observed on the mucosal surface. Therefore, it is necessary for the observer to determine whether or not the mucosal surface is captured in a series of in-vivo images. It becomes important information. In the present embodiment, the mucosal surface including both the abnormal mucosa and the normal mucosa is set as the region of interest, but only the abnormal mucosa can be set as the region of interest.

  The dividing unit 57 divides each in-vivo image into image portions, and the selection unit 58 selects a region of interest from each image portion. The in-vivo image specifying unit 59 specifies an in-vivo image having an image portion selected as the region of interest as an in-vivo image obtained by capturing the region of interest.

  Next, an image processing procedure performed by the image processing apparatus 1 will be described with reference to FIG. As illustrated in FIG. 2, the control unit 40 acquires in-vivo images constituting a series of in-vivo image groups via the image acquisition unit 11 (step S100). Thereafter, the calculation unit 50 controls the section setting unit 51 to set a section of interest (step S110). In addition, the calculation unit 50 controls the specifying unit 53 to specify the organ site imaged in each in-vivo image (step S120). Next, the calculation unit 50 controls the presence / absence determination unit 52 to determine whether or not an in-vivo image obtained by imaging the entire area of interest exists in the series of in-vivo image groups based on the organ identification result. Accordingly, an index value indicating the probability that the organ site of interest is imaged is calculated (step S130).

  Thereafter, the calculation unit 50 determines whether or not the calculated index value is greater than or equal to a predetermined threshold (step S140). The predetermined threshold value is a threshold value for determining whether or not there is a high probability that an organ site of interest is imaged in a series of in-vivo image groups. When the index value is equal to or greater than the threshold value (step S140: Yes), the calculation unit 50 controls the dividing unit 57, the selection unit 58, and the in-vivo image specifying unit 59 to specify the in-vivo image obtained by capturing the region of interest in the target section ( Step 150). Next, the calculation unit 50 calculates an index value indicating the probability that the region of interest in the section of interest is captured based on the ratio of the region of interest captured in the in-vivo image obtained by capturing the section of interest (step S160). After that, the control unit 40 displays, in the index value display unit 21, the index value indicating the probability that the attention section is captured and the index value indicating the probability that the region of interest in the attention section is captured in the series of in-vivo images. It is displayed (step S170). On the other hand, when the index value indicating the probability that the section of interest is captured in the series of in-vivo image groups is less than the threshold (step S140: No), the control unit 40 displays only this index value (step S170). .

  Next, the control unit 40 determines whether each index value is less than a threshold value set in advance for each index value (step S180). When at least one of the index values is less than the threshold value (step S180: Yes), the control unit 40 notifies the warning corresponding to the index value that is less than the threshold value using the warning notification unit 22, and performs image processing. Is finished (step S190). If each index value is equal to or greater than the threshold value, the control unit 40 ends the image processing as it is (step S180: No).

  The image processing apparatus 1 performs the processing of steps S100 to S190, calculates and displays two index values, and notifies a warning corresponding to each index value. Therefore, the observer can check the index value and the warning content, determine whether or not the capsule endoscope has taken a series of in-vivo images, and has caused a poor examination. In this case, it is possible to estimate what inspection failure has occurred.

  In step 120, the specifying unit 53 uses the fact that the file size of each in-vivo image after DCT encoding and the amount of change in the file size differ depending on the organ site imaged in the image, Determine the organ that is being imaged.

  The reason why the file size of each in-vivo image after DCT encoding differs depending on the organ part being imaged is that the shape of the mucosal surface of each organ part is different. The mucosal surface of the stomach has fewer irregularities than the small intestine with villi. For this reason, the in-vivo image obtained by imaging the stomach has a higher correlation between peripheral pixels in the image than the in-vivo image obtained by imaging the small intestine. Further, when the image is converted into a frequency space, the frequency component of the in-vivo image obtained by imaging the stomach is biased to a low frequency, and the frequency component of the in-vivo image obtained by imaging the small intestine is distributed from a low frequency to a high frequency. Therefore, as shown in FIG. 3, the file size after DCT encoding of the in-vivo image obtained by imaging the esophagus or stomach is smaller than the file size after DCT encoding of the in-vivo image obtained by imaging the small intestine. The mucosal surface of the large intestine has less irregularities than the mucosal surface of the small intestine. Therefore, as shown in FIG. 3, the file size after DCT encoding of the in-vivo image capturing the large intestine is the same as the in-vivo image capturing the small intestine. It becomes smaller than the file size after DCT encoding.

  In addition, the amount of change in the file size of each in-vivo image after DCT encoding differs depending on the organ part being imaged because the moving speed of the capsule endoscope in each organ differs. For example, since the contents of the large intestine are clogged, the capsule endoscope is stagnant. Therefore, when the in-vivo images are compared in the time series in which the images are taken, the change between the images is small. On the other hand, in the small intestine, the capsule endoscope moves more smoothly than the large intestine, so that the change between images is large. Therefore, as shown in FIG. 4, the change amount of the file size after DCT encoding of the in-vivo image obtained by imaging the large intestine is small, and the change amount of the file size after DCT encoding of the in-vivo image obtained by imaging the small intestine is large.

  Therefore, first, the identifying unit 53 determines whether the organ site captured in each in-vivo image is the esophagus, stomach, or small intestine based on the file size after DCT encoding. In addition, the specifying unit 53 sets the amount of change in the file size after DCT encoding for an in-vivo image captured later in time than an in-vivo image in which it is determined that the organ site being imaged is the small intestine. Based on this, it is determined whether the organ part being imaged is the small intestine or the large intestine.

  The organ identification processing procedure performed by the identification unit 53 in step S120 will be described with reference to FIG. First, the identifying unit 53 calculates the file size of each in-vivo image after DCT encoding (step S121). Thereafter, the specifying unit 53 calculates an average value of the file sizes of the entire series of in-vivo image groups (step S122). Further, the identifying unit 53 calculates the amount of change in the file size of the in-vivo images before and after adjacent in time series in a series of in-vivo images (step S123). Next, the specifying unit 53 calculates an average value of the amount of change in the file size of the entire series of in-vivo image groups (step S124). Thereafter, the specifying unit 53 specifies the organ part imaged in each in-vivo image based on the file size and the overall average of the file size, or the overall average of the change amount of the file size and the change amount of the file size, The process returns to step S120 (step S125). In addition, when a series of in-vivo image groups memorize | stored in the in-vivo image storage part 31 is not DCT-encoded, the control part 40 performs a DCT encoding process with respect to a series of in-vivo image groups previously.

In step S125, the specifying unit 53 first calculates a threshold value T _Fsize that serves as an organ determination criterion by the following equation (1) based on the overall average F _sizeAve of the file size and a preset variable M. The variable M is a determination parameter that can be arbitrarily changed by the observer according to the age, past history, etc. of the subject. Thereafter, the identification unit 53 determines whether or not the file size F _size of each in-vivo image is smaller than the threshold value T _Fsize as shown in the following equation (2).
T _Fsize = F _sizeAve + M (1)
F _size <T _Fsize (2)

When the file size F _size is smaller than the threshold value T _Fsize , the specifying unit 53 determines that the organ part captured in the in-vivo image is the esophagus or the stomach. When the file size F _size is larger than the threshold value T _Fsize , It is determined that the organ part imaged in (1) is the small intestine or the large intestine.

Further, the specifying unit 53 calculates a threshold value T _{FsizeDiff serving} as an organ determination criterion by the following equation (3) based on the overall average F _sizeDiffAve of the change amount of the file size and the preset variable N. The variable N is a determination parameter that can be arbitrarily changed by the observer according to the age, past history, etc. of the subject. After that, the specifying unit 53 associates the in-vivo images captured in time series later than the in-vivo images determined to have captured the small intestine with each in-vivo image as shown in the following formula (4). It is determined whether or not the _obtained file size F _sizeDiff is smaller than the threshold value T _FsizeDiff .
T _FsizeDiff = F _sizeDiffAve + N (3)
F _sizeDiff <T _FsizeDiff (4)

When the file size change amount F _sizeDiff is smaller than the threshold value T _FsizeDiff , the specifying unit 53 determines that the organ part imaged in the in-vivo image is the large intestine, and the file size change amount F _sizeDiff is larger than the threshold value T _FsizeDiff. In this case, it is determined that the organ part imaged in the in-vivo image is the small intestine.

  FIG. 3 is a diagram showing the transition of the change in file size with respect to the imaging time when each in-vivo image was taken. The file size shown in the figure is an average calculated for each predetermined number of time series. The file size. FIG. 4 is a diagram showing the change of the change amount of the file size with respect to the imaging time. The change amount of the file size shown in the figure is an average file calculated for each predetermined number of time series. The amount of change in size. In the processing using the equations (2) and (4), the specifying unit 53 replaces the file size of each in-vivo image and calculates the average calculated for each predetermined number of time series as shown in FIGS. Using the file size and the amount of change thereof, the organ site captured in the in-vivo image may be determined for each predetermined number.

  Further, immediately after the process of step S122, the specifying unit 53 performs the determination according to the formula (2), and performs the process after step 123 on the in-vivo image in which it is determined that the small intestine or the large intestine is captured. Good. In this case, the specifying unit 53 replaces the average value of the change amount of the file size in the whole series of in-vivo images with the average of the change amount of the file size in the in-vivo image determined that the small intestine or the large intestine is imaged. Organ identification processing is performed using the value.

  Next, the procedure of the index value calculation process performed by the calculation unit 50 and the presence / absence determination unit 52 in step S130 will be described with reference to FIG. First, the presence / absence determination unit 52 determines whether or not the imaging time of the organ part corresponding to the target section is equal to or longer than a predetermined threshold time (step S131). When the imaging time is equal to or longer than the threshold time (step S131: Yes), the presence / absence determining unit 52 determines whether the imaging time of the organ part above the target section is within a predetermined threshold range (step). S132). When the imaging time is within the threshold range (step S132: Yes), the presence / absence determining unit 52 determines whether the imaging time of the organ part below the target section is within a predetermined threshold range ( Step S133). When the imaging time is also within the threshold range (step S133: Yes), the calculation unit 50 sets the index value to 1.0, for example, and returns to step S130 (step S134). On the other hand, when the imaging time of the organ part corresponding to the attention interval is less than the threshold time (step S131: No), the imaging time of the organ part above the attention interval is outside the threshold range (step S132: No), or When the imaging time of the organ part below the target section is outside the threshold range (step S133: No), the calculation unit 50 sets the index value to 0.0 and returns to step S130 (step S135).

  Here, the threshold time for determining the imaging time of the organ part corresponding to the target section is a time required at least when the capsule endoscope passes through the organ part, and is set in advance for each organ part. Further, the threshold range for determining the imaging time of the organ part above or below the target section is a time range in which it can be determined that the capsule endoscope has passed each organ part normally, and is previously set for each organ part. Is set. If there is no upper or lower organ part as in the case where the attention section is the esophagus or the large intestine, whether or not there is an image taken outside the body instead of an image obtained by imaging the upper or lower organ part Judge with.

  The presence / absence determination unit 52 determines whether or not the capsule endoscope has performed imaging within the attention section in the process of step S131. Further, the presence / absence determination unit 52 performs imaging while normally passing through an organ part above or below the target section when the capsule endoscope captures a series of in-vivo images in steps S132 to S133. It is determined whether or not. If it is determined that the capsule endoscope has taken images within the interval of interest, and that it has been taken while passing normally through the upper and lower organs of the interval of interest, it will be included in a series of in-vivo images. It can be estimated that the entire area from the start point to the end point of the section of interest is normally captured. In this case, since it can be seen that the in-vivo images obtained by capturing the entire region of interest exist in the series of in-vivo image groups, the calculation unit 50 sets the index value indicating the probability that the target organ part is imaged to 1 .0.

  On the other hand, if it is determined that the imaging is not performed within the attention section, or if it is determined that the upper or lower organ part of the attention section is not normally captured, It can be estimated that any position from the start point to the end point of the section is not normally captured. In this case, since it is understood that the entire region of interest is not captured in the series of in-vivo image groups, the calculation unit 50 sets the index value to 0.0.

  Next, the in-vivo image specifying process performed by the dividing unit 57, the selecting unit 58, and the in-vivo image specifying unit 59 in step S150 will be described with reference to FIG. The division unit 57 and the selection unit 58 divide the image using the DCT coefficient calculated for each pixel block of 8 × 8 pixels when an image encoded by JPEG or the like is decoded. A region of interest is selected from the image portion. Specifically, as illustrated in FIG. 7, first, the calculation unit 50 acquires the DCT coefficient of each pixel block of each in-vivo image decoded by the control unit 40 (step S151). Next, the dividing unit 57 divides each in-vivo image for each predetermined category based on the DCT coefficient of each pixel block (step S152). Thereafter, the selection unit 58 selects an image portion belonging to a category set as a region of interest in advance from the divided image portions as a region of interest (step S153). The image specifying unit 59 specifies the in-vivo image having the image portion selected as the region of interest by the selecting unit 58 as the in-vivo image obtained by capturing the region of interest in the section of interest, and returns to step 150 (step S154).

  Here, the DCT coefficient will be described. The JPEG DCT coefficient is a coefficient obtained by performing discrete cosine transform on an image portion corresponding to each pixel block, and is a coefficient indicating a frequency component of the image portion corresponding to each pixel block. Further, as shown in FIG. 8, the DCT coefficient is specifically a set of 64 DCT coefficients which are each element of the 8 × 8 quantization matrix. Of these 64 DCT coefficients DCT1 to DCT64, DCT1 corresponds to a DC component, and DCT2 to DCT64 correspond to an AC component. Furthermore, as shown in FIG. 8, DCT2 to DCT64 indicate frequency information from low frequency components to high frequency components, respectively. In JPEG, RGB values are converted into Y values indicating luminance information and U values and V values indicating color information, and then discrete cosine transform is performed for each of the Y values, U values, and V values. A DCT coefficient for each of the U value and the V value is calculated. That is, each pixel block has 3 × 64 = 192 DCT coefficients.

  By the way, in the in-vivo image, for example, the bleeding region is greatly different in color from the surrounding mucosal surface. Further, for example, an abnormal mucosa in which the villi are peeled off and faded in white is greatly different in color and mucous membrane pattern (texture) compared to a normal mucous membrane. In addition, the color and texture of the image portion where each of the contents of the mucous membrane surface, the portion where bubbles are generated, and the stool are imaged are greatly different. Therefore, the dividing unit 57 uses the DCT coefficient including information on the color and the surface pattern (texture) as a feature amount, thereby allowing the contents region in which feces etc. are floating, the bubble region in which bubbles are generated, The image portion of each in-vivo image is divided into regions such as an abnormal mucosa region and a normal mucosa region.

  FIG. 9 is a diagram illustrating a procedure of the dividing process performed by the dividing unit 57 in step S152. As shown in FIG. 9, the dividing unit 57 calculates a multidimensional DCT feature amount of each image block based on the DCT coefficient of each pixel block (step S521). Thereafter, the dividing unit 58 classifies each pixel block into a plurality of clusters based on the multi-dimensional DCT feature amount (step S522). Next, the dividing unit 57 reads teacher data (step S523). The teacher data is dictionary data of multidimensional DCT feature quantities that have already been classified for each category, and is stored in the storage unit 20 in advance. Thereafter, the dividing unit 57 determines the category to which each cluster belongs based on the teacher data, and estimates the category of each pixel block classified into each cluster (step S524). Thereafter, the dividing unit 57 divides each in-vivo image into image parts for each category, and returns to step S152 (step S525).

  Here, the multi-dimensional DCT feature quantity is a feature vector having DCT1 to DCT64 as elements, and is a feature vector provided on a feature space having frequency information indicated by a DCT coefficient as a feature axis. For this reason, the multi-dimensional DCT feature amount is a maximum 192-dimensional feature amount.

  In step S522, the dividing unit 57 classifies the multidimensional DCT feature values of each pixel block into a plurality of clusters using a known clustering method such as hierarchical clustering and a k-means method. In the present embodiment, the clustering method is not particularly limited.

  In step S524, the dividing unit 57 determines a category to which each cluster belongs based on, for example, the distance between the clusters in the feature space. For example, the dividing unit 57 calculates the inter-cluster distance between each cluster after the clustering process and each cluster in the teacher data, and the category to which the cluster in the teacher data having the smallest inter-cluster distance belongs is obtained after the clustering process. It is determined that the category belongs to the cluster.

  Next, an index value calculation process performed by the calculation unit 50 in step S160 will be described. In step S160, the calculation unit 50 calculates an index value indicating the probability that the region of interest in the section of interest is imaged based on the area ratio of the region of interest in the section of interest with respect to the entire image. As illustrated in FIG. 10, the calculation unit 50 calculates the area ratio of the region of interest (the number of pixels in the region of interest) with respect to the entire image (the total number of pixels) for each in-vivo image obtained by capturing the attention section (step S161). Thereafter, the calculation unit 50 calculates the average value of the area ratio of the region of interest as the ratio of the region of interest captured, and returns to step S160 using this average value as an index value (step S162). The calculation unit 50 calculates an area ratio other than the region of interest, for example, the area ratio of the bubble region and the content region with respect to the entire image, instead of the area ratio of the region of interest or together with the area ratio of the region of interest. You may calculate the average value of area ratio as an index value.

  In step S190, the warning content performed by the control unit 40 includes, for example, the fact that there is no in-vivo image that captures the section of interest, the fact that the region of interest is captured at a low rate, or the region other than the region of interest is captured. For example, there are a large number of cases.

  The image processing apparatus 1 according to the embodiment of the present invention calculates and displays an index value of a series of in-vivo images, so that an observer can view a series of in-vivo images before observing the series of in-vivo images. The imaging situation can be grasped. For this reason, according to the embodiment of the present invention, it is possible to reduce the burden on the observer when the inside of the body is observed using the capsule endoscope.

  In the present embodiment, the ratio of the region of interest captured in the in-vivo image obtained by capturing the section of interest is calculated as the index value. However, in the entire in-vivo images constituting the series of in-vivo images, the region of interest and / or A ratio in which an area other than the region of interest is imaged may be calculated as an index value. For example, an average value of the area ratios of the mucosal regions captured in a series of in-vivo image groups may be calculated as an index value regardless of the organ site.

  FIG. 11 is a diagram illustrating an example of a display screen displayed on the output unit 20. On the display screen 60, an in-vivo image 61, a playback button 62, a reverse playback button 63, a position display 64, index value displays 65 and 66, and a warning display 69 are displayed.

  The in-vivo image 61 is sequentially updated in accordance with the selection of the playback button 62 or the reverse playback button 63 by the observer, in the time series order of imaging or the reverse order of the time series. The position display 64 displays how much the in-vivo image 61 is positioned in the time-series order in the series of in-vivo image groups.

  The index value display 65 displays an index value indicating the probability that an organ site of interest is imaged in a series of in-vivo image groups. In the case shown in FIG. 11, a plurality of organ parts, for example, the stomach, the small intestine and the large intestine are set as target organ parts of interest, and the index value display 65 is a character part indicating an organ part having an index value of 1.0. Is turned on, and the character portion indicating the organ part with the index value of 0.0 is turned off. In the case shown in FIG. 11, the character portions of the stomach and small intestine are lit, and it can be seen that the index value indicating that the target organ site is the stomach and small intestine is 1.0. On the other hand, the character portion of the large intestine is extinguished, and it can be seen that the index value indicating that the organ site of interest is the large intestine is 0.0.

  The index value display 66 displays an index value indicating the probability that the mucous membrane area that is the region of interest is imaged. In particular, the index value display 67 displays an index value indicating the rate at which the mucous membrane region is imaged in a series of in-vivo image groups, and the index value display 68 is the target organ site, in the case shown in FIG. An index value indicating the rate at which the mucosa region is imaged in the in-vivo image in which the small intestine is imaged is displayed.

  The warning display 69 displays a warning based on the index value. In the case illustrated in FIG. 11, the index value indicating the probability that the large intestine is imaged is equal to or less than the threshold value, and the warning display 69 displays that there is a possibility that the large intestine is not normally imaged.

  According to the display screen 60, the observer can check the index value indicating the probability that the in-vivo region of interest is imaged in the series of in-vivo image groups, and thus grasps the imaging state of the series of in-vivo image groups. be able to.

(Modification 1)
In the above-described embodiment, the results of determining whether the organ part corresponding to the target section, the upper part of the target section, or the lower part of the target part are imaged are summarized, and attention is paid to a series of in-vivo image groups. Although the index value indicating the probability that the target organ part is imaged is calculated, in the first modification, a plurality of index values are calculated according to each determination result, and the target object is included in the series of in-vivo image groups. The probability that the organ part of this is imaged is shown.

  FIG. 12 is a flowchart illustrating a procedure of index value calculation processing performed by the calculation unit 50 and the presence / absence determination unit 52 in the case of the first modification. As shown in FIG. 12, first, the presence / absence determining unit 52 determines whether or not the imaging time of the organ part corresponding to the target section is equal to or longer than a predetermined threshold time (step S231). When the imaging time is equal to or longer than the threshold time (step S231: Yes), the calculation unit 50 sets the index value indicating the presence or absence of the in-vivo image captured in the attention interval to 1.0 (step S232). On the other hand, when the imaging time is less than the threshold time (step S231: No), the calculation unit 50 sets the index value indicating the presence or absence of the in-vivo image captured in the attention section to 0.0 (step S233).

  Thereafter, the presence / absence determining unit 52 determines whether or not the imaging time of the organ part above the target section is within a predetermined threshold range (step S234). When the imaging time is within the threshold range (step S234: Yes), the calculation unit 50 sets the index value indicating whether or not the imaging is started from the start point portion of the attention section to 1.0 (step S235). On the other hand, when the imaging time is outside the threshold time range (step S234: No), the calculation unit 50 sets 0.0 as an index value indicating whether or not the imaging is started from the start point portion of the target section (step S236). .

  Further, the presence / absence determining unit 52 determines whether or not the imaging time of the organ part below the target section is within a predetermined threshold range (step S237). When the imaging time is within the threshold range (step S237: Yes), the calculation unit 50 sets the index value indicating whether or not the imaging has been performed up to the end point portion of the attention section (step S238). On the other hand, when the imaging time is out of the threshold time range (step S237: No), the calculation unit 50 sets the index value indicating whether or not the image has been captured up to the end point portion of the section of interest, and returns to step S130. (Step S239).

  The image processing apparatus according to the first modification calculates and displays a plurality of index values as index values indicating the probability that an organ site of interest is imaged in a series of in-vivo image groups. Therefore, according to the first modification, the observer can grasp in detail the imaging state of the target organ site before observing a series of in-vivo image groups. The burden on the observer can be reduced when observing the image.

(Modification 2)
Further, in the above-described embodiment, referring to the results of determining whether or not the organ part in the upper part of the target section or the organ part in the lower part of the target section has been imaged, the target in the series of in-vivo image groups is referred to. The index value indicating the probability that the organ site is imaged is calculated to be 0.0 or 1.0. However, in the second modification, the numerical values corresponding to the respective determination results are added together, and a series of in-vivo image groups. An index value indicating the probability that the organ part of interest is being imaged is calculated.

  FIG. 13 is a flowchart illustrating a procedure of index value calculation processing performed by the calculation unit 50 and the presence / absence determination unit 52 in the case of the second modification. As shown in FIG. 13, first, the presence / absence determination unit 52 determines whether or not the imaging time of the organ part corresponding to the target section is equal to or longer than a predetermined threshold time (step S331). When the imaging time is equal to or longer than the threshold time (step S331: Yes), the calculation unit 50 adds, for example, 0.6 as an index value (step S332). On the other hand, when the imaging time is less than the threshold time (step S331: No), the calculation unit 50 does not add to the index value.

  Thereafter, the presence / absence determination unit 52 determines whether or not the imaging time of the organ part above the target section is within a predetermined threshold range (step S333). When the imaging time is within the threshold range (step S333: Yes), the calculation unit 50 adds 0.2 as an index value (step S334). On the other hand, when the imaging time is outside the threshold time range (step S333: No), the calculation unit 50 does not add to the index value.

  Further, the presence / absence determining unit 52 determines whether or not the imaging time of the organ part below the target section is within a predetermined threshold range (step S335). When the imaging time is within the threshold range (step S335: Yes), the calculation unit 50 adds, for example, 0.2 as an index value (step S336). On the other hand, when the imaging time is outside the threshold time range (step S335: No), the calculation unit 50 does not add to the index value. Thereafter, the calculation unit 50 adds the numerical values added in steps S331 to S336, calculates an index value indicating the probability that the target organ part is captured in a series of in-vivo image groups, and returns to step S130 ( Step S337).

  Compared with the embodiment, the image processing apparatus according to the second modification uses a numerical value that more reflects the imaging state as an index value indicating the probability that the organ part of interest is captured in a series of in-vivo image groups. calculate. Therefore, according to the second modification, the observer can grasp the imaging state of the organ site of interest before observing a series of in-vivo image groups, and thus observes the body using a capsule endoscope. In this case, the burden on the observer can be reduced.

(Modification 3)
By the way, in the above-described embodiment, the average value of the area ratio of the region of interest is calculated as an index indicating the probability that the region of interest is imaged. In the third modification, the probability that the region of interest is imaged is shown. As the index value, the ratio of the number of in-vivo images in which the area ratio of the region of interest is equal to or greater than the threshold to the number of in-vivo images obtained by capturing the entire series of in-vivo image groups or the target section is calculated.

  For example, as illustrated in FIG. 14, the calculation unit 50 calculates the area ratio of the region of interest (the number of pixels of the region of interest) with respect to the entire image (the total number of pixels) for the in-vivo image obtained by capturing the section of interest (step S261). . Thereafter, the calculation unit 50 calculates the ratio of the number of images in which the area ratio of the region of interest is equal to or greater than the threshold to the number of images of the entire region of interest as the ratio of the region of interest captured, and the ratio of the number of images is calculated by the region of interest. The index value indicating the probability of being imaged is set, and the process returns to step S160 (step S262).

  The calculation unit 50 calculates the ratio of the number of images in which the area ratio of the region of interest is equal to or greater than the threshold to the total number of images in the series of in-vivo images as an index value indicating the probability that the region of interest is captured. Also good. Further, the calculation unit 50 may calculate both the average value of the area ratio of the region of interest and the number ratio of images in which the area ratio of the region of interest is equal to or greater than a threshold value as the index value.

  According to the image processing apparatus according to the modified example 3, before an observer observes a series of in-vivo images, an in-vivo image in which a region of interest is captured is an in-vivo image in which a series of in-vivo images or a section of interest is captured. Since it can be grasped how much is contained, the burden on the observer can be reduced when the inside of the body is observed using a capsule endoscope.

(Modification 4)
Further, in the above-described embodiment, the final index value is calculated by summarizing whether or not there is an in-vivo image showing an in-vivo region that is an organ part and the ratio of the in-vivo region that is a mucosal surface. However, as a modification, the final index value is calculated depending on whether there is an in-vivo image that shows the internal region that is an organ part or the proportion of the internal region that is a mucosal surface. May be.

  By the way, in the above-described embodiment, the specifying unit 53 determines the organ site captured in each in-vivo image using the file size after DCT encoding of the in-vivo image and the change amount of the file size. It is also possible to perform the organ determination process using information relating to the frequency when is converted into a frequency space. This is because, when an in-vivo image is converted into a frequency space, the distribution of frequency components differs depending on the organ site imaged in the image. Examples of the information relating to the frequency include a power spectrum calculated by Fourier transform, a DCT coefficient calculated at the time of decoding after DCT encoding, and the like. Since the DCT coefficient is calculated by the in-vivo image decoding process, the calculation amount of the organ determination process itself is small, and the organ determination result can be obtained quickly. Hereinafter, an organ determination process using a DCT coefficient will be described.

  As shown in FIG. 15, in the process of step S120, the specifying unit 53 first calculates a feature vector of each in-vivo image based on the DCT coefficient (step S221). Next, the specifying unit 53 reads reference data (step S222). After that, the specifying unit 53 determines the organ site captured in each in-vivo image based on the feature vector and the reference data, and returns to step S120 (step S223).

  Here, the feature vector is a vector whose element is a feature amount based on the average value of DCT coefficients calculated for each in-vivo image. The average value of DCT coefficients is a set of average DCT1 to average DCT64 calculated for each of DCT1 to DCT64 using the DCT coefficients of all pixel blocks on the image for each in-vivo image. The feature vector is calculated based on the DCT coefficient for each of the Y value, U value, and V value, and is a 192-dimensional vector at the maximum.

  In the present embodiment, a three-dimensional feature vector will be described as an example of the feature vector of each in-vivo image. In the three-dimensional feature vector according to the present embodiment, feature amounts A, B, and C calculated based on the DCT coefficient for the Y value are used as vector elements. Here, the feature amount A is a representative DCT coefficient calculated by selecting a plurality of average DCT coefficients indicating low frequency components from the average DCT2 to the average DCT64 for the Y value of each in-vivo image and averaging them. is there. The feature amount B is a representative DCT coefficient calculated by selecting a plurality of average DCT coefficients indicating high-frequency components from the average DCT2 to the average DCT64 for the Y value of each in-vivo image and averaging them. The feature amount C is an average DCT1 that is an average value of DC components for the Y value of each in-vivo image.

  FIG. 16 is a diagram illustrating an example of the reference data. The reference data is dictionary data in which feature vectors of in-vivo images are classified in advance for each organ site being imaged, and stored in the storage unit 30 in advance. As shown in FIG. 16, in the image obtained by imaging the stomach, the feature amount A indicating the low frequency component is larger than the image obtained by imaging the small intestine or the large intestine. On the other hand, in an image obtained by imaging the small intestine or the large intestine, a feature amount B indicating a high frequency component is larger than that in an image obtained by imaging the stomach. Therefore, as shown in FIG. 16, feature vectors can be classified in the feature space for each of the imaged organ parts. Using this property, the specifying unit 53 determines the organ site captured in each in-vivo image by a known identification method such as the kNN method or the subspace method based on the feature vector and reference data of each in-vivo image. To do.

  Up to this point, the best mode for carrying out the present invention has been described as the embodiment and the first to fourth modified examples. However, the present invention is not limited thereto, and various modifications are possible within the scope of the present invention. It is.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. 3 is a flowchart illustrating an image processing procedure performed by the image processing apparatus illustrated in FIG. 1. It is a figure which shows transition of the change of the file size with respect to imaging time. It is a figure which shows transition of the change of the variation | change_quantity of the file size with respect to imaging time. It is a flowchart which shows the procedure of the process which specifies the organ site | part currently imaged by each in-vivo image based on the file size of each in-vivo image, and its variation | change_quantity. It is a flowchart which shows the procedure of the calculation process of the index value which shows the probability that the organ site | part of attention object is imaged. It is a flowchart which shows the procedure of the process which specifies the in-vivo image which imaged the region of interest of the attention area. It is a figure which shows the DCT coefficient of an 8x8 pixel block. It is a flowchart which shows the procedure which divides | segments each in-vivo image into an image part. It is a flowchart which shows the procedure which calculates the index value which shows the probability that the region of interest of the attention area is imaged. It is a figure which shows an example of the display screen displayed on the output part shown in FIG. It is a flowchart which shows the procedure of the calculation process of the index value of the image processing apparatus concerning the modification 1 of embodiment. It is a flowchart which shows the procedure of the calculation process of the index value of the image processing apparatus concerning the modification 2 of embodiment. 14 is a flowchart illustrating a procedure of index value calculation processing of the image processing apparatus according to the third modification of the embodiment. It is a flowchart which shows the procedure of the process which determines the organ site | part imaged on the in-vivo image based on a DCT coefficient. It is a figure which shows an example of reference | standard data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Input part 11 Image acquisition part 20 Output part 21 Index value display part 22 Warning notification part 30 Storage part 31 In-vivo image storage part 40 Control part 50 Calculation part 51 Section setting part 52 Existence determination part 53 Identification part 54 Correspondence Attaching unit 55 Measuring unit 56 Region of interest setting unit 57 Dividing unit 58 Selecting unit 59 In-vivo image specifying unit 60 Display screen 61 In-vivo image 62 Play button 63 Reverse playback button 64 Position display 65, 66, 67, 68 Index value display 69 Warning display

Claims (26)

An image acquisition unit that acquires a series of in-vivo images formed by a plurality of in-vivo images obtained by repeatedly imaging while moving in the body;
Using the in-vivo image acquired by the image acquisition unit, a calculation unit that calculates an index value indicating the probability that the in-vivo region of interest is copied in the series of in-vivo image groups;
An image processing apparatus comprising:   The probability that the in-vivo region of the target object is copied in the series of in-vivo image groups is the in-vivo image obtained by imaging the organ part of the target object in the series of in-vivo image groups. The image processing apparatus according to claim 1, wherein the image processing apparatus is present.   The calculation unit includes a presence / absence determination unit that determines whether or not an in-vivo image representing the organ site of interest exists in the series of in-vivo image groups, and the result of the presence / absence determination is obtained. The image processing apparatus according to claim 2, wherein the index value is calculated with reference to the image processing apparatus. The calculation unit includes an interval setting unit that sets an attention interval that is an interval including the organ site of interest on a movement route in the body that passes when repeatedly capturing in-vivo images.
The image processing apparatus according to claim 3, wherein the presence / absence determination unit performs the presence / absence determination on an in-vivo image obtained by copying the attention section. The presence / absence determination unit performs the presence / absence determination based on whether the imaging time of the attention section is equal to or greater than a predetermined threshold,
The image processing apparatus according to claim 4, wherein the calculation unit calculates the index value with reference to a result of the existence determination. The presence / absence determination unit performs the presence / absence determination based on whether the start point portion and / or the end point portion of the section of interest are imaged,
The image processing apparatus according to claim 4, wherein the calculation unit calculates the index value with reference to a result of the existence determination.   The start point portion and / or the end point portion of the attention section are determined according to whether or not an organ part located in front of or behind the attention section is normally imaged. Item 7. The image processing apparatus according to Item 6.   Whether or not the organ part located in front of or behind the target section has been normally imaged depends on whether the imaging time of the organ part located in front of or behind the target section is within a predetermined threshold range. The image processing apparatus according to claim 7, wherein the image processing apparatus is defined. The presence / absence determination unit includes:
A specific unit that refers to a feature amount indicating a feature of each in-vivo image that constitutes the series of in-vivo images, and that identifies an organ site that is captured in each in-vivo image for each in-vivo image;
An associating unit that associates, for each in-vivo image, an organ site identified by the identifying unit and an in-vivo image obtained by copying the organ site;
A measurement unit that measures an imaging time, which is a time during which an in-vivo image associated with the same organ part is imaged, as the imaging time of the organ part;
The image processing apparatus according to claim 5, further comprising:   The measuring unit calculates a difference between a time when the first in-vivo image is captured and a time when the last in-vivo image is captured among a plurality of in-vivo images associated with the same organ site, The image processing apparatus according to claim 9, wherein the image processing time is measured.   The calculation unit includes a result of the presence / absence determination based on whether or not an imaging time of the target organ part included in the attention section is equal to or greater than a predetermined threshold, and a start point portion and / or an end point portion of the attention section. The image processing apparatus according to claim 4, wherein the index value is calculated with reference to a result of the presence / absence determination based on whether or not an image is captured.   The calculation unit captures the index value calculated based on the presence / absence determination based on whether the imaging time of the section of interest is equal to or greater than a predetermined threshold, and the start point and / or end point of the section The image processing apparatus according to claim 4, wherein the index value is calculated by adding the index value calculated based on the presence / absence determination based on whether or not the determination is made.   The probability that the in-vivo region of the target object is copied in the series of in-vivo image groups is the image portion that represents the region of interest that is the in-vivo region of the target object in the entire in-vivo images constituting the series of in-vivo image groups. The image processing apparatus according to claim 1, wherein the image processing apparatus is a ratio in which the image data is present.   The calculating unit occupies a ratio of an image part representing the region of interest in the whole in-vivo images constituting the series of in-vivo images, and / or an image part representing other than the region of interest constitutes the series of in-vivo images. The image processing apparatus according to claim 13, wherein the index value is calculated according to a ratio of the whole in-vivo image. The calculation unit includes:
An in-vivo image specifying unit that specifies an in-vivo image in which a region of interest that is the in-vivo region of the target of interest included in the attention section is selected from among a plurality of in-vivo images constituting the series of in-vivo images;
Further, the ratio of the image portion representing the region of interest in the entire in-vivo image specified by the in-vivo image specifying unit and / or the image portion representing the in-vivo region other than the region of interest specified by the in-vivo image specifying unit The image processing apparatus according to claim 4, wherein the index value is calculated according to a ratio occupied in the entire in-vivo image. The calculation unit includes:
A dividing unit that divides each in-vivo image constituting the series of in-vivo images into a plurality of the image portions;
A selection unit for selecting an image part representing the region of interest from the plurality of image parts;
The image processing apparatus according to claim 13, further comprising:   The selection unit calculates a feature amount indicating a feature of each image portion for each of the plurality of image portions, refers to the calculated feature amount, and selects an image portion representing the region of interest. The image processing apparatus according to claim 16.   The image processing apparatus according to claim 13, wherein the region of interest is a surface of a mucous membrane. The calculation unit includes a specifying unit that specifies an organ part that is captured in each in-vivo image based on a feature amount that indicates a feature of the in-vivo image.
The presence / absence determination unit compares the organ part identified as captured in each in-vivo image by the specifying unit with the target organ part, and the in-vivo image showing the target organ part is the series of in-vivo images. The image processing apparatus according to claim 3, wherein it is determined whether or not the image group exists. The calculation unit further includes an association unit for associating the specified organ part with an in-vivo image obtained by copying the specified organ part,
The presence / absence determination unit compares, for each in-vivo image, whether or not an organ part associated with each of the plurality of in-vivo images matches an organ part of interest, and the associated organ part The image processing apparatus according to claim 19, wherein when it matches the target organ part, it is determined that an in-vivo image representing the target organ part is present in the series of in-vivo image groups.   The image processing apparatus according to claim 1, further comprising a display unit that displays the index value via a display.   The image processing apparatus according to any one of claims 1 to 21, further comprising a warning notification unit that notifies a warning when the index value is equal to or less than a predetermined threshold value.   The image processing apparatus according to any one of claims 1 to 22, wherein the in-vivo image is an image captured by a capsule endoscope. An image acquisition unit that acquires a series of in-vivo images formed by a plurality of in-vivo images obtained by repeatedly imaging while moving in the body;
A display unit that displays an index value that indicates a probability that an in-vivo region of interest is copied in the series of in-vivo image groups, calculated using the in-vivo image acquired by the image acquisition unit;
An image processing apparatus comprising: An image acquisition step of acquiring a series of in-vivo image groups composed of a plurality of in-vivo images obtained by repeatedly imaging while moving in the body;
Using the in-vivo image acquired in the image acquisition step, a calculation step for calculating an index value indicating the probability that the in-vivo region of interest is copied in the series of in-vivo image groups;
An image processing method comprising: An image acquisition step of acquiring a series of in-vivo image groups composed of a plurality of in-vivo images obtained by repeatedly imaging while moving in the body;
Using the in-vivo image acquired in the image acquisition step, a calculation step for calculating an index value indicating the probability that the in-vivo region of interest is copied in the series of in-vivo image groups;
An image processing program for causing a computer to exhibit the above.

Images