JP4616076B2 - Image display device - Google Patents

Description

  The present invention relates to an image display device that sequentially displays a series of input images, and more particularly to an image display device suitable for application to display of a series of images obtained by imaging the inside of a subject using a capsule endoscope. It is.

  In recent years, swallowable capsule endoscopes have been developed in the field of endoscopes. This capsule endoscope has an imaging function and a wireless communication function. After being swallowed from the patient's mouth for observation in various organs, until it is naturally discharged from the human body, for example, the stomach, the small intestine, Images are sequentially taken while moving inside the digestive organs such as the large intestine according to the peristaltic movement.

  While moving inside the organ, image data captured inside the body by the capsule endoscope is sequentially transmitted outside the body by wireless communication and stored in a memory provided in the receiver outside the body or provided in the receiver. The image is displayed on the displayed display. A doctor, nurse, or the like can make a diagnosis based on the image displayed on the display based on the image data stored in the memory or the image displayed on the display provided in the receiver along with the reception.

  Usually, the series of images captured by the capsule endoscope is enormous, and doctors, nurses, and the like require a great deal of time and effort to observe and diagnose this series of images. Correspondingly, when an image is displayed based on the image data stored in the memory for observation, the degree of similarity between the two images is determined, and the image display rate is determined based on the determination result. There has been proposed a display device that changes the angle (see, for example, Patent Document 1). In this display device, focusing on the fact that there are many similar images that are continuously captured when the movement of the capsule endoscope is stagnant, when the similarity between the two images is low, the image is displayed at a low display rate. If the similarity is high, an image is displayed at a high display rate.

JP-T-2004-521626

  However, in such a display device, an image having high similarity and low necessity for observation is displayed at a high display rate. For example, this is an image obtained by imaging a portion that is not an observation target, and needs to be observed. There is a problem that even if there are no images, the images are sequentially displayed, and extra time and labor may be required to observe a series of images.

  The present invention has been made in view of the above, and provides an image display device that can display only an image obtained by imaging a desired observation target region and can efficiently observe a series of images. The purpose is to do.

  In order to achieve the above object, an image display device according to a first aspect of the present invention is an image display device that sequentially displays a series of inputted images, wherein predetermined characteristics are selected from the images included in the series of images. Area detecting means for detecting the feature image area having the feature image area, and calculating the occupancy rate of the feature image area with respect to the entire image area of each image; and each feature image having the feature image area in accordance with the calculated occupancy ratio The series of images are classified into one or more image groups based on the image detection means for detecting from the series of images and the number of consecutive images in the time series of the feature images detected by the image detection means. Image classification means, and display control means for performing control to sequentially display each image included in at least one image group designated in advance from each image group classified by the image classification means, And said that there were pictures.

  According to a second aspect of the present invention, in the above invention, the image classification unit classifies the feature images that are continuous for a predetermined number of continuous images or more into the same image group.

  According to a third aspect of the present invention, in the image display device according to the third aspect, the image classification unit is an image that is an image that is earlier in time series than each feature image that continues for the predetermined number of consecutive images or more, and is not a feature image. Each image is classified into an image group different from the image group into which each feature image is classified.

  According to a fourth aspect of the present invention, in the above invention, the predetermined feature includes a main feature and a sub-feature different from the main feature. A main feature image area having the main feature and a sub-feature image area having the sub-feature, and the image detection means detects the main feature occupancy ratio of the main feature image area with respect to the entire image area of each image And calculating a sub-feature occupancy ratio of the sub-feature image area relative to the main feature image area of each image, and determining the main feature image area according to the calculated main feature occupancy ratio and sub-feature occupancy ratio. Each feature image is detected from the series of images.

  According to a fifth aspect of the present invention, in the image display apparatus according to the fifth aspect, the predetermined feature has a plurality of different features, and the region detection unit has a feature image having each feature from the images. Detecting a region, and the image detecting means calculates the occupancy rate of the feature image region having the respective features of the respective images, and calculates the feature images for each of the plurality of features according to the calculated occupancy rates. And the image classifying means classifies the series of images into an image group for each of the plurality of features based on the number of continuous images for each of the plurality of features.

  According to a sixth aspect of the present invention, in the above invention, the image display device further comprises input means for receiving designation information for designating at least one image group from the image groups, and the display control means includes the designation information. Control is performed to sequentially display each image included in the image group designated by.

  Further, in the image display device according to claim 7, in the above invention, each of the images is an image of the inside of a digestive organ, and the predetermined feature is an intestine that specifies at least one of a small intestine and a large intestine. The image classification means classifies the series of images into an image group obtained by imaging at least one of the small intestine and the large intestine.

  Further, in the image display device according to claim 8, in the above invention, each of the images is an image of the inside of a stomach and a small intestine, and the predetermined feature is an intestinal feature that identifies the small intestine, The image classification means classifies each image before the image obtained by imaging the small intestine in time series as an image group obtained by imaging the stomach.

  According to a ninth aspect of the present invention, in the above invention, the intestinal characteristic is at least one of a mucosa, a villi on the surface of the mucosa, and feces.

  According to a tenth aspect of the present invention, in the above invention, the series of images are generated using a capsule endoscope.

  According to the image display device of the present invention, it is possible to display only an image obtained by imaging a desired observation target part, and it is possible to efficiently observe a series of images.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of an image display device according to the invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments. In the description of the drawings, the same parts are denoted by the same reference numerals.

(Embodiment 1)
First, the image display apparatus according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing the configuration of the image display apparatus 1 according to the first embodiment. As shown in FIG. 1, the image display device 1 includes an image processing unit 2 that processes an image stored in a storage unit 5, an input unit 3 that receives input of various information, and a display unit 4 that displays various information. A storage unit 5 that stores various types of information; and a control unit 6 that controls processing and operation of each unit of the image display device 1. The image processing unit 2, the input unit 3, the display unit 4, and the storage unit 5 are electrically connected to the control unit 6.

  The image processing unit 2 includes an image processing control unit 2a, a region detection unit 2b, a feature image detection unit 2c, and an image classification unit 2d. The image processing control unit 2a sequentially acquires a series of images stored in the storage unit 5, controls various image processing for each acquired image, and outputs and stores each image of the processing result in the storage unit 5. In particular, the image processing control unit 2a controls the region detection unit 2b, the feature image detection unit 2c, and the image classification unit 2d to execute predetermined image processing.

  The area detection unit 2b detects a feature image area that is an image area having a predetermined feature from each image acquired by the image processing control unit 2a. At this time, the region detection unit 2b detects a feature image region by identifying a predetermined feature based on texture or color information in the image. When there are a plurality of features to be detected, the region detection unit 2b detects a feature image region having each feature. Note that the region detection unit 2b may detect a feature image region based on various feature amounts such as a contour shape of a subject in the image and a density gradient, for example. Alternatively, the area detection unit 2b may extract an area corresponding to a predetermined feature after the entire image is divided into areas each having a certain characteristic in advance.

  The feature image detection unit 2c calculates, for each image acquired by the image processing control unit 2a, the occupancy rate of the feature image region detected by the region detection unit 2b with respect to the entire image region, and according to the calculated occupancy rate, the feature image detection unit 2c Each feature image having an image region is detected from a series of images. Specifically, the feature image detection unit 2c refers to a threshold value with respect to the occupation rate input in advance, detects the feature image according to the magnitude relationship between the threshold value and the calculated occupation rate, and detects the feature image. Feature image information indicating a feature image is added to the feature image. The feature image detection unit 2c calculates the occupancy rate of each feature image region corresponding to a plurality of features, and at least 1 is calculated based on the sum, difference, maximum value, minimum value, and various statistics of the calculated occupancy rates. The feature image information may be added according to one occupation ratio.

  The image classifying unit 2d classifies a series of images into one or more image groups based on the number of feature images detected by the feature image detecting unit 2c in time series. Specifically, the image classification unit 2d classifies the consecutive feature images as the same image group when the feature images are continuous for a predetermined number of images or more. At this time, the image classifying unit 2d sets the feature images until the number of feature images stops to a predetermined number or more as the same image group. In addition, the image classification unit 2d classifies each image that is a previous image and not a feature image in time series before the feature image classified in this way into an image group different from the image group in which the feature image is classified. .

  The input unit 3 receives input of an image to be processed by the image display device 1 and various processing information. Specifically, the input unit 3 includes a communication interface such as USB or IEEE 1394, and accepts input of an image from an external device. The input unit 3 includes various switches, input keys, a mouse, a touch panel, and the like. Various processing information such as a threshold value referred to by the feature image detection unit 2c and image group designation information referred to by the image display control unit 6a. Accepts input. The input unit 3 may include an interface corresponding to a portable storage medium such as a compact flash (registered trademark), and may accept an input of an image or the like from the portable storage medium.

  The display unit 4 includes a liquid crystal display and displays various information including images. In particular, the display unit 4 displays a series of images stored in the storage unit 5 and a GUI (Graphical User Interface) screen that requests the operator of the image display device 1 to input various processing information.

  The storage unit 5 is realized by a ROM that stores various processing programs and the like in advance, and a RAM that stores processing parameters, processing data, and the like of each processing. In particular, the storage unit 5 includes an image storage unit 5a as a storage area for storing an image input from the outside and an image processed by the image processing unit 2. Note that the storage unit 5 may include a portable storage medium such as a compact flash (registered trademark) as a removable image storage unit.

  The control unit 6 is realized by a CPU or the like that executes various processing programs stored in the storage unit 5. The control unit 6 includes an image display control unit 6a, and the image display control unit 6a is previously selected from among a series of images stored in the image storage unit 5a from among the image groups classified by the image classification unit 2d. Control is performed to sequentially display each image included in the designated at least one image group on the display unit 4. Specifically, the image display control unit 6 a performs control to sequentially display each image included in the image group designated by the designation information input from the input unit 3.

  Here, processing and operations performed by the image display apparatus 1 will be described. FIG. 2 is a schematic diagram illustrating an outline of processing in which the image display device 1 classifies and displays a series of images stored in the image storage unit 5a. FIG. 2 illustrates an outline of a process for classifying and displaying a series of images generated by imaging the inside of the stomach, small intestine, and large intestine of a subject using a capsule endoscope (not shown).

  As shown in FIG. 2, the image classification unit 2d classifies a series of images stored in the image storage unit 5a before processing into a stomach image group, a small intestine image group, and a large intestine image group. At this time, in images 1 to N, which are a series of images before processing, it is unknown from where to where the stomach, small intestine, and large intestine are imaged. In the following, “m” and “n” indicate image numbers, respectively, and the relationship of 1 ≦ n ≦ m ≦ N is satisfied in the example of FIG.

  Therefore, first, the region detection unit 2b detects a villi region, which is an image region having villi as an intestinal feature that identifies the small intestine, from each image included in a series of images before processing. Next, the feature image detection unit 2c calculates the occupation rate of the detected villi region, and has the villi region detected as the feature image in an image in which the calculated occupation rate is larger than a threshold value input in advance. Villi detection information indicating a villi image is added. Then, the image classification unit 2d refers to the villi detection information of each image, and after the villi images that are first continuous for a predetermined number of images or more, the small intestine images 1 to (mn) obtained by imaging the small intestine are the same small intestine. Classify into image groups.

  Similarly, the region detection unit 2b detects a stool region that is an image region having stool as an intestinal feature that identifies the large intestine from each image, and the feature image detection unit 2c detects the detected stool region. The occupancy rate is calculated, and flight detection information indicating that the calculated occupancy rate is larger than a threshold value input in advance is a flight image having a flight area detected as a feature image. The image classification unit 2d refers to the stool detection information of each image, and after the stool images that are consecutive for the number of images at first, the large intestine images 1 to (Nm) obtained by imaging the large intestine. Classify into:

  Further, the image classification unit 2d classifies the images in time series earlier than the small intestine images classified into the small intestine image group into the same stomach image group as the stomach images 1 to n obtained by imaging the stomach. Then, the image display control unit 6a sequentially displays each image included in at least one image group of the stomach image group, the small intestine image group, and the large intestine image group designated by the designation information input in advance.

  In this way, the image display apparatus 1 can classify a series of images obtained by imaging the stomach, the small intestine, and the large intestine into the same image group for each image obtained by imaging each part, and input designations. Only the images included in the image group specified by the information can be displayed.

  Next, a processing procedure performed by the image display device 1 will be described. FIG. 3 is a flowchart illustrating a processing procedure in which the image display apparatus 1 processes and displays a series of images stored in the image storage unit 5 a under the control of the control unit 6. This flowchart exemplifies a processing procedure for classifying and displaying a series of images into a stomach image group, a small intestine image group, and a large intestine image group as shown in FIG.

  As shown in FIG. 3, the area detection unit 2b and the feature image detection unit 2c detect the villi region from each image, calculate the occupation rate of the detected villi region, and perform a series of operations according to the calculated occupation rate. A villi image detection process for detecting a villi image from the image is performed (step S101), and the image classification unit 2d classifies the villi images subsequent to the predetermined number of images or more into small intestine images as a small intestine image group. Processing is performed (step S103).

  In addition, the region detection unit 2b and the feature image detection unit 2c detect a stool region from each image, calculate an occupancy rate of the detected stool region, and stool from a series of images according to the calculated occupancy rate. A stool image detection process for detecting an image is performed (step S105), and the image classification unit 2d performs a large intestine image classification process for classifying a stool image subsequent to a predetermined number of images or more into a large intestine image group as a large intestine image (step S105). S107).

  Further, the image classification unit 2d performs a stomach image classification process for classifying each image before the small intestine image group in time series as a stomach image into a stomach image group (step S109), and the image display control unit 6a inputs in advance. The image display process for sequentially displaying the images included in the image group designated by the designated information is performed (step S111), and the control unit 6 ends the series of processes.

  Next, the villi image detection process in step S101 shown in FIG. 3 will be described. FIG. 4 is a flowchart showing a processing procedure of villi image detection processing. As shown in FIG. 4, the image processing control unit 2a reads the first image in time series among the series of images stored in the image storage unit 5a (step S121), and the area detection unit 2b reads the read image. A villus region indicating villi on the surface of the mucous membrane is detected as a feature image region (step S123).

  Thereafter, the feature image detection unit 2c calculates the occupation ratio of the detected villi area with respect to the entire image area (step S125), and determines whether the calculated occupation ratio is greater than a threshold value input in advance. If the occupation ratio is larger than the threshold value (step S127: Yes), villus detection information indicating that the image is a villus image is added to the image to be processed (step S129).

  Then, the image processing control unit 2a records the image to which the villi image information is added in the image storage unit 5a (step S131), determines whether or not a series of all images has been processed (step S133), When all the images have not been processed (step S133: No), the processing from step S121 is repeated for the unprocessed images, and when all the images have been processed (step S133: Yes), step S101. Return to Note that the image processing control unit 2a immediately determines in step S133 when it is determined in step S127 that the occupancy rate of the villi region is equal to or less than the threshold value (step S127: No).

  In step S123, the region detection unit 2b detects a villi region from the processing target image by texture analysis. That is, the region detection unit 2b utilizes the fact that each of the villuss is arranged almost periodically, and Fourier-transforms the image, thereby converting the image region having a spatial frequency component corresponding to the villus cycle into the villus region. Detect as.

  At this time, for example, the area detection unit 2b divides the entire image area into small areas in advance, and performs a Fourier transform on each divided area to detect a villi area. In step S125, the feature image detection unit 2c associates whether or not a villi region is detected with respect to each small region, and the villi region is determined according to the number of small regions associated with the detection of the villi region. Calculate the occupancy ratio. Alternatively, the villi region may be detected from the color information.

  Next, the small intestine image classification process in step S103 shown in FIG. 3 will be described. FIG. 5 is a flowchart showing a processing procedure of small intestine image classification processing. As shown in FIG. 5, the image processing control unit 2a reads the first image in time series among the series of images stored in the image storage unit 5a (step S141), and the image classification unit 2d reads the read image. It is determined whether or not villus detection information is added (step S143).

  When the villi detection information is added (step S143: Yes), the image classification unit 2d counts the number of times the villi images are read (step S145), and whether or not the villi images are read continuously for a predetermined number of images. Is determined (step S147).

  When the predetermined number of images have not been read continuously (step S147: No), the image processing control unit 2a determines whether or not a series of all images has been processed (step S149), and all the images are processed. If YES in step S149, the process returns to step S103, and if all the images have not been processed (step S149: NO), the control from step S141 is repeated for the unprocessed image. .

  Note that the image processing control unit 2a also determines whether or not all the series of images have been processed in step S149 even when it is determined in step S143 that the villi detection information has not been added (step S143: No). to decide.

  On the other hand, when the villi images are read continuously for the predetermined number of images (step S147: Yes), the image processing control unit 2a rereads the images at the time point that goes back in time series by the predetermined number of images (step S151). ), The image classification unit 2d determines whether or not the villi detection information is added (step S153). When the villi detection information is added (step S153: Yes), the image classification unit 2d adds small intestine image information indicating that it is a small intestine image to the processing target image (step S155).

  Then, the image processing control unit 2a records the image to which the small intestine image information is added in the image storage unit 5a (step S157), determines whether or not a series of all the images has been processed (step S159), When all the images are not processed (step S159: No), the processing from step S151 is repeated for the unprocessed images, and when all the images are processed (step S159: Yes), step S103 is performed. Return to

  If it is determined in step S153 that the villi detection information is not added (step S153: No), the image classification unit 2d counts the number of non-reads that is the number of times that the villi image cannot be read (step S161). Then, it is determined whether or not the reading of the villi images has stopped for a predetermined number of images (step S163). Then, if the predetermined number of images has not been interrupted (step S163: No), the image processing control unit 2a performs the determination of step S159. If the predetermined number of images has been interrupted (step S163: Yes), the process returns to step S103. To do.

  In this way, in the small intestine image classification process, the reading of the villi image is continued for a predetermined number of images to identify the image from which imaging of the small intestine is started, and the reading of the villi image is interrupted for a predetermined number of images. Then, it is possible to identify an image for which imaging of the small intestine has been completed, add small intestine image information to each image between them, and classify each image as a small intestine image into the same small intestine image group.

  In step S145, the image classification unit 2d includes, for example, a counter that counts the number of readings of the villi image, and may detect the number of continuous readings by incrementing the counter each time the villi image is read, step S161. Then, similarly, the number of consecutive non-reads may be detected by incrementing the counter. In this case, the image classification unit 2d may reset these counters when the reading number or the non-reading number stops or when a predetermined number of images continues.

  Further, in step S151 immediately after it is determined in step S147 that the predetermined number of villi images have continued, the image processing control unit 2a does not necessarily reread the image at the time point back in time series by the predetermined number of images. There may be no need, and an image subsequent to an already read image may be newly read.

  In step S141, the image processing control unit 2a reads a plurality of images having a predetermined number of images at the same time, and in step S143, the image classification unit 2d adds villus detection information to each read image. By determining whether or not, the head of the small intestine image may be specified.

  Next, the stool image detection process in step S105 shown in FIG. 3 will be described. FIG. 6 is a flowchart showing the processing procedure of the fecal image detection process. In the stool image detection process shown in FIG. 6, a stool image is detected by the same processing procedure as the villus image detection process shown in FIG. 4 by detecting a stool area instead of the villi area.

  As shown in FIG. 6, the image processing control unit 2a reads the first image in time series from the series of images stored in the image storage unit 5a (step S171), and the area detection unit 2b reads the read image. A stool area indicating a stool is detected as a feature image area from among (step S173). The feature image detection unit 2c calculates the occupation ratio of the detected villi area with respect to the entire image area (step S175), and determines whether the calculated occupation ratio is larger than a threshold value input in advance. (Step S177) When the occupation ratio is larger than the threshold value (Step S177: Yes), the flight detection information indicating that the image is a flight image is added to the processing target image (Step S179).

  Then, the image processing control unit 2a records the image to which the stool detection information is added in the image storage unit 5a (step S181), determines whether or not a series of all the images has been processed (step S183), When all the images are not processed (step S183: No), the processing from step S171 is repeated for the unprocessed images, and when all the images are processed (step S183: Yes), step S105 is performed. Return to If it is determined in step S177 that the occupancy rate of the stool area is equal to or less than the threshold value (step S177: No), the image processing control unit 2a immediately determines in step S133.

  In step S171, the image processing control unit 2a may read the subsequent images classified into the small intestine image group by the small intestine image classification process. In step S173, the area detection unit 2b detects an image area indicating a brown color as a fecal area.

  Next, the large intestine image classification process in step S107 shown in FIG. 3 will be described. FIG. 7 is a flowchart showing a processing procedure of large intestine image classification processing. In the large intestine image classification process shown in FIG. 7, the colon image is classified by the same processing procedure as the small intestine image classification process shown in FIG. 5 by counting the number of consecutive stool images read instead of the villi image. ing.

  As shown in FIG. 7, the image processing control unit 2a reads the first image in time series among the series of images stored in the image storage unit 5a (step S191), and the image classification unit 2d reads the read image. It is determined whether or not the flight detection information is added (step S193). When the flight detection information is added (step S193: Yes), the image classification unit 2d counts the number of times the flight image has been read (step S195), and whether or not the flight image has been read continuously for a predetermined number of images. Is determined (step S197).

  When the predetermined number of images have not been read continuously (step S197: No), the image processing control unit 2a determines whether or not a series of all the images has been processed (step S199), and processes all the images. If yes (step S199: Yes), the process returns to step S107, and if all the images have not been processed (step S199: No), the control from step S191 is repeated for the unprocessed image. .

  Note that even when it is determined in step S193 that no fecal detection information is added (step S193: No), the image processing control unit 2a determines whether or not a series of all images has been processed in step S199. to decide.

  On the other hand, when the stool images are read continuously for a predetermined number of images (step S197: Yes), the image processing control unit 2a rereads the images at the time point back in time series by the predetermined number of images (step S201). ), The image classification unit 2d determines whether or not the flight detection information is added (step S203). When the fecal detection information is added (step S203: Yes), the image classification unit 2d adds colon image information indicating that it is a colon image to the processing target image (step S205).

  Then, the image processing control unit 2a records the image to which the large intestine image information is added in the image storage unit 5a (step S207), determines whether or not a series of all the images has been processed (step S209), When all the images are not processed (step S209: No), the processing from step S201 is repeated for the unprocessed images, and when all the images are processed (step S209: Yes), step S107. Return to

  When it is determined in step S203 that the flight detection information is not added (step S203: No), the image classification unit 2d counts the number of non-reads that is the number of times the flight image cannot be read (step S211). Then, it is determined whether or not the reading of the flight images has stopped for a predetermined number of images (step S213). If the predetermined number of images has not been interrupted (step S213: No), the image processing control unit 2a performs the determination in step S209. If the predetermined number of images has been interrupted (step S213: Yes), the process returns to step S107. To do.

  In this way, in the large intestine image classification process, the stool image reading is continued for a predetermined number of images to identify the image from which imaging of the large intestine is started, and the stool image reading is interrupted for a predetermined number of images. Then, it is possible to identify an image for which imaging of the large intestine has been completed, add large intestine image information to each image between them, and classify each image as a large intestine image into the same large intestine image group.

  In step S195, the image classification unit 2d may include, for example, a counter that counts the number of read fecal images, and may detect the number of continuous reads by incrementing the counter each time a fecal image is read. Then, similarly, the number of consecutive non-reads may be detected by incrementing the counter. In this case, the image classification unit 2d may reset these counters when the reading number or the non-reading number stops or when a predetermined number of images continues.

  In step S201 immediately after it is determined in step S197 that the predetermined number of stool images have continued, the image processing control unit 2a does not necessarily reread the image at the time point back in time series by the predetermined number of images. There may be no need, and an image subsequent to an already read image may be newly read.

  In step S191, the image processing control unit 2a simultaneously reads a plurality of images having a predetermined number of images. In step S193, the image classification unit 2d adds fecal detection information to each read image. The head of the large intestine image may be specified by determining whether or not. In step S191, the image processing control unit 2a may read the subsequent images classified into the small intestine image group by the small intestine image classification process.

  Next, the stomach image classification process in step S109 shown in FIG. 3 will be described. FIG. 8 is a flowchart showing the procedure of the stomach image classification process. As shown in FIG. 8, the image processing control unit 2a resets variables N and M for counting the image number of the image to be read as N = 0 and M = 0 (step S221), and stores them in the image storage unit 5a. The Nth image in the time series among the series of images read is read (step S223), and the variable N is incremented with N = N + 1 (step S225).

  Then, the image classification unit 2d determines whether or not small intestine image information is added to the read image (step S227). When small intestine image information is not added (step S227: No), image processing is performed. The control unit 2a determines whether or not a series of all images has been processed (step S229). If all the images have not been processed (step S229: No), the control unit 2a performs step S223 on the unprocessed image. If all the images have been processed (step S229: Yes), the process returns to step S109.

  On the other hand, when the small intestine image information is added to the read image (step S227: Yes), the image processing control unit 2a is the Mth image in time series among the series of images stored in the image storage unit 5a. (Step S231), the variable M is incremented as M = M + 1 (step S233), and the image classification unit 2d adds stomach image information indicating a stomach image to the read image (step S235). .

  Thereafter, the image processing control unit 2a records the image to which the stomach image information is added in the image storage unit 5a (step S237), and determines whether or not the variables N and M have a relationship of “M ≧ N”. (Step S239), if the relationship is not “M ≧ N” (Step S239: No), the processing from Step S231 is repeated for the unprocessed image, and the relationship is “M ≧ N” ( Step S239: Yes), the process returns to Step S109.

  In this way, in the stomach image classification process, the stomach image information is added to each previous image in time series than the images classified into the small intestine image group by the small intestine image classification process, and each image is the same as the stomach image. Can be classified into a group of stomach images.

  In the stomach image classification process, for example, the image processing control unit 2a records the first image number classified into the small intestine image group in the small intestine image classification process, and the recorded image number is set as the variable N in step S221. By setting, the processing of steps S223 to S229 may be omitted, and the processing from step S231 may be performed immediately.

  Next, the image display process in step S111 shown in FIG. 3 will be described. FIG. 9 is a flowchart showing the processing procedure of the image display processing. As shown in FIG. 9, the image display control unit 6a determines whether or not a target region for displaying an image among the stomach, the small intestine, and the large intestine is designated by the designation information (step S251). (Step S251: No), the all-parts reproduction mode for displaying the images of all parts is set (Step S253), and if designated (Step S251: Yes), the designation to display only the image of the designated part The part reproduction mode is set (step S255).

  Subsequently, the image display control unit 6a determines whether or not the image reproduction button included in the input unit 3 is pressed (step S257). When the button is not pressed (step S257: No), this determination is performed until it is pressed. Wait for the process repeatedly.

  When the image reproduction button is pressed (step S257: Yes), the image display control unit 6a is the first image in time series among a series of images classified into image groups for each part and stored in the image storage unit 5a. Is read (step S259), it is determined whether or not image information indicating that it is a display target part is added (step S261), and if image information of the display target part is added (step S261: Yes). The read image is displayed on the display unit 4 (step S263).

  Then, the image display control unit 6a determines whether or not a series of all images has been processed (step S265), and when all the images have not been processed (step S265: No), the image is not processed. On the other hand, if the processing from step S259 is repeated and all the images have been processed (step S265: Yes), the process returns to step S111. In addition, when the image information of the display target part is not added at step S261 (step S261: No), the image display control unit 6a immediately performs the determination at step S265.

  As described above, in the image display processing, the display mode for displaying all the sites or the specified site is set with reference to the designation information of the display target site input in advance, and the display target site corresponding to the set display mode is set. Only the image to which the image information is added can be displayed. That is, the image display control unit 6a can sequentially display each image of at least one of the stomach image group, the small intestine image group, and the large intestine image group according to the reproduction mode and the designated display target portion. .

  In the image display process, for example, in the small intestine image classification process and the large intestine image classification process, the image processing control unit 2a records the leading image numbers of the small intestine image group and the large intestine image group, and the image display control unit 6a In step S259, the first image of the image group corresponding to the display target portion may be read with reference to the recorded image number of each first image.

  FIG. 10 is a diagram illustrating an example of a GUI screen displayed on the display unit 4 when inputting designation information for designating a display target part. In the example shown in FIG. 10, “all site playback” and “designated site playback” are displayed in the “playback site” window as check items for selecting the all site playback mode or the specified site playback mode. "" Column further displays a check item that can designate at least one region of the stomach, small intestine, and large intestine. As shown in FIG. 10, when each item of “designated part reproduction” and “small intestine” is checked as the designated part reproduction mode, the image display control unit 6a The text information indicating the attribute is sequentially displayed in the image display area PI in the “diagnosis / diagnosis” window.

  As described above, in the image display device 1 according to the first embodiment, for the series of images stored in the image storage unit 5a, the region detection unit 2b performs a villi region and a feature image region from each image as a feature image region. The stool area is detected, and the feature image detection unit 2c calculates the occupation rate of the detected villi region and the stool region, detects the villi image and the stool image as the feature image according to the calculated occupation rate, and classifies the images. The unit 2d adds image information indicating that the image is a stomach image, a small intestine image, or a large intestine image to each image based on the number of consecutive villi images and stool images, and a stomach image group for each added image information. The image display control unit 6a refers to the image information added to each image, and is classified according to the image reproduction mode and the designated display target part. Images Since each image included in at least one image group is sequentially displayed, only an image obtained by imaging a desired observation target region can be displayed, and a series of images can be efficiently observed. .

  When recording an image to which various types of information are added in each processing of steps S101 to S109, the image processing control unit 2a may record the image already stored in the image storage unit 5a so as to update it. Of course, an already stored image may be added and recorded while being stored, or only additional information may be newly recorded in association with the image.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. In Embodiment 1 described above, when a large intestine image is classified, the large intestine image is classified based on the number of consecutive stool images detected according to the occupancy rate of the stool region. In the second aspect, feature image regions having mucous membranes and villi are detected from each image, and the large intestine images are classified based on the number of consecutive feature images detected according to the occupancy of each feature image region. I have to.

  FIG. 11 is a block diagram showing a configuration of the image display apparatus 11 according to the second embodiment. As shown in FIG. 11, the image display device 11 includes an image processing unit 12 instead of the image processing unit 2 provided in the image display device 1. Further, the image processing unit 12 replaces the image processing control unit 2a, the region detection unit 2b, and the feature image detection unit 2c included in the image processing unit 2, with the image processing control unit 12a, the region detection unit 12b, and the feature image detection unit. 12c. Other configurations are the same as those of the first embodiment, and the same reference numerals are given to the same components.

  Similar to the image processing control unit 2a, the image processing control unit 12a acquires and processes the image stored in the image storage unit 5a, and causes the image storage unit 5a to store the processed image. However, the image processing control unit 12a controls the region detection unit 12b and the feature image detection unit 12c instead of controlling the region detection unit 2b and the feature image detection unit 2c.

  Similar to the region detection unit 2b, the region detection unit 12b detects a feature image region having a predetermined feature from each image. Further, the region detection unit 12b detects a main feature image region and a sub feature image region from each image corresponding to the main feature as the predetermined feature and the sub feature different from the main feature.

  Similar to the feature image detection unit 2c, the feature image detection unit 12c calculates the occupancy rate of the feature image region detected by the region detection unit 12b with respect to the entire image region, and detects the feature image according to the calculated occupancy rate. The feature image information is added to the detected feature image. The feature image detection unit 12c calculates the main feature occupancy ratio of the main feature image region with respect to the entire image region based on the main feature image region and the sub feature image region detected by the region detection unit 12b. The feature image area calculates the sub feature occupancy ratio of the main feature image area to the main feature image area, detects the feature image having the main feature image area according to the calculated main feature occupancy ratio and the sub feature occupancy ratio, and detects the feature image in the detected feature image. Add information.

  Here, processing and operations performed by the image display device 11 will be described. FIG. 12 is a flowchart illustrating a processing procedure in which the image display device 11 processes and displays a series of images stored in the image storage unit 5a. Note that the flowchart shown in FIG. 12 classifies a series of images captured and generated using a capsule endoscope into a stomach image group, a small intestine image group, and a large intestine image group, as in the processing procedure shown in FIG. The processing procedure to be displayed is shown as an example.

  As shown in FIG. 12, the region detection unit 12b, the feature image detection unit 12c, and the image classification unit 2d execute steps S301 and S303 in the same manner as steps S101 and S103 shown in FIG. Classify into small intestine image group.

  Subsequently, the region detection unit 12b and the feature image detection unit 12c detect a villi region and a mucosal region having a mucous membrane as an intestinal feature from each image, calculate an occupancy ratio of each detected region, According to each calculated occupancy rate, a low villus image detection process for detecting a low villus image as a feature image indicating a state where there are few villuses on the surface of the mucous membrane is performed (step S305), and the image classification unit 2d Large intestine image classification processing is performed to classify each small villus image continuous in the number of images or more into a large intestine image group as a large intestine image (step S307).

  Thereafter, the image classification unit 2d and the image processing control unit 6a execute steps S309 and S311 in the same manner as steps S109 and S111, respectively, and the control unit 6 ends the series of processes.

  Next, the low villi image detection process in step S305 shown in FIG. 12 will be described. FIG. 13 is a flowchart showing a processing procedure of the oligovillus image detection processing. As shown in FIG. 13, the image processing control unit 12a reads the first image in time series among the series of images stored in the image storage unit 5a (step S321), and the area detection unit 12b reads the read image. A mucous membrane region having a mucous membrane as a main feature is detected as a main feature image region (step S323), and a villi region having villi as a sub feature is detected as a sub feature image region (step S325). In step S323, the region detection unit 12b detects an image region showing skin color to red as a mucous membrane region.

  Subsequently, the feature image detection unit 12c calculates the occupation ratio of the detected mucosal area with respect to the entire image area as the main feature occupation ratio (step S327), and also uses the occupation ratio of the villi area with respect to the mucosal area as the sub-feature occupation ratio. Calculate (step S329). Further, the feature image detection unit 12c determines whether or not the calculated occupancy rate of the mucosa region is larger than a threshold value input in advance (step S331), and when the occupancy rate of the mucosa region is large (step S331). : Yes), it is determined whether or not the calculated occupation rate of the villi region is smaller than a threshold value inputted in advance (step S333). If the occupation rate of the villi region is small (step S333: Yes), processing is performed. Low villus image information indicating that the image is a low villus image is added to the target image (step S335).

  Then, the image processing control unit 12a records the image to which the low villi image information is added as a low villi image in the image storage unit 5a (step S337), and determines whether or not all the series of images have been processed. (Step S339) When all the images are not processed (Step S339: No), the processing from Step S321 is repeated for the unprocessed images, and all the images are processed (Step S339: Yes), the process returns to step S305.

  In step S331, the image processing control unit 12a determines that the mucosal area occupancy is equal to or lower than the threshold value (step S331: No) and the villus area occupancy is equal to or higher than the threshold value in step S333 ( Step S333: No), the determination of step S339 is immediately performed.

  In this way, in the low villi image detection processing, the small villi are images that have a large mucosal area in the entire image area and a high necessity for observation, and that show the characteristics of the large intestine with fewer villi on the surface of the mucosa. An image can be detected from a series of images.

  In step S325, the region detection unit 12b records, for example, the villi region detected in the villi image detection process in step S301 in association with each image, and refers to the recorded villi region, thereby Detection may be omitted.

  Further, the region detection unit 12b may change the processing order of steps S323 and S325, and the region detection unit 12b and the feature image detection unit 12c may change the processing order of steps S325 and S327.

  Next, the large intestine image classification process in step S307 shown in FIG. 12 will be described. FIG. 14 is a flowchart illustrating a processing procedure of large intestine image classification processing. In the large intestine image classification process shown in FIG. 14, the colon image is classified by the same processing procedure as the large intestine image classification process shown in FIG. I have to.

  As shown in FIG. 14, the image processing control unit 12a reads the first image in time series among the series of images stored in the image storage unit 5a (step S351), and the image classification unit 2d converts the read image into the read image. It is determined whether or not the low villus detection information is added (step S353). If the low villus detection information is added (step S353: Yes), the number of times of reading the low villus image is counted (step S355). Then, it is determined whether or not a predetermined number of images of the villus image are continuously read (step S357).

  When the predetermined number of images have not been read continuously (step S357: No), the image processing control unit 12a determines whether or not a series of all images has been processed (step S359), and all the images are processed. If YES in step S359, the process returns to step S307, and if all the images have not been processed (step S359: NO), the control from step S351 is repeated for the unprocessed image. .

  Note that the image processing control unit 12a also determines whether or not all of the series of images have been processed in step S359 even when it is determined in step S353 that the low villi detection information is not added (step S353: No). Judging.

  On the other hand, in the case where the predetermined number of images are continuously read, the image processing control unit 12a rereads the images that have been traced back in time series by the predetermined number of images (step S357: Yes). S361), the image classification unit 2d determines whether or not the low villus detection information is added (step S363). When the oligovillus detection information is added (step S363: Yes), the image classification unit 2d adds the colon image information to the image to be processed (step S365).

  Then, the image processing control unit 12a records the image with the colon image information added thereto in the image storage unit 5a (step S367), and determines whether or not a series of all images has been processed (step S369). When all the images are not processed (step S369: No), the processing from step S361 is repeated for the unprocessed images, and when all the images are processed (step S369: Yes), step S307 is performed. Return to

  If it is determined in step S363 that the low villus detection information is not added (step S363: No), the image classification unit 2d counts the number of non-reads that is the number of times that the low villus image cannot be read (step S363). S371), it is determined whether or not the reading of a few villus images has been interrupted for a predetermined number of images (step S373). Then, if the predetermined number of images has not been interrupted (step S373: No), the image processing control unit 12a determines in step S369. If the predetermined number of images has been interrupted (step S373: Yes), the process returns to step S307. To do.

  In this way, in the large intestine image classification process according to the second embodiment, as in the large intestine image classification process according to the first embodiment, the reading of the small villi images continues for a predetermined number of images, thereby imaging the large intestine. In addition, the images of the large intestine are identified by stopping the predetermined number of images being read, and the colon image information is added to each image between them, Each of these images can be classified as a large intestine image into the same large intestine image group.

  As described above, in the image display device 11 according to the second embodiment, the area storage unit 12b, the feature image detection unit 12c, and the image classification unit 2d store the image storage unit 5a in the same manner as the image display device 1. The series of images can be classified into a stomach image group, a small intestine image group, and a large intestine image group, and each image included in at least one image group of the classified image group is displayed as an image by the image display control unit 6a. Since the display is sequentially performed according to the playback mode and the display target portion specified by the specification information, only an image obtained by imaging the desired observation target portion can be displayed, and a series of images can be efficiently observed. Can be done.

  In Embodiments 1 and 2 described above, a series of images stored in the image storage unit 5a is classified into a stomach image group, a small intestine image group, and a large intestine image group, and at least one image group is displayed. However, at least one image group may be displayed by classifying into a small intestine image group and / or a large intestine image group, or at least one image group is displayed by classifying into a small intestine image group and a stomach image group. You may make it do.

  Further, in the first and second embodiments described above, a series of images obtained by imaging the inside of the stomach, small intestine, and large intestine using the capsule endoscope has been described as being classified, but other organs are imaged. The images may be classified and displayed, and not limited to images obtained by imaging organs, a series of images obtained by imaging various subjects may be classified and displayed for each subject. .

It is a block diagram which shows the structure of the image display apparatus concerning Embodiment 1 of this invention. It is a schematic diagram explaining the outline | summary of the process which classifies an image with the image display apparatus shown in FIG. 3 is a flowchart illustrating a processing procedure performed by the image display apparatus illustrated in FIG. 1. It is a flowchart which shows the process sequence of the villi image detection process shown in FIG. It is a flowchart which shows the process sequence of the small intestine image classification process shown in FIG. It is a flowchart which shows the process sequence of the stool image detection process shown in FIG. It is a flowchart which shows the process sequence of the large intestine image classification process shown in FIG. It is a flowchart which shows the process sequence of the stomach image classification | category process shown in FIG. It is a flowchart which shows the process sequence of the image display process shown in FIG. It is a figure which shows an example of the GUI screen which the image display apparatus shown in FIG. 1 displays. It is a block diagram which shows the structure of the image display apparatus concerning Embodiment 2 of this invention. It is a flowchart which shows the process sequence which the image display apparatus shown in FIG. 11 performs. It is a flowchart which shows the process sequence of the low villus image detection process shown in FIG. It is a flowchart which shows the process sequence of the large intestine image classification | category process shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,11 Image display apparatus 2,12 Image processing part 2a, 12a Image processing control part 2b, 12b Area | region detection part 2c, 12c Feature image detection part 2d Image classification part 3 Input part 4 Display part 5 Storage part 5a Image storage part 6 Control unit 6a Image display control unit

Claims (11)

In an image display device that sequentially displays a series of input images,
A region detecting means for detecting a feature image region having a predetermined feature from each image included in the series of images;
Image detecting means for calculating an occupancy ratio of the feature image area with respect to the entire image area of each image, and detecting each feature image having the feature image area from the series of images according to the calculated occupancy ratio When,
Image classification means for classifying the series of images into one or more image groups based on the number of consecutive images in time series of feature images detected by the image detection means;
Display control means for performing control to sequentially display each image included in at least one image group designated in advance from each image group classified by the image classification means;
An image display device comprising:   The region detection means divides the entire image region into one or more small regions, detects the feature image region for each small region,
  The image detection means determines whether or not the feature image area is detected for each of the small areas, and calculates the occupation ratio based on the number of areas of the small areas in which the feature image areas are detected. The image display apparatus according to claim 1. The image display device according to claim 1 , wherein the image classifying unit classifies the feature images that are continuous for a predetermined number of continuous images or more into the same image group. The image classification unit is configured to display images that are images before the feature images that are consecutive in the number of continuous images that are equal to or greater than the predetermined number of consecutive images and that are not feature images, different from the image group in which the feature images are classified. The image display device according to claim 3 , wherein the image display device is classified into image groups. The predetermined feature includes a main feature and a sub feature different from the main feature,
The region detecting means detects a main feature image region having the main feature and a sub feature image region having the sub feature from the images,
The image detecting means calculates a main feature occupancy ratio of the main feature image area with respect to the entire image area of each image, and calculates a sub feature occupancy ratio of the sub feature image area with respect to the main feature image area of each image. 5. A feature image according to claim 1 , wherein each feature image having the main feature image area is detected from the series of images according to the calculated main feature occupancy rate and sub feature occupancy rate. The image display apparatus as described in any one. The predetermined feature has a plurality of different features,
The region detection means detects a feature image region having each feature from the respective images,
The image detection means calculates the occupancy rate of the feature image region having the features of the images, detects the feature images for the plurality of features according to the calculated occupancy rates,
6. The image classification unit according to claim 1 , wherein the series of images is classified into an image group for each of the plurality of features based on the number of the continuous images for each of the plurality of features. The image display device described in one. Input means for receiving designation information for designating at least one image group from the image groups;
The image display device according to claim 1 , wherein the display control unit performs control to sequentially display each image included in an image group designated by the designation information. Each of the images is an image of the inside of the digestive organs,
The predetermined characteristic is an intestinal characteristic that identifies at least one of the small intestine and the large intestine,
The image display device according to claim 1 , wherein the image classification unit classifies the series of images into an image group obtained by capturing at least one of the small intestine and the large intestine. Each of the images is an image of the inside of the stomach and small intestine,
The predetermined characteristic is an intestinal characteristic that identifies the small intestine,
The image according to any one of claims 1 to 8 , wherein the image classifying unit classifies each image before the image obtained by imaging the small intestine in time series as an image group obtained by imaging the stomach. Display device. The image display apparatus according to claim 8 or 9 , wherein the intestinal characteristic is at least one of a mucous membrane, villi on the surface of the mucous membrane, and feces. The image display device according to claim 1 , wherein the series of images is generated using a capsule endoscope.

Images