JP2006320650A - Image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To observe a series of images efficiently by reducing time for displaying images of less need for the observation. <P>SOLUTION: An image display device 1 is equipped with an image processing control part 2a which acquires images from a storage part 5 and controls various kinds of image processing for the acquired images to make the storage part 5 store the processed image, an image sorting part 2b which calculates a correlation value of images continuing in the chronological order and sorts each image to image groups depending on the calculated correlation value, a marked image detecting part 2c which detects a characteristic image region with a prescribed characteristic from each image and detects the characteristic image with the detected characteristic image region as a marked image, a representative image extracting part 2d which extracts the marked image and a beginning image in each image group as the representative image and sets a display rate for the each extracted representative image, and an image display control part 6a which controls to display a series of representative images sequentially depending on the set displayed rate. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image display device that sequentially displays a series of input images, and more particularly to an image display device suitable for application to display of a series of images obtained by imaging the inside of a subject using a capsule endoscope. It is.

  In recent years, swallowable capsule endoscopes have been developed in the field of endoscopes. This capsule endoscope has an imaging function and a wireless communication function. After being swallowed from the patient's mouth for observation in various organs, until it is naturally discharged from the human body, for example, the stomach, the small intestine, Images are sequentially taken while moving inside the digestive organs such as the large intestine according to the peristaltic movement.

  While moving inside the organ, image data captured inside the body by the capsule endoscope is sequentially transmitted outside the body by wireless communication and stored in a memory provided in the receiver outside the body or provided in the receiver. The image is displayed on the displayed display. A doctor, nurse, or the like can make a diagnosis based on the image displayed on the display based on the image data stored in the memory or the image displayed on the display provided in the receiver along with the reception.

  Usually, the series of images captured by the capsule endoscope is enormous, and doctors, nurses, and the like require a great deal of time and effort to observe and diagnose this series of images. Correspondingly, when an image is displayed based on the image data stored in the memory for observation, the degree of similarity between the two images is determined, and the image display rate is determined based on the determination result. There has been proposed a display device that changes the angle (see, for example, Patent Document 1). In this display device, focusing on the fact that there are many similar images that are continuously captured when the movement of the capsule endoscope is stagnant, when the similarity between the two images is low, the image is displayed at a low display rate. If the similarity is high, an image is displayed at a high display rate.

JP-T-2004-521626

  However, in such a display device, the display rate of the image is changed in accordance with the similarity between the two images. For example, even if the image includes a bleeding site and is highly necessary to be observed, the bleeding When the region is a small region, there is a problem that the similarity is determined to be high and the image is displayed at a high display rate, which may make observation difficult.

  The present invention has been made in view of the above, and even when a series of images is classified according to the similarity of images, it is easy to observe an image having a high necessity for observation, and the necessity for observation is low. An object of the present invention is to provide an image display device that can reduce the display time of an image and, as a result, can efficiently observe a series of images.

  In order to achieve the above object, an image display device according to claim 1 is an image display device for sequentially displaying a series of inputted images, and correlates each image included in the series of images with each other. Image classification means for classifying the image into one or more image groups according to the degree, a feature image area having a predetermined feature is detected from each of the images, and each feature image having the detected feature image area is Feature image detection means for detecting from among the images, representative image extraction means for extracting the feature images in each image group classified by the image classification means as representative images representing each image group, and the representative And image display control means for performing control to sequentially display the representative images extracted by the image extraction means.

  In the image display device according to claim 2, in the above invention, the representative image extraction means extracts at least a head image at the head of each image group in time series as a representative image of each image group. Features.

  According to a third aspect of the present invention, there is provided the image display device according to the above-described invention, wherein a predetermined image of the feature image region generated between the feature images consecutive in time series among the plurality of feature images detected by the feature image detection means is provided. A feature image selecting unit that calculates a change amount of the feature amount and selects a feature representative image that represents the plurality of feature images based on the calculated change amount, and the feature image detection unit detects each feature detected The predetermined feature amount of the image area is calculated, the feature image selection unit calculates the change amount based on the feature amount calculated by the feature image detection unit, and the representative image extraction unit Of the feature images in the group, the feature representative image is extracted as a representative image.

  According to a fourth aspect of the present invention, in the above invention, the predetermined feature amount is a position of a feature image region in the feature image.

  In the image display device according to claim 5, in the above invention, when the image display control means displays a representative image that is the feature image, a mark indicating that the image is a feature image in the vicinity of the representative image It is characterized by performing control to display.

  Further, in the image display device according to claim 6, in the above invention, each image is an image obtained by imaging the inside of the organ, and the predetermined feature is a feature showing a lesion inside the organ. It is characterized by.

  The image display device according to a seventh aspect of the present invention is characterized in that, in the above invention, the feature indicating the lesion is at least one of bleeding, discoloration, and shape abnormality.

  An image display apparatus according to an eighth aspect of the present invention is characterized in that, in the above invention, the series of images is generated using a capsule endoscope.

  According to the image display device of the present invention, even when a series of images are classified according to the similarity of images, it is possible to easily observe an image having a high necessity for observation, and to display an image with a low necessity for observation. As a result, a series of images can be observed efficiently.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, exemplary embodiments of an image display device according to the invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments. In the description of the drawings, the same parts are denoted by the same reference numerals.

(Embodiment 1)
First, the image display apparatus according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing the configuration of the image display apparatus 1 according to this embodiment. As shown in FIG. 1, the image display device 1 includes an image processing unit 2 that processes an image stored in a storage unit 5, an input unit 3 that receives input of various information, and a display unit 4 that displays various information. , A storage unit 5 that stores various information, and a control unit 6 that controls processing and operation of each unit of the image display device 1. The image processing unit 2, the input unit 3, the display unit 4, and the storage unit 5 are electrically connected to the control unit 6.

  The image processing unit 2 includes an image processing control unit 2a, an image classification unit 2b, an attention image detection unit 2c, and a representative image extraction unit 2d. The image processing control unit 2 a acquires an image from the storage unit 5, controls various types of image processing on the acquired image, and outputs and stores the processing result image in the storage unit 5. In particular, the image processing control unit 2a controls the image classification unit 2b, the attention image detection unit 2c, and the representative image extraction unit 2d to execute predetermined image processing.

  The image classification unit 2b calculates a correlation value between two time-series images acquired by the image processing control unit 2a, and associates an image to be processed with an existing or new image group according to the calculated correlation value. By repeating this association process for all the series of images, this series of images is classified into one or more image groups.

  Specifically, the image classification unit 2b refers to a threshold value for a correlation value input in advance, and determines an image to be processed as an existing image according to the magnitude relationship between the threshold value and the calculated correlation value. A group or a new image group is associated, and the group number of the associated image group is added to the image to be processed. Note that the image classification unit 2b calculates, for example, a normalized cross-correlation between corresponding pixel values between two images as a correlation value. Further, the image classification unit 2b may obtain a color difference, a luminance difference, or the like of each corresponding pixel as a correlation value based on each pixel value.

  The attention image detection unit 2c serving as the feature image detection unit detects a feature image region having a predetermined feature from the image acquired by the image processing control unit 2a, and at the time of observing the feature image having the detected feature image region. It detects as an attention image which is an image which should be noted. The attention image detection unit 2c repeats this detection process for all the series of images, thereby detecting all feature images as attention images. At this time, the attention image detection unit 2c adds attention image information indicating the attention image to each image detected as the attention image.

  Note that the attention image detection unit 2c detects a feature image region by identifying a predetermined feature based on color information indicated by each pixel constituting the image, for example. Further, the attention image detection unit 2c may detect a feature image region based on various feature amounts such as a contour shape, a texture, and a density gradient, without being limited to color information.

  The representative image extraction unit 2d extracts a representative image representing each image group from among the image groups classified by the image classification unit 2b. Specifically, the representative image extraction unit 2d extracts a target image in each image group as a representative image of each image group, and extracts a head image at the head in time series in each image group as a representative image. To do. In addition, the representative image extraction unit 2d sets a low-speed display rate for displaying the image at a low speed so that sufficient observation time is given to the representative image that is the target image, and for the representative image that is the top image, Sets the normal display rate for displaying at a normal speed faster than the low speed display rate.

  The representative image extraction unit 2d extracts all the attention images for each image group, and extracts only the top image in the image group that does not include the attention image. Further, instead of using the head image as the representative image, the representative image extracting unit 2d may extract, for example, an image in a predetermined order such as an image at the end in time series in the image group as the representative image. Good.

  Each representative image extracted by the representative image extraction unit 2d is output to the storage unit 5 by the image processing control unit 2a and stored in the representative image storage unit 5b which is a storage area for storing the representative image. Here, instead of storing each representative image, the image processing control unit 2a may newly store only the group number, attention image information, and display rate of each representative image in association with the original image.

  The input unit 3 receives input of an image to be processed by the image display device 1 and various processing information. Specifically, the input unit 3 includes a communication interface such as USB or IEEE1394, and accepts input of an image from an external device. The input unit 3 includes various switches, input keys, a mouse, a touch panel, and the like. Accepts input of processing information. Note that the input unit 3 may include an interface corresponding to a portable storage medium such as various memory cards, CDs, and DVDs, and may accept input of an image from the portable storage medium.

  The display unit 4 includes a liquid crystal display and displays various information including images. In particular, the display unit 4 displays an image stored in the storage unit 5 and a GUI (Graphical User Interface) screen that requests an operator of the image display device 1 to input various processing information.

  The storage unit 5 is realized by a ROM that stores various processing programs and the like in advance, and a RAM that stores processing parameters, processing data, and the like of each processing. In particular, the storage unit 5 includes an image storage unit 5a and a representative image storage unit 5b, which are storage areas for storing an externally input image and a representative image extracted by the representative image extraction unit 2d. Note that the storage unit 5 may include a portable storage medium such as various memory cards, CDs, and DVDs as a removable image storage unit.

  The control unit 6 is realized by a CPU or the like that executes various processing programs stored in the storage unit 5. In particular, the control unit 6 includes an image display control unit 6a. The image display control unit 6a uses a series of representative images stored in the representative image storage unit 5b based on the display rate set for each representative image. The display unit 4 is sequentially displayed. In addition, when displaying the representative image that is the attention image, the image display control unit 6a performs control to display the attention image mark indicating the attention image in the vicinity of the representative image. The attention image mark is stored in advance in the storage unit 5, and the image display control unit 6 a acquires the attention image mark from the storage unit 5 when attention image information is added to the representative image to be processed. Display with image.

  Here, processing and operations performed by the image display apparatus 1 will be described. FIG. 2 is a flowchart illustrating a processing procedure in which the image display apparatus 1 processes and displays a series of images stored in the image storage unit 5 a under the control of the control unit 6. The flowchart shown in FIG. 2 exemplifies a processing procedure for displaying a series of images generated by imaging the inside of an organ such as a digestive organ using a capsule endoscope (not shown).

  As shown in FIG. 2, the image classification unit 2b performs a grouping process for classifying the images included in the series of images stored in the image storage unit 5a into image groups (step S101), and the target image detection unit 2c. Performs a target image detection process for detecting a target image from a series of images (step S103), and the representative image extraction unit 2d extracts the target image and the head image in each classified image group as a representative image. A representative image extraction process is performed (step S105), and the image display control unit 6a performs a representative image display process for sequentially displaying a series of extracted representative images based on a display rate set for each representative image (step S105). S107), the control unit 6 ends the series of processing.

  Next, the grouping process in step S101 shown in FIG. 2 will be described. FIG. 3 is a flowchart showing the processing procedure of the grouping process. As shown in FIG. 3, the image processing control unit 2a initializes a variable G indicating the group number of the image group to G = 0 (step S111), and then, among the series of images stored in the image storage unit 5a. Are read from the first and second images in time series (step S113).

  Then, the image classification unit 2b uses the second image as a processing target image, calculates a correlation value with the previous image that is the first image and is the image at the previous time point in time series (step S115), and calculates It is determined whether or not the correlation value is greater than a threshold value input in advance (step S117). When the correlation value is smaller than the threshold value (step S117: No), the image classification unit 2b determines that the similarity between the images is low, updates the group number by incrementing the variable G (step S119), The updated group number is set in the processing target image (step S121). On the other hand, when the correlation value is larger than the threshold value (step S117: Yes), the image classification unit 2b determines that the similarity between the images is high, and sets the group number indicated by the current variable G to the image to be processed. Setting is performed (step S121). The image classification unit 2b sets the group number “0” for the top image.

  Thereafter, the image processing control unit 2a records the image with the group number set in the image storage unit 5a (step S123), and determines whether the group number is set for all the images included in the series of images. (Step S125). When the group number is not set for all the images (step S125: No), the image processing control unit 2a performs control so that the processing from step S113 is repeated for the images that are not set. On the other hand, when the group number is set for all the images (step S125: Yes), the image processing control unit 2a returns to step S101.

  In this way, in the grouping process in step S101, an image having a high similarity with the previous image is associated with the same image group as the previous image, and an image with a low similarity is a new group number updated. Corresponding to an image group. As a result, a series of images is classified into a group of images similar to each other.

  Next, the attention image detection process in step S103 shown in FIG. 2 will be described. FIG. 4 is a flowchart showing the processing procedure of the attention image detection process. As shown in FIG. 4, the image processing control unit 2a reads the first image in time series among the series of images set with the group number and stored in the image storage unit 5a (step S131), and the target image detection unit 2c. Detects an image region indicating a bleeding site as a feature image region from the read image (step S133), and determines whether there is a detected bleeding site (step S135). Note that, when detecting the bleeding site, the attention image detection unit 2c may detect an image region that is more reddish than the mucous membrane inside the organ as the bleeding site.

  When there is a detected bleeding site (step S135: Yes), the attention image detection unit 2c adds attention image information to the image to be processed (step S137), and the image processing control unit 2a After the added image is recorded in the image storage unit 5a (step S139), it is determined whether or not all the series of images have been processed (step S141).

  When all the images are not processed (step S141: No), the image processing control unit 2a repeats the processing from step S131 on the unprocessed images, and when all the images are processed (step S1). S141: Yes), the process returns to step S103. If it is determined in step S135 that there is no detected bleeding site (step S135: No), the image processing control unit 2a immediately determines in step S141.

  In this way, in the attention image detection process in step S103, an image obtained by imaging a bleeding site inside an organ can be detected as a attention image from a series of images. Note that the image to be detected as the attention image is not limited to the image in which the bleeding site is imaged, and for example, an image in which various sites that are suspected to be lesions, such as a faint region and an abnormal shape region in the organ, are detected as the attention image. You may do it. In this case, the attention image detection unit 2c may detect an image region showing features such as a fading color and an abnormal shape as a feature image region.

  Next, the representative image extraction process in step S105 shown in FIG. 2 will be described. FIG. 5 is a flowchart showing a processing procedure of representative image extraction processing. As shown in FIG. 5, the image processing control unit 2a reads the first image in time series among the series of images subjected to the grouping process and the target image detection process and stored in the image storage unit 5a (step S151). The representative image extraction unit 2d determines whether attention image information is added to the read image (step S153).

  When attention image information is added (step S153: Yes), the representative image extraction unit 2d sets a low speed display rate for the image to be processed (step S155), and the image processing control unit 2a sets the low speed display rate. Is recorded in the representative image storage unit 5b as a representative image (step S157).

  On the other hand, when attention image information is not added (step S153: No), the representative image extraction unit 2d determines whether or not the image to be processed is the first image of the image group (step S159). If it is the first image (step S159: Yes), the representative image extraction unit 2d sets the normal display rate for the image to be processed (step S161), and the image processing control unit 2a sets the normal display rate. The head image is recorded as a representative image in the representative image storage unit 5b (step S163).

  After step S157 or step S163, the image processing control unit 2a determines whether or not a series of all images has been processed (step S165). When all the images have not been processed (step S165: No), If the processing from step S151 is repeated for the unprocessed image and all the images are processed (step S165: Yes), the process returns to step S105. If it is determined in step S159 that the image to be processed is not the top image (step S159: No), the image processing control unit 2a immediately determines in step S165.

  In this way, in the representative image extraction process in step S105, for example, as shown in FIG. 6, the head image and the target image of each image group can be extracted as representative images. In the example shown in FIG. 6, the “normal image 1” that is the first image and the “bleeding image 1” to “bleeding image 3” that are the attention images from the image group n that is the n-th image group in time series, “Normal image 3”, which is the first image, is extracted from the image group n + 1 as a representative image. The series of representative images extracted in this way are stored in the representative image storage unit 5b in time series.

  Next, the representative image display process in step S107 shown in FIG. 2 will be described. FIG. 7 is a flowchart showing the processing procedure of the representative image display processing. As shown in FIG. 7, the image display control unit 6a reads the first image in time series among a series of representative images stored in the representative image storage unit 5b (step S171), and whether attention image information is added. It is determined whether or not (step S173).

  When attention image information is added (step S173: Yes), the image display control unit 6a reads the attention image mark from the storage unit 5 (step S175), and reads the read representative image and attention image mark at a low speed display rate. It is displayed on the display unit 4 (step S177). As a result, the image display control unit 6a can display a representative image, which is an attention image having a high necessity for observation, for a longer time than usual. On the other hand, when attention image information is not added (step S173: No), the image display control unit 6a displays the read representative image on the display unit 4 at the normal display rate (step S179).

  Thereafter, the image display control unit 6a determines whether or not a series of representative images are all displayed (step S181). If all the representative images are not displayed (step S181: No), the image is not displayed for the representative images that are not displayed. If the processing from step S171 is repeated and all the images are displayed (step S181: Yes), the process returns to step S107. In this manner, the image display control unit 6a sequentially displays a series of representative images stored in the representative image storage unit 5b according to the display rate set for each image.

  FIG. 8 is a diagram illustrating an example of a GUI screen displayed on the display unit 4 when a representative image is displayed. In the example shown in FIG. 8, character information indicating various attributes of the representative image Pi that is the target image, the target image mark Ma, and the representative image Pi is displayed in the “diagnosis / diagnosis” window.

  As described above, in the image display device 1 according to the first embodiment, the image classification unit 2b calculates the correlation value of images that are continuous in time series with respect to a series of images stored in the image storage unit 5a. A series of images are classified into image groups according to the calculated correlation value, and the attention image detection unit 2c detects a feature image area from each image and pays attention to a feature image having the detected feature image area. The image is detected as an image, and the representative image extraction unit 2d extracts the attention image and the head image in each image group as a representative image, sets a display rate for each extracted representative image, and further displays the image display control unit 6a. However, since a series of representative images are sequentially displayed according to the set display rate, for example, it is possible to easily observe an image including a bleeding site and having a high necessity for observation. The display time can be reduced by limiting the display of images that do not contain normal images and need only to be observed to the first image of each image group, and as a result, efficiently observe a series of images. Can do.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. In the first embodiment described above, all the target images detected by the target image detection unit 2c are displayed as representative images. In the second embodiment, when a plurality of target images exhibit the same characteristics, One of the images of interest is selected and displayed as a representative image.

  FIG. 9 is a block diagram showing a configuration of the image display apparatus 11 according to the second embodiment. As illustrated in FIG. 9, the image display device 11 includes an image processing unit 12 instead of the image processing unit 2 included in the image display device 1. The image processing unit 12 includes an image processing control unit 12a and a target image detection unit 12c instead of the image processing control unit 2a and the target image detection unit 2c included in the image processing unit 2, and a target image selection unit 12e. Newly prepared. Other configurations are the same as those of the first embodiment, and the same reference numerals are given to the same components.

  Similar to the image processing control unit 2 a, the image processing control unit 12 a acquires and processes the image stored in the storage unit 5, and causes the storage unit 5 to store the processing result image. However, the image processing control unit 12a controls the attention image detection unit 12c instead of the attention image detection unit 2c, and newly controls the attention image selection unit 12e, so that among the plurality of attention images showing the same characteristics. Only representative attention images are selected as representative images.

  The attention image detection unit 12c detects the attention image from the series of images in the same manner as the attention image detection unit 2c, calculates the feature amount of the feature image area of each detected attention image, and indicates the calculated feature amount. Associate information with the image of interest. Specifically, the attention image detection unit 12c calculates the position of the feature image area in the attention image as the feature amount, generates position information indicating the calculated position, and adds the position information to the attention image. The position of the feature image area is indicated by at least one of, for example, the barycentric position of the feature image area, the maximum or minimum luminance position, the position having a predetermined hue, and the outermost peripheral position of the feature image area.

  The attention image selection unit 12e, based on the positional change of the feature image region that occurs between the attention images that are consecutive in time series, from among the plurality of attention images detected by the attention image detection unit 12c. A representative representative image that represents the image is selected. Specifically, the attention image selection unit 12e calculates a motion vector indicating a change in the position of the feature image area generated between two consecutive attention images, and based on the calculated motion vector, the two feature images. It is determined whether or not the region is an image region showing the same feature. If the region is an image region showing the same feature, one of the attention images is changed to a attention cancellation image that is not extracted as a representative image. The attention image selection unit 12e repeats this process for a series of attention images, and as a result, selects an attention image that does not become the attention cancellation image as the attention representative image.

  Here, processing and operations performed by the image display device 11 will be described. FIG. 10 is a flowchart illustrating a processing procedure in which the image display device 11 processes and displays a series of images stored in the image storage unit 5 a under the control of the control unit 6. The flowchart shown in FIG. 10 exemplifies a processing procedure for displaying a series of images generated by imaging the inside of an organ such as a digestive organ using a capsule endoscope (not shown).

  As shown in FIG. 10, the image classification unit 2b performs a grouping process similar to step S101 (step S201), and the target image detection unit 12c detects a target image from a series of images. (Step S203), the attention image selection unit 12e performs attention image selection processing for selecting a representative representative image from the attention images detected in Step S203 (Step S205), and the representative image extraction unit 2d The representative image extraction process similar to S105 is performed (step S207), the image display control unit 6a performs the representative image display process similar to step S107 (step S209), and the control unit 6 ends the series of processes.

  The processing procedures of the grouping process in step S201, the representative image extraction process in step S207, and the representative image display process in step S209 are shown by the flowcharts shown in FIGS. 3, 5, and 7, respectively. Here, the attention image detection process in step S203 and the attention image selection process in step S205 will be described.

  FIG. 11 is a flowchart illustrating a processing procedure of the target image detection processing in step S203. As shown in FIG. 11, the image processing control unit 12a reads the first image in time series among the series of images set with the group number and stored in the image storage unit 5a (step S211), and the target image detection unit 12c. Detects an image region indicating a bleeding site as a feature image region from the read image and generates position information indicating the position of the bleeding site (step S213).

  Subsequently, the attention image detection unit 12c determines whether or not there is a detected bleeding part (step S215), and when there is a detected bleeding part (step S215: Yes), the attention image is included in the processing target image. The information and the position information generated in step S213 are added (step S217), and the image processing control unit 12a records the image to which the attention image information and the position information are added in the image storage unit 5a (step S219). Then, it is determined whether or not a series of all images has been processed (step S221).

  When all the images have not been processed (step S221: No), the image processing control unit 12a repeats the processing from step S211 on the unprocessed image, and has processed all the images (step S21). S221: Yes), the process returns to step S203. If it is determined in step S215 that there is no detected bleeding site (step S215: No), the image processing control unit 12a immediately determines in step S221.

  In this way, in the attention image detection process in step S203, an image obtained by imaging the bleeding site inside the organ is detected as a attention image from the series of images, and the position information indicating the position of the bleeding region in the detected attention image. Can be added. Note that the image area detected as the feature image area is not limited to the image area showing the bleeding part, but may be an image area showing various features suspected of being a lesion such as a faint part or an abnormal shape part inside the organ.

  Next, the attention image selection process in step S205 shown in FIG. 10 will be described. FIG. 12 is a flowchart illustrating the processing procedure of the attention image selection process. As shown in FIG. 12, the image processing control unit 12a reads two images at the top and second in time series among a series of images subjected to the target image detection process and stored in the image storage unit 5a ( In step S231), the target image selection unit 12e sets the second image as a processing target image, and whether the target image information is added together with the previous image that is the first image and the previous time point image in time series. It is determined whether or not (step S233).

  When attention image information is added to the processing target image and the previous image (step S233: Yes), the attention image selection unit 12e refers to the position information indicating the position of the bleeding part, and includes the bleeding part of the previous image. A region is extracted as a bleeding part template which is a template for pattern matching processing (step S235), and pattern matching processing is performed on the processing target image based on the bleeding part template, and a bleeding part having a high correlation with the bleeding part template is obtained. While detecting the image area to show, the motion vector which shows the position change of the bleeding part of the process target image with respect to a previous image is detected (step S237).

  Subsequently, the attention image selection unit 12e extracts an image region around the bleeding site having a predetermined positional relationship with the bleeding site from the previous image as a peripheral template (step S239), and based on this peripheral template. Then, pattern matching processing is performed on the processing target image to detect an image area having a high correlation with the peripheral template, and a motion vector indicating the motion of the peripheral portion of the bleeding site of the processing target image with respect to the previous image is detected (step S241). ).

  Then, the attention image selection unit 12e determines whether or not the detected motion vector of the bleeding part and the motion vector of the peripheral part of the bleeding part are substantially the same (step S243), and when they are substantially the same (step S243: Yes). Then, attention image cancellation information indicating that the image is the attention cancellation image is added to the processing target image (step S245).

  Thereafter, the image processing control unit 12a records the processing target image to which attention image cancellation information is added in the image storage unit 5a (step S247), and determines whether or not all the series of images have been processed (step S249). ), If not all images have been processed (step S249: No), if the processing from step S231 is repeated for unprocessed images and all images have been processed (step S249: Yes), The process returns to step S205.

  When it is determined in step S233 that attention image information is not added (step S233: No), and when it is determined in step S243 that the motion vectors are not the same (step S243: No). The image processing control unit 12a immediately determines in step S249.

  In step S237 and step S241, the target image selection unit 12e compares each template extracted from the previous image and the image area in the processing target image corresponding by pattern matching on the same screen. A motion vector indicating a change in the center of gravity position of the region is detected. However, in step S237, the target image selection unit 12e may obtain the motion vector only from the position coordinates of the bleeding part between the previous image and the processing target image without performing pattern matching.

  Further, in step S243, the target image selection unit 12e sets, for example, a threshold in which a difference in direction and size of each motion vector is set in advance for the detected motion vector of the bleeding site and the motion vector of the peripheral portion of the bleeding site. It is determined whether or not each motion vector is the same depending on whether or not it is less than or equal to the value. In addition, the attention image selection unit 12e calculates a vector difference, an inner product, an outer product, and the like of each motion vector, and determines whether or not each motion vector is the same based on at least one of these calculation results. You may make it do.

  In step S245, when the detected motion vector of the bleeding site and the motion vector of the peripheral portion of the bleeding site are substantially the same, the attention image selection unit 12e determines that the bleeding site of the processing target image and the bleeding site of the previous image are the same. The attention image cancellation information is added to the processing target image on the assumption that there is a low possibility that the processing target image is displayed at the time of observation. In addition, in step S239, the attention image selection unit 12e may detect a plurality of image regions as peripheral templates so that the identity of the bleeding site can be determined with high accuracy.

  In this way, in the attention image selection process in step S205, the movement of the bleeding region and the image area around the bleeding portion are compared between two consecutive attention images, so that the bleeding portions of the two attention images are the same. It is possible to determine whether one of the images of interest is the same as the image of cancellation of attention. Note that the processing order of steps S235 to S241 may be changed as appropriate, for example, by exchanging the processing order of steps S237 and S239.

  Here, the attention image selection process shown in FIG. 12 will be specifically described. FIG. 13 is a diagram illustrating an example of an image processed in the attention image selection process. As illustrated in FIG. 13, the attention image selection unit 12e extracts the image region TPa1 including the bleeding site BL1 of the previous image as a bleeding site template, and image regions TPb1 and TPc1 located on the left and right of the bleeding site BL1 in the drawing. Is extracted as a peripheral template. Thereafter, the attention image selection unit 12e performs template matching on the processing target image based on these templates.

  As a result of template matching, when the image areas TPa2, TPb2, TPc2 of the processing target image are detected for the image areas TPa1, TPb1, TPc1 of the previous image, the target image selection unit 12e Motion vectors Va2, Vb2, and Vc2 indicating the motion of the center of gravity are detected, and it is determined whether or not the difference between the direction and the magnitude of each motion vector is equal to or less than a predetermined threshold value.

  In the case of the motion vectors Va2, Vb2, and Vc2 shown in FIG. 13, the attention image selecting unit 12e determines that the motion vectors are substantially the same, and the bleeding site BL2 of the processing target image is the same bleeding site as the bleeding site BL1. As a thing, attention image cancellation information is added to this processing target image.

  On the other hand, when image regions TPa3, TPb3, and TPc3 of the processing target image are detected as a result of template matching, the target image selection unit 12e detects the motion vectors Va3, Vb3, and Vc3, and the direction and magnitude of each motion vector. It is determined whether the difference is equal to or less than a predetermined threshold value.

  In the case of the motion vectors Va3, Vb3, and Vc3 shown in FIG. 13, the target image selection unit 12e determines that each motion vector is a different motion vector, and the bleeding site BL3 of the processing target image is different from the bleeding site BL1. As a result, the processing target image is selected as the attention representative image.

  As a result of performing the attention image selection processing in this way, in the representative image extraction processing in step S207, for example, as shown in FIG. 14, the head image and the attention representative image are extracted as representative images from each image group. In the example shown in FIG. 14, the “normal image 1” that is the first image and the “bleeding image 1” that is the attention representative image from the image group n that is the nth image group in time series, and the first from the image group n + 1. “Normal image 3” which is an image is extracted as a representative image. These series of representative images are stored in the representative image storage unit 5b in time series.

  As described above, in the image display device 11 according to the second embodiment, the attention image detection unit 12c detects the attention image from the series of images classified into the image groups by the image classification unit 2b. Then, the position of the feature image area of each detected target image is calculated, position information indicating the calculated position is added to the target image, and the target image selecting unit 12e causes the feature image generated between the target images that are continuous in time series. The position change of the image area around the area and the feature image area is detected, and based on the detected position change, a representative representative image representing a plurality of similar target images is selected, and the representative image extraction unit 2d The representative representative image and the head image in the image group are extracted as representative images, a display rate is set for each extracted representative image, and the image display control unit 6a further sets the display rate. Since a series of representative images are sequentially displayed according to the image, for example, it is possible to display only images that are highly similar to the previous and subsequent images in the time series and that have a high need for observation among images including bleeding sites. In addition, it is possible to reduce the display time by limiting the display of images that do not include a bleeding site and show a normal state and that is less necessary for observation to only the first image of each image group. It can be observed efficiently.

  In the first and second embodiments described above, it has been described that a series of ungrouped images are first read and then various processes are performed. However, the group number, target image information, Each process may be executed by reading an image associated with a display rate or the like and updating the information.

  In the first and second embodiments described above, the image display control unit 6a sequentially displays from the first image in time series among the series of representative images, and displays all the series of representative images. As described above, for example, based on instruction information related to a display start image input in advance, display is started from a representative image in the middle of time series, and based on instruction information related to a display end image input in advance. Alternatively, the image display process may be terminated by displaying up to a representative image in the middle of time series.

  Further, in the first and second embodiments described above, the image display control unit 6a has been described so as to display only the representative image. For example, only the representative image is displayed based on the instruction information input from a predetermined switch or the like. It may be possible to switch between the display of and the display of all images.

  In Embodiment 1 described above, the control unit 6 performs the attention image detection process after the grouping process. However, the control unit 6 may perform the grouping process after the attention image detection process. Similarly, in the second embodiment, the control unit 6 may perform the grouping process after the attention image detection process or the attention image selection process.

It is a block diagram which shows the structure of the image display apparatus concerning Embodiment 1 of this invention. 3 is a flowchart illustrating a processing procedure performed by the image display apparatus illustrated in FIG. 1. It is a flowchart which shows the process sequence of the grouping process shown in FIG. FIG. 3 is a flowchart showing a processing procedure for attention image detection processing shown in FIG. 2. FIG. It is a flowchart which shows the process sequence of the representative image extraction process shown in FIG. It is a schematic diagram explaining an example of the result of the representative image extraction process shown in FIG. It is a flowchart which shows the process sequence of the representative image display process shown in FIG. It is a figure which shows an example of the GUI screen which the image display apparatus shown in FIG. 1 displays. It is a block diagram which shows the structure of the image display apparatus concerning Embodiment 2 of this invention. It is a flowchart which shows the process sequence which the image display apparatus shown in FIG. 9 performs. It is a flowchart which shows the process sequence of the attention image detection process shown in FIG. It is a flowchart which shows the process sequence of the attention image selection process shown in FIG. It is a schematic diagram explaining an example of the attention image selection process shown in FIG. It is a schematic diagram explaining an example of the result of the attention image selection process shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,11 Image display apparatus 2,12 Image processing part 2a, 12a Image processing control part 2b Image classification part 2c, 12c Attention image detection part 2d Representative image detection part 12e Attention image selection part 3 Input part 4 Display part 5 Storage part 5a Image storage unit 5b Representative image storage unit 6, 16 Control unit 6a Image display control unit

Claims (8)

In an image display device that sequentially displays a series of input images,
Image classification means for classifying each image included in the series of images into one or more image groups according to the degree of correlation between the images;
Feature image detection means for detecting a feature image region having a predetermined feature from each of the images, and detecting each feature image having the detected feature image region from the series of images;
Representative image extraction means for extracting the feature images in each image group classified by the image classification means as a representative image representing each image group;
Image display control means for controlling to sequentially display the representative images extracted by the representative image extraction means;
An image display device comprising:   2. The image display device according to claim 1, wherein the representative image extracting unit extracts at least a head image at the head of each image group in time series as a representative image of each image group. A change amount of a predetermined feature amount of the feature image region generated between feature images consecutive in a time series among a plurality of feature images detected by the feature image detection unit is calculated, and based on the calculated change amount Comprises a feature image selection means for selecting a feature representative image representing the plurality of feature images,
The feature image detection means calculates the predetermined feature amount of each detected feature image region,
The feature image selection unit calculates the change amount based on the feature amount calculated by the feature image detection unit,
The image display apparatus according to claim 1, wherein the representative image extraction unit extracts the feature representative image as a representative image among the feature images in each image group.   The image display device according to claim 3, wherein the predetermined feature amount is a position of a feature image region in the feature image.   5. The display according to claim 1, wherein when the representative image that is the feature image is displayed, the image display control unit performs a control to display a mark indicating the feature image in the vicinity of the representative image. The image display apparatus as described in any one. Each of the images is an image of the inside of an organ,
The image display device according to claim 1, wherein the predetermined feature is a feature that indicates a lesion inside the organ.   The image display apparatus according to claim 6, wherein the feature indicating the lesion is at least one of bleeding, discoloration, and shape abnormality.   The image display device according to claim 1, wherein the series of images is generated using a capsule endoscope.

Images