Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
  • A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00002—Operational features of endoscopes
  • A61B1/00043—Operational features of endoscopes provided with output arrangements
  • A61B1/00045—Display arrangement
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
  • A61B1/00002—Operational features of endoscopes
  • A61B1/00043—Operational features of endoscopes provided with output arrangements
  • A61B1/00045—Display arrangement
  • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0012—Biomedical image inspection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
  • G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
  • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10068—Endoscopic image
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30028—Colon; Small intestine
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30004—Biomedical image processing
  • G06T2207/30092—Stomach; Gastric

Definitions

• the present invention relates to an image display device that sequentially displays a series of input images, and is particularly suitable for application to display of a series of images obtained by imaging a subject using a capsule endoscope.
• the present invention relates to an image display device.
• This capsule endoscope has an imaging function and a wireless communication function. After being swallowed from the patient's mouth for observation in various organs, until it is naturally discharged from the human body, for example, stomach, small intestine, Images are taken sequentially while moving inside the extinct organ such as the large intestine according to its peristaltic movement.
• image data imaged inside the body by the capsule endoscope is sequentially transmitted outside the body by wireless communication and stored in a memory provided in the receiver outside the body or received.
• An image is displayed on a display provided in the machine. Doctors, nurses, etc. make a diagnosis based on the image displayed on the display based on the image data accumulated in the memory or the image displayed on the display provided in the receiver together with the reception be able to.
• Patent Document 1 Japanese Translation of Special Publication 2004-521662 Disclosure of the invention
• the display rate of the image is changed in accordance with the similarity between the two images. For example, there is a need for observation including a bleeding site. Even if the image is high, if the bleeding site is a small area, the similarity will be judged high and the image will be displayed at a high display rate, which may make observation difficult.
• the present invention has been made in view of the above, and even when a series of images is classified according to the similarity of images, it is easy to observe an image having a high necessity for observation.
• An object of the present invention is to provide an image display apparatus that can reduce the display time of an image with a low necessity and, as a result, can efficiently observe a series of images.
• an image display device is an image display device that sequentially displays a series of input images, and each image included in the series of images.
• Image classification means for classifying an image into one or more image groups according to the degree of correlation between the images, a feature image region having a predetermined feature is detected from the images, and the detected feature image region
• a feature image detecting means for detecting each of the feature images from the series of images, and extracting the feature image in each image group classified by the image classification means as a representative image representing each image group Representative image extracting means, and image display control means for controlling to sequentially display the representative images extracted by the representative image extracting means.
• the representative image extracting means selects at least a head image at the head of each image group in time series for each image group. It is characterized by extracting as a representative image.
• the image display device is generated between feature images consecutive in time series among a plurality of feature images detected by the feature image detecting means.
• Feature image selection means for calculating a change amount of a predetermined feature amount in the feature image region and selecting a feature representative image representing the plurality of feature images based on the calculated change amount
• the feature image detection means calculates the predetermined feature amount of each detected feature image region, and the feature image selection means calculates the change based on the feature amount calculated by the feature image detection means.
• the representative image extracting means extracts the feature representative image as a representative image from the feature images in each image group.
• the image display device is characterized in that, in the above invention, the predetermined feature amount is a position of a feature image region in the feature image.
• the image display control means when the image display control means displays the representative image that is the characteristic image, the image display device is characterized in the vicinity of the representative image. Control for displaying a mark indicating an image is performed.
• each of the images is an image of an inside of an organ, and the predetermined feature indicates a lesion inside the organ. It is a feature.
• the feature indicating the lesion is at least one of bleeding, discoloration, and shape abnormality.
• the series of images is generated using a capsule endoscope.
• the image display device of the present invention even when a series of images are classified according to the similarity of images, it is possible to easily observe an image having a high necessity for observation, and an image having a low necessity for observation. Display time can be reduced, and as a result, a series of images can be observed efficiently.
• FIG. 1 is a block diagram showing a configuration of an image display apparatus according to Embodiment 1 of the present invention.
• FIG. 2 is a flowchart showing a processing procedure performed by the image display apparatus shown in FIG.
• FIG. 3 is a flowchart showing a processing procedure of the grouping process shown in FIG.
• FIG. 4 is a flowchart showing a processing procedure of the target image detection processing shown in FIG. The
• FIG. 5 is a flowchart showing a processing procedure of representative image extraction processing shown in FIG. 2.
• FIG. 6 is a schematic diagram for explaining an example of a result of the representative image extraction process shown in FIG.
• FIG. 7 is a flowchart showing a processing procedure of the representative image display process shown in FIG.
• FIG. 8 is a diagram showing an example of a GUI screen displayed by the image display device shown in FIG.
• FIG. 9 is a block diagram showing a configuration of the image display apparatus according to the second embodiment of the present invention.
• FIG. 10 is a flowchart showing a processing procedure performed by the image display device shown in FIG.
• FIG. 11 is a flowchart showing a processing procedure of the target image detection processing shown in FIG.
• FIG. 12 is a flowchart showing a processing procedure of the target image selection processing shown in FIG.
• FIG. 13 is a schematic diagram for explaining an example of attention image selection processing shown in FIG.
• FIG. 14 is a schematic diagram for explaining an example of a result of the attention image selection process shown in FIG.
• FIG. 1 is a block diagram showing a configuration of an image display device 1 that is useful for this embodiment.
• an image display device 1 includes an image processing unit 2 that processes an image stored in a storage unit 5, an input unit 3 that receives input of various types of information, and a display unit 4 that displays various types of information.
• a storage unit 5 that stores various types of information, and a control unit 6 that controls processing and operation of each unit of the image display device 1.
• the image processing unit 2, the input unit 3, the display unit 4, and the storage unit 5 are electrically connected to the control unit 6.
• the image processing unit 2 includes an image processing control unit 2a, an image classification unit 2b, an attention image detection unit 2c, and a representative image extraction unit 2d.
• the image processing control unit 2a acquires an image from the storage unit 5, controls various image processing on the acquired image, and outputs and stores an image of the processing result in the storage unit 5.
• the image processing control unit 2a controls the image classification unit 2b, the attention image detection unit 2c, and the representative image extraction unit 2d to execute predetermined image processing.
• the image classification unit 2b calculates a correlation value between two time-series images acquired by the image processing control unit 2a, and sets an image to be processed as an existing or new image according to the calculated correlation value. By associating with a group and repeating this association process for all the series of images, this series of images is classified into one or more image groups. [0023] Specifically, the image classification unit 2b refers to a threshold value for the correlation value that is preliminarily input, and performs processing according to the magnitude relationship between the threshold value and the calculated correlation value. Is associated with an existing image group or a new image group, and the group number of the associated image group is added to the image to be processed.
• the image classification unit 2b calculates, for example, the normal correlation between the corresponding pixel values between two images as a correlation value. Also, the image classification unit 2b may obtain the color difference, luminance difference, etc. of each corresponding pixel as a correlation value based on each pixel value.
• the attention image detection unit 2c as the feature image detection means detects a feature image region having a predetermined feature from the image acquired by the image processing control unit 2a, and also has a feature having the detected feature image region. The image is detected as a noticed image that is a noticeable image during observation. The attention image detection unit 2c repeats this detection processing for all the series of images, thereby detecting all feature images as attention images. At this time, the attention image detection unit 2c adds attention image information indicating the attention image to each image detected as the attention image.
• the attention image detection unit 2c detects a feature image region by identifying a predetermined feature based on color information indicated by each pixel constituting the image, for example. Further, the attention image detection unit 2c may detect a feature image area based on various feature amounts such as a contour shape, a texture, and a density gradient as well as color information.
• the representative image extraction unit 2d extracts a representative image representing each image group from among the image groups classified by the image classification unit 2b. Specifically, the representative image extraction unit 2d extracts the target image in each image group as the representative image of each image group, and sets the first image in the time series in each image group as the representative image. And extract. Further, the representative image extraction unit 2d sets a low speed display rate for displaying the image at a low speed so that sufficient observation time is given to the representative image that is the target image, and the representative image that is the leading image is displayed. Set the normal display rate for displaying at a normal speed faster than the low speed display rate.
• the representative image extraction unit 2d extracts all the target images for each image group, and extracts only the first image in an image group that does not include the target image.
• the unit 2d may extract, for example, an image in a predetermined order, such as an image at the end in time series in the image group, as the representative image.
• Each representative image extracted by the representative image extraction unit 2d is output to the storage unit 5 by the image processing control unit 2a, and is stored in the representative image storage unit 5b which is a storage area for storing the representative image.
• the image processing control unit 2a instead of storing each representative image, the image processing control unit 2a newly stores only the group number, attention image information, and display rate of each representative image in association with the original image. Oh ,.
• the input unit 3 accepts input of an image to be processed by the image display device 1 and various processing information.
• the input unit 3 includes a communication interface such as USB, IEEE1394, etc., and accepts an image input from an external device.
• the input unit 3 includes various switches, input keys, a mouse, a touch panel, and the like.
• the input unit 3 may include an interface corresponding to a portable storage medium such as various memory cards, CDs, and DVDs, and may accept input of an image from the portable storage medium.
• the display unit 4 includes a liquid crystal display or the like, and displays various types of information including images.
• the display unit 4 displays an image stored in the storage unit 5 and a GUI (Graphical User Interface) screen that requests the operator of the image display device 1 to input various processing information.
• GUI Graphic User Interface
• the storage unit 5 is realized by a ROM that stores various processing programs and the like, and a RAM that stores a processing meter, processing data, and the like for each processing.
• the storage unit 5 includes an image storage unit 5a and a representative image storage unit 5b, which are storage areas for storing an externally input image and a representative image extracted by the representative image extraction unit 2d.
• the storage unit 5 may include various types of memory cards, CDs, DVDs, and other portable storage media as removable image storage units.
• the control unit 6 is realized by a CPU or the like that executes various processing programs stored in the storage unit 5.
• the control unit 6 includes an image display control unit 6a.
• the image display control unit 6a sets a series of representative images stored in the representative image storage unit 5b to each representative image. Based on the display rate, the display unit 4 is sequentially displayed. Further, when displaying the representative image that is the attention image, the image display control unit 6a performs control to display the attention image mark indicating the attention image in the vicinity of the representative image.
• This attention image mark is preliminarily stored in the storage unit 5, and the image display control unit 6a reads the attention image from the storage unit 5 when the attention image information is added to the representative image to be processed. Acquire the mark and display it with the image.
• FIG. 2 is a flowchart showing a processing procedure in which the image display device 1 processes and displays a series of images stored in the image storage unit 5a under the control of the control unit 6.
• the flowchart shown in FIG. 2 exemplifies a processing procedure for displaying a series of images generated by imaging the inside of an organ such as a extinct organ using a capsule endoscope (not shown).
• the image classification unit 2b performs a grouping process for classifying each image included in the series of images stored in the image storage unit 5a into an image group (step S101).
• the attention image detection unit 2c performs attention image detection processing for detecting the attention image from the series of images (step S103), and the representative image extraction unit 2d detects the attention image and the head in each classified image group.
• a representative image extraction process is performed to extract an image as a representative image! (Step S105), and the image display control unit 6a has a display rate set for each representative image with a series of extracted representative images.
• the control unit 6 ends the series of processes.
• FIG. 3 is a flowchart showing the processing procedure of the grouping process.
• the image classification unit 2b sets the second image as a processing target image, and calculates a correlation value with the previous image that is the first image and is the image at the previous time point in time series (step S 115), it is determined whether or not the calculated correlation value is larger than the input threshold value (step S117).
• the correlation value is smaller than the threshold value (step SI 17: No)
• the image classification unit 2b Then, it is determined that the similarity between the images is low, the variable G is incremented to update the group number (step S119), and the updated group number is set as the processing target image (step S121).
• step S117: Yes the image classification unit 2b determines that the similarity between the images is high, and the group number indicated by the current variable G in the processing target image. Is set (step S121). The image classification unit 2b sets the group number “0” for the first image.
• the image processing control unit 2a records the image with the group number set in the image storage unit 5a (step SI23), and the group number is set for all the images included in the series of images. It is determined whether or not it is correct (step S 125). When the group number is not set for all the images (step S125: No), the image processing control unit 2a performs control so that the processing from step S113 is repeated for the images that are not set. On the other hand, when the group number is set for all the images (step S 125: Yes), the image processing control unit 2a returns to step S101.
• step S101 an image having a high degree of similarity with the previous image is associated with the same image group as the previous image, and an image with a low degree of similarity is assigned a group number. Is associated with the new image group updated. As a result, a series of images are grouped into a single image group of similar images.
• FIG. 4 is a flowchart showing the processing procedure of the target image detection process.
• the image processing control unit 2a reads the first image in time series among the series of images set with the group number and stored in the image storage unit 5a (step S131), and the target image detection unit 2c From this read image, an image region indicating a bleeding site is detected as a feature image region (step S133), and it is determined whether or not the detected bleeding site has a force (step S135).
• the target image detection unit 2c may detect an image region that is more reddish than the mucous membrane inside the organ as the bleeding site.
• step S135 When there is a detected bleeding site (step S135: Yes), the attention image detection unit 2c adds attention image information to the image to be processed (step S137), and the image processing control unit 2a After recording the image with the attention image information added to the image storage unit 5a (step S139), It is determined whether or not a series of all images has been processed (step S141).
• step S141: No the image processing control unit 2a repeats the processing from step S131 on the unprocessed images to process all the images. If yes (step S141: Yes), return to step S103. If it is determined in step S135 that there is no detected bleeding site (step S135: No), the image processing control unit 2a immediately determines in step S141.
• an image obtained by imaging a bleeding site inside the organ can be detected as a attention image from a series of images.
• the image to be detected as a focus image is not limited to an image obtained by imaging a bleeding site.
• an image obtained by imaging various sites suspected to be lesions such as a faint site or an abnormally shaped site inside an organ is used as a target image. You may make it detect.
• the attention image detection unit 2c should detect an image area showing features such as a fading color and an abnormal shape as a feature image area.
• FIG. 5 is a flowchart showing a processing procedure of representative image extraction processing.
• the image processing control unit 2a reads the first image in time series among the series of images that have been subjected to grouping processing and target image detection processing and stored in the image storage unit 5a (step S 151), the representative image extraction unit 2d determines whether or not the read image has the power to which attention image information is added (step S153).
• step S153 When attention image information is added (step S153: Yes), the representative image extraction unit 2d sets a low-speed display rate for the image to be processed (step S155), and the image processing control unit 2 a records the attention image set with the low-speed display rate as a representative image in the representative image storage unit 5b (step S157).
• the representative image extraction unit 2d determines whether or not the image to be processed is the first image in the image group. Yes (Step S159). If it is the first image (step S159: Yes), the representative image extraction unit 2d sets the normal display rate for the image to be processed (step S161), and the image processing control unit 2a sets the normal display rate. In the representative image storage 5b Record (step SI 63).
• step S165 the image processing control unit 2a determines whether or not a series of all images has been processed (step S165), and if all the images have not been processed (step S165).
• step S 165: No process! /, Na! /, Repeat the process from step S151 on the image, and if all the images have been processed (step S165: Yes), return to step S105 . If it is determined in step S159 that the image to be processed is not the first image (step S159: No), the image processing control unit 2a immediately determines in step S165.
• the head image and the target image of each image group can be extracted as the representative image.
• the power of “normal image 1” as the first image and “bleeding image 1” to “bleeding image 3” from the image group n which is the nth image group in time series
• the “normal image 3” force that is the first image is extracted as a representative image from image group n + 1.
• the series of representative images extracted in this way are stored in the representative image storage unit 5b in time series.
• FIG. 7 is a flowchart showing the processing procedure of the representative image display processing.
• the image display control unit 6a reads the first image in time series from the series of representative images stored in the representative image storage unit 5b (step S171), and the noticed image information is added. It is determined whether or not (step S173).
• step S173: Yes When attention image information is added (step S173: Yes), the image display control unit 6a reads the attention image mark from the storage unit 5 (step S175), and reads the representative image and attention The image mark is displayed on the display unit 4 at the low speed display rate (step S177). As a result, the image display control unit 6a can display a representative image, which is a noticed image having a high necessity for observation, for a longer time than usual. On the other hand, when attention image information is not added (step S173: No), the image display control unit 6a displays the read representative image on the display unit 4 at the normal display rate (step S179).
• the image display control unit 6a determines whether or not all the series of representative images are displayed. (Step S181): If not all are displayed (Step S181: No), repeat the processing from Step S171 for V and representative images that are not displayed, and display all (Step S181: Yes), return to step S107. In this way, the image display control unit 6a sequentially displays a series of representative images stored in the representative image storage unit 5b according to the display rate set for each image.
• FIG. 8 is a diagram showing an example of a GUI screen displayed on the display unit 4 when displaying a representative image.
• the “diagnosis / diagnosis” window displays character information indicating various attributes of the representative image Pi, the attention image mark Ma, and the representative image Pi, which are the attention images.
• the image classification unit 2b performs time-series continuous image processing on a series of images stored in the image storage unit 5a.
• the correlation value is calculated, and a series of images are classified into image groups according to the calculated correlation value, and the target image detection unit 2c detects the feature image region from each image and also has the detected feature image region.
• the image is detected as an image of interest, and the representative image extraction unit 2d force extracts the image of interest and the first image in each image group as representative images, sets the display rate for each extracted representative image, and further controls image display Since the part 6a sequentially displays a series of representative images according to the set display rate, for example, it is easy to observe an image that includes a bleeding part and has a high necessity for observation, and It is possible to reduce the display time by limiting the display of images that do not include normal images that are less necessary to be observed to the first image of each image group, and as a result, efficiently observe a series of images. Can do.
• Embodiment 1 all the target images detected by the target image detection unit 2c are displayed as representative images.
• a plurality of target images exhibit the same characteristics. In this case, select one of the multiple images of interest as the representative image and display it!
• FIG. 9 is a block diagram showing a configuration of the image display device 11 according to the second embodiment.
• the image display device 11 includes an image processing unit 12 instead of the image processing unit 2 included in the image display device 1.
• the image processing unit 12 is an image provided in the image processing unit 2.
• An image processing control unit 12a and a target image detection unit 12c are provided in place of the processing control unit 2a and the target image detection unit 2c, and a target image selection unit 12e is newly provided.
• Other configurations are the same as those of the first embodiment, and the same components are denoted by the same reference numerals.
• the image processing control unit 12a acquires and processes the image stored in the storage unit 5, and stores the processing result image in the storage unit 5.
• the image processing control unit 12a controls the attention image detection unit 12c instead of the attention image detection unit 2c, and newly controls the attention image selection unit 12e, so that among the plurality of attention images showing the same characteristics, Only representative attention images are selected as representative images.
• the attention image detection unit 12c detects the attention image from the series of images in the same manner as the attention image detection unit 2c, and calculates and calculates the feature amount of the feature image area of each detected attention image.
• the information indicating the feature amount is associated with the target image.
• the attention image detection unit 12c calculates the position of the feature image area in the attention image as the feature amount, generates position information indicating the calculated position, and adds the position information to the attention image.
• the position of the feature image area is indicated by at least one of the position of the center of gravity of the feature image area, the maximum or minimum position of the brightness, the position having a predetermined hue, the outermost peripheral position of the feature image area, and the like.
• the attention image selection unit 12e is based on the position change of the feature image area that occurs between the attention images that are consecutive in time series from among the plurality of attention images detected by the attention image detection unit 12c. An attention representative image representing a plurality of attention images is selected. Specifically, the target image selection unit 12e calculates a motion vector indicating a change in the position of the feature image area that occurs between two consecutive target images, and based on the calculated motion vector, the two features It is determined whether or not the image area is an image area showing the same feature. If the image area is an image area showing the same feature, one of the attention images is not extracted as a representative image, but is changed to the attention cancellation image. The attention image selection unit 12e repeats this process for a series of attention images, and selects the attention image as the attention representative image that does not become the attention cancellation image as a result.
• FIG. 10 is a flowchart showing a processing procedure in which the image display device 11 processes and displays a series of images stored in the image storage unit 5a under the control of the control unit 6. Note that the flow chart shown in FIG. 10 uses a capsule endoscope (not shown) to image the inside of an organ such as the extinct organ. The process sequence which displays the produced
• the image classification unit 2b performs a grouping process similar to step S101 (step S201), and the target image detection unit 12c detects a target image from a series of images.
• Attention image detection processing is performed! (Step S203)
• the attention image selection unit 12e performs attention image selection processing for selecting a representative representative image from the attention images detected at Step S203 (Step S205)
• the representative image extraction unit 2d performs a representative image extraction process similar to step S105 (step S207)
• the image display control unit 6a performs a representative image display process similar to step S107 (step S209). 6 ends the series of processing.
• step S201 The processing procedures of the grouping process in step S201, the representative image extraction process in step S207, and the representative image display process in step S209 are shown by the flowcharts shown in FIGS. 3, 5, and 7, respectively.
• the attention image detection process in step S203 and the attention image selection process in step S205 will be described.
• FIG. 11 is a flowchart showing a processing procedure of the attention image detection process in step S203.
• the image processing control unit 12a reads the first image in time series among the series of images set with the group number and stored in the image storage unit 5a (step S211), and detects the target image.
• the unit 12c detects an image region indicating the bleeding site as a feature image region from the read image and generates position information indicating the position of the bleeding site (step S213).
• the target image detection unit 12c determines whether or not there is a detected bleeding site (step S215). If there is a detected bleeding site (step S215: Yes), the target image detection unit 12c The attention image information and the position information generated in step S213 are added to the image (step S217), and the image processing control unit 12a records the image added with the attention image information and the position information in the image storage unit 5a. (Step S219), it is determined whether or not all the series of images have been processed (Step S221).
• step S221: No the image processing control unit 12a repeats the processing from step S211 on the unprocessed image, and processed all the images. If yes (step S221: Yes), return to step S203. If it is determined in step S215 that there is no detected bleeding site (step S215: No ), The image processing control unit 12a immediately determines in step S221.
• an image obtained by imaging the bleeding site inside the organ is detected as a attention image from the series of images, and the bleeding image is detected in the detected attention image.
• Position information indicating the position can be added.
• the image region detected as the feature image region is not limited to the image region showing the bleeding site, but may be an image region showing various features suspected of being lesioned, such as a discolored site inside the organ or a shape abnormal site.
• FIG. 12 is a flowchart showing the processing procedure of the attention image selection process.
• the image processing control unit 12a reads the two images at the top and second in time series among the series of images that have been subjected to the target image detection process and stored in the image storage unit 5a (In step S231), the attention image selection unit 12e sets the second image as a processing target image, and adds attention image information together with the preceding image that is the first image and the previous time point image in time series. It is determined whether or not the power is correct (step S233).
• the attention image selection unit 12e refers to the position information indicating the position of the bleeding site, and the previous image
• the image area including the bleeding part is extracted as a bleeding part template that is a template for pattern matching processing (step S235), and the pattern matching process is performed on the processing target image based on the bleeding part template.
• a motion vector indicating a change in position of the bleeding site of the processing target image with respect to the previous image is detected (step S237).
• the attention image selection unit 12e extracts, as a peripheral part template, an image region of the peripheral part of the bleeding part that has a predetermined positional relationship with the bleeding part from the previous image (step S239). Based on the above, pattern matching processing is performed on the processing target image to detect an image region having a high correlation with the peripheral template, and a motion vector indicating the motion of the peripheral portion of the bleeding site of the processing target image with respect to the previous image is detected. Detect (step S241).
• the attention image selection unit 12e then detects the motion vector of the detected bleeding site and the vicinity of the bleeding site. Whether or not the motion vector is almost the same force (step S243), and if they are almost the same (step S243: Yes), the target image cancel information indicating that the target cancel image is the target image It is added (step S 245).
• the image processing control unit 12a records the processing target image to which attention image cancellation information is added in the image storage unit 5a (step S247), and determines whether or not all the series of images have been processed. (Step S249), if all the images have been processed! /, N! /, (Step S249: No), repeat the processing from Step S231 on the unprocessed images to process all the images. If yes (step S249: Yes), return to step S205.
• step S233 it is determined in step S243 that attention image information has been added (step S233: No)
• step S243 it is determined in step S243 that the motion vectors are not the same.
• step S243: No the image processing control unit 12a immediately makes the determination in step S249.
• step S237 and step S241 the target image selection unit 12e compares each template extracted from the previous image and the image area in the processing target image corresponding by pattern matching on the same screen, For example, a motion vector indicating a change in the center of gravity position of each image area is detected.
• the target image selection unit 12e may not perform pattern matching, and may obtain a motion vector only from the position coordinates of the bleeding part between the previous image and the processing target image.
• the attention image selection unit 12e determines, for example, the difference in direction and size of each motion vector between the detected motion vector of the bleeding site and the motion vector around the bleeding site. It is determined whether or not each motion vector has the same force depending on whether or not it is equal to or less than a preset threshold.
• the target image selection unit 12e calculates the vector difference, inner product, outer product, and the like of each movement outer frame, and each motion vector is the same based on at least one of these calculation results. It may be determined whether or not.
• step S245 the attention image selection unit 12e determines that the bleeding part of the processing target image and the previous image are the same when the motion vector of the bleeding part detected and the motion vector of the peripheral part of the bleeding part are substantially the same. Since it is unlikely that the image to be processed will be displayed during observation when there is a high possibility that the bleeding site is the same as the bleeding site, To do.
• the attention image selection unit 12e may detect a plurality of image regions as peripheral templates so that the identity of the bleeding site can be determined with high accuracy.
• step S205 the movement of the image area around the bleeding part and the peripheral part of the bleeding part is compared between the two consecutive attention images. It is possible to determine whether or not the bleeding sites have the same force. If they are the same, one attention image can be changed to a attention cancellation image. Note that the processing order of steps S235 to S241 may be changed as appropriate, for example, by exchanging the processing order of step S237 and step S239.
• FIG. 13 is a diagram illustrating an example of an image processed in the target image selection process.
• the attention image selection unit 12e extracts the image region TPal including the bleeding site BL1 in the previous image as a bleeding site template, and the image region located on the left and right of the bleeding site BL1 in the figure.
• TPbl and TP cl are extracted as peripheral templates. Thereafter, the attention image selection unit 12e performs template matching on the processing target image based on these templates.
• the attention image selection unit 12e corresponds respectively The motion vectors Va2, Vb2, and Vc2 indicating the motion of the center of gravity of the image area to be detected are detected, and it is determined whether or not the difference between the direction and size of each motion vector is below a predetermined threshold value.
• the target image selection unit 12e determines that the motion vectors are substantially the same, and the bleeding site BL2 of the processing target image is the same as the bleeding site BL1. Attention image cancellation information is added to the processing target image as a bleeding site.
• the target image selection unit 12e detects motion vectors Va3, Vb3, and Vc3, and each of these motion vectors It is determined whether or not the difference in direction and size is less than a predetermined threshold value.
• the target image selection unit 12e determines that each motion vector is a different motion vector, and the bleeding part BL3 in the processing target image is the bleeding part BL1. This processing target image is selected as a representative representative image as a different bleeding site.
• each image group force includes the first image and the attention representative image as representative images. Extracted.
• the nth image group that is the nth image group in time series “normal image 1” that is the first image and “bleeding image 1” that is the attention representative image
• the “normal image 3” force, which is the first image is extracted from group n + 1 as a representative image.
• the attention image detection unit 12 c selects the attention image from the series of images classified into image groups by the image classification unit 2 b. And the position of the feature image area of each detected target image is calculated, position information indicating the calculated position is added to the target image, and the target image selection unit 12e The generated feature image area and the position change of the image area around the feature image area are detected, and based on the detected position change, a representative representative image representing a plurality of similar target images is selected, and the representative image extraction unit 2d extracts the representative representative image and the leading image in each image group as representative images, sets a display rate for each extracted representative image, and further, the image display control unit 6a sets the set display rate.
• a series of representative images are sequentially displayed. For example, among images including bleeding sites, only images that have high similarity and need to be observed in time series can be displayed. At the same time, it is possible to reduce the display time by limiting the display of images that do not include the bleeding site and show a low level of observation to the normal state to only the first image of each image group, resulting in a series of images. Furthermore, it can observe efficiently.
• the image display control unit 6a sequentially displays the leading image force in time series among a series of representative images, and displays all the representative images in the series. For example, based on the instruction information about the display start image input in advance, the display of the representative image power in the middle is started in time series, and the display end is input in advance. Based on the instruction information about the image, display up to a representative image in the middle of time series and finish the image display process.
• image display control unit 6a has been described to display only representative images. For example, based on instruction information in which a predetermined switch equal force is also input, It may be possible to switch between the display of only the representative image and the display of all images.
• control unit 6 may perform the grouping process after the force-attention image detection process in which the attention image detection process is performed after the grouping process. Good. Similarly, in the second embodiment, the control unit 6 may perform the grouping process after the attention image detection process or the attention image selection process.
• the image display device according to the present invention is useful for an image display device that sequentially displays a series of input images, and particularly in a specimen using a capsule endoscope. Suitable for image display devices that display a series of captured images.

Landscapes

• Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Surgery (AREA)
• Physics & Mathematics (AREA)
• General Health & Medical Sciences (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Medical Informatics (AREA)
• Radiology & Medical Imaging (AREA)
• Animal Behavior & Ethology (AREA)
• Biomedical Technology (AREA)
• Heart & Thoracic Surgery (AREA)
• Pathology (AREA)
• Molecular Biology (AREA)
• Optics & Photonics (AREA)
• Biophysics (AREA)
• Public Health (AREA)
• Veterinary Medicine (AREA)
• Computer Vision & Pattern Recognition (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Quality & Reliability (AREA)
• Endoscopes (AREA)
• Closed-Circuit Television Systems (AREA)
• User Interface Of Digital Computer (AREA)
• Measuring And Recording Apparatus For Diagnosis (AREA)
• Image Analysis (AREA)
• Image Processing (AREA)
• Digital Computer Display Output (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Related Child Applications (2)

Publications (1)

Family

ID=37431038

Family Applications (1)

Country Status (5)

Cited By (5)

Families Citing this family (52)

Citations (3)

Family Cites Families (15)

• 2005
  • 2005-05-20 JP JP2005148670A patent/JP4418400B2/ja not_active Expired - Fee Related
• 2006
  • 2006-02-02 US US11/631,273 patent/US8830307B2/en active Active
  • 2006-02-02 WO PCT/JP2006/301795 patent/WO2006123455A1/ja not_active Ceased
  • 2006-02-02 EP EP06712938.7A patent/EP1882440A4/en not_active Withdrawn
  • 2006-02-02 CN CN200680015612A patent/CN100577088C/zh not_active Expired - Fee Related
• 2007
  • 2007-04-18 US US11/787,980 patent/US8502861B2/en not_active Expired - Fee Related

Patent Citations (3)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events