WO2016006284A1 - Image playback control device and image playback control method - Google Patents

Image playback control device and image playback control method Download PDF

Info

Publication number
WO2016006284A1
WO2016006284A1 PCT/JP2015/059361 JP2015059361W WO2016006284A1 WO 2016006284 A1 WO2016006284 A1 WO 2016006284A1 JP 2015059361 W JP2015059361 W JP 2015059361W WO 2016006284 A1 WO2016006284 A1 WO 2016006284A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
display
series
blink
Prior art date
Application number
PCT/JP2015/059361
Other languages
French (fr)
Japanese (ja)
Inventor
律郎 西村
Original Assignee
オリンパス株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オリンパス株式会社 filed Critical オリンパス株式会社
Publication of WO2016006284A1 publication Critical patent/WO2016006284A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions

Definitions

  • the present invention relates to an image reproduction control device and an image reproduction control method for reproducing an image on a connected display device.
  • a capsule endoscope is an apparatus having an imaging function and a wireless communication function inside a capsule-type housing. After being swallowed from the mouth of the subject, the capsule endoscope moves inside the digestive tract by a peristaltic motion or the like. The inside of the organ is sequentially imaged to acquire image data, and wirelessly transmitted to a receiving device outside the subject. The receiving device sequentially receives the image data transmitted from the capsule endoscope, and sequentially records the image data on a portable recording medium.
  • the image display device takes in the image data recorded on the recording medium, performs predetermined image processing, and then plays back the in-vivo images corresponding to the taken-in image data by switching one frame at a time (frame playback).
  • a doctor or the like who is an operator diagnoses the subject by checking the in-vivo image reproduced by the image display device.
  • the present invention has been made in view of the above, and an object of the present invention is to provide an image reproduction control device and an image reproduction control method capable of preventing oversight of reproduced images and reducing the operation burden on the operator. To do.
  • an image reproduction control device for reproducing an image on a connected display device, and a series of images are displayed on a display screen of the display device.
  • a display control unit that sequentially reproduces the images, and a detection unit that detects whether an operator of the image reproduction control device is viewing the display screen with the eyes.
  • the detection unit detects that the operator does not see the display screen with eyes during image reproduction, at least the operator uses the eyes to display the display screen with the reproduction positions of the series of images.
  • the display device is returned to the time point when it is estimated that it is no longer seen, and the series of images is regenerated on the display device.
  • the detection unit is a blink detection unit that detects an end time of the operator's blink
  • the display control unit is an blink end time detected by the detection unit.
  • the display control unit is based on a predetermined time corresponding to the time required for the one blink and a frame rate in the series of image reproduction processing.
  • the number of frames of the image reproduced on the display device during the predetermined time is obtained, and the reproduction position is returned at least by the obtained number of frames from the blinking end time detected by the detection unit, and is reproduced again. It is characterized by.
  • the display control unit has a fixed time longer than a predetermined time corresponding to the time required for one blink from the end time of the blink detected by the detection unit.
  • the reproduction position is returned and the series of images is regenerated again.
  • the detection unit detects a start time and an end time in a state where the operator cannot see the display screen with eyes, and the display control unit The reproduction position of the series of images is returned and regenerated until the start time in which the operator detects the display screen with his / her eyes detected by the detection unit.
  • the detection unit is a blink detection unit that detects a blink start time and an end time of the operator, and the display control unit is detected by the detection unit. Based on the period between the blink start time and end time and the frame rate in the series of image reproduction processing, the image was reproduced on the display device while the operator's eyes were blinking. The number of frames of the image is obtained, and the reproduction position is returned at least by the obtained number of frames from the end time detected by the detection unit, and the series of images is regenerated again.
  • the display control unit sets the reproduction position until the image reproduced on the display device is estimated to be at least when the operator stops viewing the display screen with eyes.
  • the display device is made to regenerate the series of images on a screen including notification information for notifying that the image has been regenerated and regenerated again.
  • the series of images are images captured by a capsule endoscope, and the display control unit is connected to the display device by the capsule endoscope.
  • the captured image is played back.
  • the image reproduction control method is an image reproduction control method performed by an image reproduction control device that reproduces an image on a connected display device, and sequentially reproduces a series of images on the display screen of the display device.
  • the detection step of detecting whether or not the operator of the image reproduction control device is viewing the display screen with eyes during the reproduction of the series of images, and in the detection step, the operator When it is detected that the display screen is not seen with the eyes, the playback position of the series of images is returned to at least the time when it is estimated that the operator no longer sees the display screen with the eyes.
  • An image regeneration step for reproducing the series of images again.
  • the playback position of the series of images is returned to at least the time when it is estimated that the operator has stopped looking at the display screen with the eyes. Since the display device automatically reproduces a series of images, it is possible to realize both prevention of overlooking of the reproduced image and reduction of the operation burden on the operator.
  • FIG. 1 is a schematic diagram illustrating a schematic configuration of a capsule endoscope system according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram schematically showing the configuration of the image reproduction control apparatus shown in FIG.
  • FIG. 3 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG.
  • FIG. 4 is a diagram for explaining the return position setting process shown in FIG.
  • FIG. 5 is a diagram illustrating an example of a display screen of the display device in the first embodiment.
  • FIG. 6 is a diagram illustrating another example of the display screen of the display device according to the first embodiment.
  • FIG. 7 is a block diagram of the configuration of the image reproduction control apparatus according to the second embodiment.
  • FIG. 1 is a schematic diagram illustrating a schematic configuration of a capsule endoscope system according to the first embodiment of the present invention.
  • FIG. 2 is a block diagram schematically showing the configuration of the image reproduction control apparatus shown in FIG.
  • FIG. 3 is a flow
  • FIG. 8 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG.
  • FIG. 9 is a diagram for explaining the return position setting process shown in FIG.
  • FIG. 10 is a block diagram of another configuration of the image reproduction control apparatus according to the second embodiment.
  • an image reproduction control device that causes a display device to reproduce an image captured by a capsule endoscope will be described as a mode for carrying out the present invention (hereinafter referred to as “embodiment”). Moreover, this invention is not limited by this embodiment. Furthermore, the same code
  • FIG. 1 is a schematic diagram showing a schematic configuration of a capsule endoscope system according to Embodiment 1 of the present invention.
  • a capsule endoscope system 1 according to Embodiment 1 includes a capsule endoscope 2, a receiving antenna unit 3, a receiving device 4, a cradle 5, an image reproduction control device 10, and a display device. 20.
  • the capsule endoscope 2 moves through the digestive tract of the subject H by peristaltic movement of the organ and the like, and moves a predetermined part of the living body (esophagus, stomach, small intestine, large intestine, etc.)
  • the image data is acquired in sequence with a period of (for example, a 0.5 second period) to acquire image data, and is superimposed on the radio signal and transmitted.
  • the receiving antenna unit 3 receives the radio signal transmitted from the capsule endoscope 2 via the plurality of receiving antennas 3a to 3h attached to the subject H.
  • the receiving device 4 performs predetermined signal processing on the image data received via the receiving antenna unit 3 and records it in a memory (not shown).
  • the image reproduction control device 10 is configured using a workstation, takes in the image data acquired by the capsule endoscope 2 from the memory of the receiving device 4 via the cradle 5, and uses the image data for the subject. Create an image in H.
  • the image reproduction control device 10 causes the created image in the subject H to be stored in a storage device (not shown), and is also reproduced on a display device 20 such as a liquid crystal display connected to the image reproduction control device 10.
  • the display device 20 is attached with an imaging device 14a that faces the display screen of the display device 20 and captures the face of the operator who operates the image reproduction control device 10, and the operation captured by the imaging device 14a.
  • the image showing the person's face is output to the image reproduction control device 10.
  • the image reproduction control apparatus 10 is not limited to a workstation, and may be configured by a mobile device such as a laptop PC or a tablet terminal, or a wearable terminal such as a glasses type or a clock type.
  • FIG. 2 is a block diagram showing a configuration of the image reproduction control apparatus 10 shown in FIG.
  • the display device 20 and the storage device 30 to which the image reproduction control device 10 is connected are also shown for explanation.
  • the storage device 30 stores a plurality of folders Fa for storing the image group Gg.
  • the image group Gg is an image captured by the capsule endoscope 2 and is composed of a series of images that are continuous in time series.
  • the image reproduction control device 10 includes an input unit 11, a data transmission / reception unit 12, a control unit 13, a blink detection unit 14, an image processing unit 15, a display control unit 16, an output unit 18, and a storage unit. 19
  • the input unit 11 is realized by an input device such as a keyboard, a mouse, a touch panel, and various switches.
  • the input unit 11 receives input of information and commands according to the operation of the operator.
  • the data transmission / reception unit 12 is an interface that can be connected to a storage medium or a communication line such as a wired LAN or a wireless LAN.
  • the data transmission / reception unit 12 is connected to the storage device 30 based on the control of the control unit 13 and receives image data of the image group Gg captured by the capsule endoscope 2 stored in the storage device 30.
  • the control unit 13 is realized by hardware such as a CPU, and reads various programs stored in the storage unit 19, so that signals input via the input unit 11 and image data input from the data transmission / reception unit 12 are read. Based on the above, instructions to each unit constituting the image reproduction control device 10 and data transfer are performed to control the overall operation of the image reproduction control device 10 in an integrated manner.
  • the blink detection unit 14 is one of detection units that detect eye movement such as eyelid movement, pupil movement, and focal position change.
  • the operator of the image reproduction control device 10 displays the display device 20 with the eyes. Detect if you are looking at the screen. Specifically, the blink detection unit 14 detects the end time of the operator's blink and outputs the detection result to the control unit 13.
  • the blink detection unit 14 processes the image of the face of the operator imaged by the imaging device 14a to detect the presence or absence of blinking eyes. When blinking eyes are detected, blinking indicating the end time of blinking eyes is detected. A detection signal is output.
  • the blink detection unit 14 detects, for example, the presence or absence of blinks of the eyes by extracting a region where the eyes appear from an image captured by the imaging device 14a and performing pattern matching with a standard eye pattern.
  • the blink detection unit 14 acquires grayscale data of a plurality of horizontal lines for a region in which an eye is captured in an image captured by the imaging device 14a, and white (white eye) is obtained by binarizing the grayscale data. And the black (black eye) part are separated, and the presence or absence of blinking eyes is detected based on the presence or absence of the black part and the size of the black part.
  • the image processing unit 15 performs optical black subtraction processing, gain adjustment processing, image data synchronization processing, gamma correction processing, white balance (WB) adjustment processing on the image data of the image group Gg received by the data transmitting / receiving unit 12.
  • Image processing such as color matrix calculation processing, color reproduction processing, and edge enhancement processing is performed.
  • the display control unit 16 generates display image data to be displayed on the display device 20 from the image data of the image group processed by the image processing unit 15 or the image data of the image group received from the storage device 30. .
  • the display control unit 16 converts display image data from a digital signal to an analog signal, changes the converted analog signal image data to a format such as a high-definition method, and then outputs the converted image data to the display device 20. Thereby, the display control unit 16 causes the display screen of the display device to sequentially reproduce a series of images in time series. Note that the series of images may be rearranged and sequentially reproduced based on the position of the capsule endoscope 2 when the series of images are captured.
  • the display control unit 16 determines at least the playback position for the series of images displayed on the display device 20 when the operator The display device 20 is returned to the time point when it is estimated that the display screen is no longer seen, and a series of images are regenerated on the display device 20.
  • the display control unit 16 includes a return position setting unit 17 that sets the return position of the reproduction position of a series of images when the blink detection unit 14 detects that the operator is not looking at the display screen with eyes.
  • the output unit 18 is realized using a speaker, a printer, or the like, and outputs information related to the reproduction processing of the image data captured by the capsule endoscope 2 under the control of the control unit 13.
  • the storage unit 19 is realized using a volatile memory or a non-volatile memory, and stores various programs for operating the image reproduction control device 10.
  • the storage unit 19 temporarily stores information being processed by the image reproduction control device 10.
  • the storage unit 19 may be configured using a memory card or the like attached from the outside of the image reproduction control device 10.
  • FIG. 3 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus 10 shown in FIG.
  • the display control unit 16 When information for instructing the start of image reproduction is input together with the image data of the image group to be reproduced (step S1), the display control unit 16 generates display image data for the image data to be reproduced, and displays the display device.
  • a reproduction process (step S2) for reproducing on the 20 display screens is performed.
  • the blink detection unit 14 Under the control of the control unit 13, the blink detection unit 14 performs blink detection processing for detecting blinks in the eyes of the operator of the image reproduction control device 10 (step S ⁇ b> 3), and the operator of the image reproduction control device 10 observes the eyes. It is detected whether or not the display screen of the display device 20 is being viewed.
  • the blink detection unit 14 detects the blink of the eye of the operator, the blink detection unit 14 outputs a blink detection signal indicating the end time of the blink of the eye to the display control unit 16 via the control unit 13.
  • the display control unit 16 determines whether or not the blink detection unit 14 has detected the blink of the eyes of the operator of the image reproduction control device 10 based on whether or not the blink detection signal is input (step S4).
  • the display position setting unit 17 is input. Based on the blink detection signal, return position setting processing for setting the return position of the reproduction position of the image group is performed (step S5).
  • the return position setting unit 17 sets the return position of the playback position of the image group at least when it is estimated that the operator has stopped viewing the display screen with his eyes.
  • FIG. 4 is a diagram for explaining the return position setting process shown in FIG. (1) of FIG. 4 is a timing chart regarding blink detection processing by the blink detection unit 14.
  • (2) of FIG. 4 is a schematic diagram of the time bar B indicating the reproduction position in the image group of the image that is actually reproduced on the display device 20.
  • the blink detection unit 14 When the blink detection unit 14 detects the blink of the eyes of the operator, the blink detection unit 14 outputs a blink detection signal Sa indicating the blinking end time ta of the eyes to the display control unit 16 via the control unit 13.
  • the return position setting unit 17 sets the time required for one blink as indicated by an arrow Yb from the reproduction position Da corresponding to the blink end time ta included in the input blink detection signal Sa. The return position is set so that the reproduction position is returned to the position Db returned by the corresponding predetermined time Tb. Since the human blinking time is about 0.10 to 0.15 (seconds), the return position setting unit 17 determines from the reproduction position Da corresponding to the blinking end time included in the input blink detection signal. Return the playback position by 0.15 (seconds).
  • the return position setting unit 17 sets the playback position to a position Dc that is returned from the playback position Da by a fixed time Tc (> Tb) that is sufficiently longer than the time required for one blink. It may be set to return.
  • the return position setting unit 17 also determines a predetermined time corresponding to the time required for one blink based on the predetermined time corresponding to the time required for one blink and the frame rate in the reproduction processing of the image group.
  • the number of frames of the image reproduced on the display device 20 during the time is obtained, and the reproduction position is returned by at least the obtained number of frames from the blink end time detected by the blink detection unit 14. For example, assuming that the human blinking time is 0.10 (seconds) and the frame rate in the image group reproduction processing is F (frames / second), reproduction using the following equation (1):
  • the number of frames for returning the reproduction position is obtained as 6 frames according to the equation (1). Return the playback position for 6 frames or more.
  • the display control unit 16 returns the reproduction position of the image group to the position set by the return position setting unit 17, and performs re-generation processing for causing the display device 20 to reproduce the image of the image group again (step S6).
  • the display control unit 16 returns the reproduction position to the time when it is estimated that the image reproduced on the display screen of the display device 20 together with the image to be regenerated is at least the point where the operator can no longer see the display screen with eyes. Then, the image group is reproduced again on the display device 20 on the screen including the notification information for notifying that the image is reproduced again.
  • FIG. 5 is a diagram illustrating an example of a display screen of the display device 20 when a re-live image is displayed.
  • a playback menu M for playing back a capsule endoscope image is displayed on the display screen of the display device 20.
  • an area Ra is provided in the upper left part
  • an area Rb is provided in the lower left part
  • an area Rc is provided in the lower part
  • an area Rd is provided in the right part.
  • Subject (patient) information (for example, name, ID, date of birth, etc.) is displayed in the region Ra, an icon image for controlling image display is displayed in the region Rb, and the region Rc actually A time bar B having a slider Bn indicating the position of the image being reproduced in the entire image group is displayed, and an image G captured by the capsule endoscope 2 is reproduced in the region Rd.
  • the display control unit 16 regenerates the image G displayed in the region Rd by surrounding it with a frame W having a color different from that at the time of reproduction or a thickness W different from that at the time of reproduction. Indicates that the image has been reproduced again with the playback position returned to at least the time when it is estimated that the operator has stopped viewing the display screen with his eyes.
  • the display control unit 16 estimates that the image being reproduced is at least near the periphery of the image G displayed in the region Rd, and at least the operator does not see the display screen with eyes.
  • the icon image A may be displayed indicating that the image has been reproduced again after returning to the reproduction position.
  • the display control unit 16 may enlarge and display a portion corresponding to the re-occurring section in the time bar B of the region Rc, or may display it with a color different from the others.
  • the display control unit 16 determines whether or not the image regenerated on the display device 20 is the final image of the image to be regenerated (step S7). When the display control unit 16 determines that the image regenerated on the display device 20 is not the final image of the image to be regenerated (step S7: No), the display control unit 16 returns to step S6 to regenerate the image. Continue. On the other hand, when the display control unit 16 determines that the image regenerated on the display device 20 is the final image of the image to be regenerated (step S7: Yes), the display control unit 16 returns to step S2 to display the image group.
  • the image reproduction process is performed on the next image data in. That is, the display control unit 16 performs a normal image reproduction process on the image data next to the image reproduced at the blink end time detected by the blink detection unit 14.
  • step S4 determines that the blink detection unit 14 has not detected the blink of the eyes of the operator of the image reproduction control device 10 (step S4: No).
  • step S8 determines that the reproduced image is not the final image of the image group instructed to be reproduced (step S8: No)
  • step S8: No the display control unit 16 returns to step S2 and reproduces the next image data in the image group. Process. Further, when the display control unit 16 determines that the reproduced image is the final image of the image group instructed to be reproduced (step S8: Yes), the reproduction process for the image group is ended.
  • the display control unit 16 performs at least one blink from the blink end time detected by the blink detection unit 14.
  • the reproduction position in the image group is returned for a predetermined time corresponding to the time required, and the display device 20 is made to reproduce the image group again.
  • the image is automatically regenerated by returning to the image before blinking. Can be prevented, and the operation burden on the operator can be significantly reduced.
  • blinking is an unconscious physiological reaction, it is difficult for the operator to notice the blink and return the playback position to the image that was missed by blinking. Even if there is an image that the operator has unknowingly missed, the reproduction position can be automatically returned to the image before blinking, so that it is possible to reliably prevent the reproduction image from being overlooked.
  • the frame W and the icon image A are displayed to notify that the image is a re-reproduction image, so that the operator may have missed the image that is actually being reproduced. Can be recognized.
  • FIG. 7 is a block diagram of the configuration of the image reproduction control apparatus according to the second embodiment.
  • the image reproduction control device 210 is different from the image reproduction control device 10 in that the control unit 13, the blink detection unit 14, and the display control unit 16 are replaced with the control unit 13. Are provided with a control unit 213, a blink detection unit 214, and a display control unit 216 having the same functions.
  • the blink detection unit 214 detects a start time and an end time in a state where the operator cannot see the display screen with the eyes due to the blink, and when a blink of the eyes is detected, the blink start time and the end time A blink detection signal indicating that is output.
  • the display control unit 216 returns the playback position of a series of images until at least the start time when the operator detected by the blink detection unit 214 stops looking at the display screen with eyes, that is, the blink start time.
  • the display control unit 216 includes a return position setting unit 217 that sets the return position of the reproduction position of the image group to the blink start time.
  • FIG. 8 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG.
  • Step S11 and step S12 are step S1 and step S2 shown in FIG.
  • the blink detection unit 214 performs blink detection processing for detecting whether or not the operator of the image reproduction control device 210 blinks under the control of the control unit 213 (step S13).
  • the blink detection signal is output to the display control unit 216 via the control unit 213.
  • the blink detection signal indicates the blink start time and end time.
  • Step S14 is step S4 shown in FIG.
  • the return position setting unit 217 receives the input blink. Based on the detection signal, return position setting processing for setting the return position of the reproduction position of the image group is performed (step S15). The return position setting unit 217 sets the return position of the reproduction position of the image group to a position corresponding to the blink start time.
  • FIG. 9 is a diagram for explaining the return position setting process shown in FIG. (1) of FIG. 9 is a timing chart regarding blink detection processing by the blink detection unit 214.
  • (2) of FIG. 9 is a schematic diagram of the time bar B indicating the reproduction position in the image group of the image currently displayed on the display device 20.
  • the blink detection unit 214 detects the blink of the eye of the operator, the blink detection signals Sd and Se indicating the blink start time td and the end time te are displayed via the control unit 213.
  • the return position setting unit 217 performs the blink start time td as indicated by the arrow Yd from the reproduction position De corresponding to the blink end time te included in the input blink detection signals Sd and Se.
  • the return position is set so that the playback position is returned to the corresponding position Dd.
  • Step S16 is step S6 shown in FIG. As a result, on the display device 20, the image that was reproduced during the blinking time Te (see FIG. 8) is reproduced again.
  • steps S17 and S18 are steps S7 and S8 shown in FIG.
  • the reproduction position of the image group is returned from the reproduction position corresponding to the blink end time detected by the blink detection unit 214 to at least the position corresponding to the blink start time, and the image is displayed on the display device. Since the group is automatically regenerated, both the oversight of the reproduced image and the reduction of the operation burden on the operator can be realized as in the first embodiment. Even if you have not seen the screen for a longer time than blinking, such as when you closed your eyes for a while, the playback position of the image group is returned to the position corresponding to the time when you closed your eyes, and the image group is automatically displayed on the display device. Therefore, similar to the first embodiment, both oversight of the reproduced image and reduction of the operation burden on the operator can be realized.
  • the line-of-sight detection unit 214A that detects the direction of the operator's line of sight. (One of the detection units described above) may detect whether the operator is viewing the display screen with his / her eyes.
  • the line-of-sight detection unit 214A processes the image of the operator's face captured by the imaging device 14a, and detects the direction of the line of sight of the operator.
  • the line-of-sight detection unit 214A detects whether the operator's line of sight is directed to the image reproduced on the display screen, and the display control unit 216 does not point the operator's line of sight to the image reproduced on the display screen. When the line-of-sight detection unit 214A detects this, the reproduction position may be returned to the point where the operator's line of sight no longer turns to the image reproduced on the display screen, and the image group may be regenerated. Further, the line-of-sight detection unit 214A is further provided with a mechanism capable of detecting the focus of the operator, detects whether the operator is focused on the image reproduced on the display screen, and is reproduced on the display screen. If the operator is out of focus, the playback position may be returned to the point in time when the focus is lost, and the image group may be regenerated.
  • the image reproduction control device 10 may include both the blink detection unit 14 and the line-of-sight detection unit 214A, and detects whether the operator is viewing the display screen with his eyes by combining these detection results. Also good.
  • An execution program for each process executed by the image reproduction control apparatuses 10, 210, and 210A according to the first and second embodiments is an installable format or an executable format file, such as a CD-ROM, a flexible disk, It may be configured to be recorded on a computer-readable recording medium such as a CD-R or DVD, and provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. You may comprise. Further, it may be configured to be provided or distributed via a network such as the Internet.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Endoscopes (AREA)

Abstract

The image playback control device 10 of the present invention is provided with a display control unit 16 for sequentially playing a series of images on a display screen of a display device 20, and a blink detection unit 14 for detecting whether or not an operator of the image playback control device 10 is watching the display screen during playback of the series of images. When the blink detection unit 14 detects that the operator is not watching the display screen, the display control unit 16 causes the display device 20 to replay the series of images by returning the playback position of the series of images to at least a point in time at which the operator is estimated to have stopped watching the display screen.

Description

画像再生制御装置および画像再生制御方法Image reproduction control apparatus and image reproduction control method
　本発明は、接続する表示装置に画像を再生させる画像再生制御装置および画像再生制御方法に関する。 The present invention relates to an image reproduction control device and an image reproduction control method for reproducing an image on a connected display device.
　内視鏡の分野においては、患者等の被検体の消化管内に導入可能な大きさに形成されたカプセル型内視鏡の開発が進められている。カプセル型内視鏡は、カプセル型筐体の内部に撮像機能及び無線通信機能を備えた装置であり、被検体の口から飲み込まれた後、蠕動運動等によって消化管内を移動しながら被検体の臓器内部を順次撮像して画像データを取得し、被検体外部の受信装置に無線送信する。受信装置は、カプセル型内視鏡から送信された画像データを順次、受信するとともに、可搬型の記録媒体に順次、記録させる。画像表示装置は、記録媒体に記録された画像データを取り込み、所定の画像処理を施した後に、取り込んだ画像データに対応する体内画像を１フレームずつ切り替えて再生（コマ再生）する。操作者である医師等は、画像表示装置によって再生された体内画像を確認することで、被検体を診断する。 In the field of endoscopes, development of capsule endoscopes having a size that can be introduced into the digestive tract of a subject such as a patient is underway. A capsule endoscope is an apparatus having an imaging function and a wireless communication function inside a capsule-type housing. After being swallowed from the mouth of the subject, the capsule endoscope moves inside the digestive tract by a peristaltic motion or the like. The inside of the organ is sequentially imaged to acquire image data, and wirelessly transmitted to a receiving device outside the subject. The receiving device sequentially receives the image data transmitted from the capsule endoscope, and sequentially records the image data on a portable recording medium. The image display device takes in the image data recorded on the recording medium, performs predetermined image processing, and then plays back the in-vivo images corresponding to the taken-in image data by switching one frame at a time (frame playback). A doctor or the like who is an operator diagnoses the subject by checking the in-vivo image reproduced by the image display device.
　近年では、画像の見落としを防ぐために、医師等に対する視線検出の結果をもとに、見落とした可能性がある画像を判別して、画像の見落としを報知するとともに、見落とした可能性がある画像の識別番号を報知する診断支援方法が提案されている（たとえば、特許文献１参照）。 In recent years, in order to prevent oversight of an image, an image that may have been overlooked is determined based on the result of eye gaze detection for a doctor, etc. A diagnosis support method for notifying an identification number has been proposed (see, for example, Patent Document 1).
特開２０１０－３５７５６号公報JP 2010-35756 A
　特許文献１記載の方法では、医師等は、見落とした画像を再度再生させるために、この見落とした画像を全画像の中から選択し、改めて再生を指示するという操作を行わなければならない。しかしながら、再生対象の画像がカプセル型内視鏡によって撮像された画像である場合、膨大な枚数の画像の中から見落とした画像を選択し、再度再生を指示するという煩雑な操作が、医師等の大きな負担となっていた。 In the method described in Patent Document 1, a doctor or the like must perform an operation of selecting an overlooked image from all the images and instructing the reproduction again in order to reproduce the overlooked image again. However, when the image to be reproduced is an image captured by a capsule endoscope, a complicated operation of selecting an overlooked image from a huge number of images and instructing reproduction again is performed by a doctor or the like. It was a big burden.
　本発明は、上記に鑑みてなされたものであって、再生画像の見落とし防止と、操作者の操作負担の軽減とを可能とした画像再生制御装置および画像再生制御方法を提供することを目的とする。 The present invention has been made in view of the above, and an object of the present invention is to provide an image reproduction control device and an image reproduction control method capable of preventing oversight of reproduced images and reducing the operation burden on the operator. To do.
　上述した課題を解決し、目的を達成するために、本発明にかかる画像再生制御装置は、接続する表示装置に画像を再生させる画像再生制御装置であって、前記表示装置の表示画面に、一連の画像を順次再生させる表示制御部と、当該画像再生制御装置の操作者が目で前記表示画面を見ているか否かを検出する検出部と、を備え、前記表示制御部は、前記一連の画像の再生中に、前記検出部が、前記操作者が目で前記表示画面を見ていないと検出した場合に、前記一連の画像の再生位置を、少なくとも前記操作者が目で前記表示画面を見なくなったと推定される時点まで戻して前記表示装置に前記一連の画像を再々生させることを特徴とする。 In order to solve the above-described problems and achieve the object, an image reproduction control device according to the present invention is an image reproduction control device for reproducing an image on a connected display device, and a series of images are displayed on a display screen of the display device. A display control unit that sequentially reproduces the images, and a detection unit that detects whether an operator of the image reproduction control device is viewing the display screen with the eyes. When the detection unit detects that the operator does not see the display screen with eyes during image reproduction, at least the operator uses the eyes to display the display screen with the reproduction positions of the series of images. The display device is returned to the time point when it is estimated that it is no longer seen, and the series of images is regenerated on the display device.
　また、本発明にかかる画像再生制御装置は、前記検出部は、前記操作者の瞬きの終了時間を検出する瞬き検出部であり、前記表示制御部は、前記検出部が検出した瞬きの終了時間から、少なくとも、一回の瞬きに要する時間に対応する所定の時間分、前記一連の画像の再生位置を戻して再々生させることを特徴とする。 Further, in the image reproduction control device according to the present invention, the detection unit is a blink detection unit that detects an end time of the operator's blink, and the display control unit is an blink end time detected by the detection unit. From the above, the reproduction position of the series of images is returned and reproduced again at least for a predetermined time corresponding to the time required for one blink.
　また、本発明にかかる画像再生制御装置は、前記表示制御部は、前記一回の瞬きに要する時間に対応する所定の時間と、前記一連の画像の再生処理におけるフレームレートとをもとに、前記所定の時間の間に前記表示装置で再生された画像のフレーム数を求め、前記検出部が検出した瞬きの終了時間から、少なくとも前記求めたフレーム数分、再生位置を戻して再々生させることを特徴とする。 Further, in the image reproduction control device according to the present invention, the display control unit is based on a predetermined time corresponding to the time required for the one blink and a frame rate in the series of image reproduction processing. The number of frames of the image reproduced on the display device during the predetermined time is obtained, and the reproduction position is returned at least by the obtained number of frames from the blinking end time detected by the detection unit, and is reproduced again. It is characterized by.
　また、本発明にかかる画像再生制御装置は、前記表示制御部は、前記検出部が検出した瞬きの終了時間から、前記一回の瞬きに要する時間に対応する所定の時間よりも長い一定の時間分、再生位置を戻して前記一連の画像を再々生させることを特徴とする。 Further, in the image reproduction control device according to the present invention, the display control unit has a fixed time longer than a predetermined time corresponding to the time required for one blink from the end time of the blink detected by the detection unit. Thus, the reproduction position is returned and the series of images is regenerated again.
　また、本発明にかかる画像再生制御装置は、前記検出部は、前記操作者が目で前記表示画面を見なくなった状態の開始時間と終了時間とを検出し、前記表示制御部は、少なくとも前記検出部が検出した前記操作者が目で前記表示画面を見なくなった状態の開始時間まで、前記一連の画像の再生位置を戻して再々生させることを特徴とする。 Further, in the image reproduction control device according to the present invention, the detection unit detects a start time and an end time in a state where the operator cannot see the display screen with eyes, and the display control unit The reproduction position of the series of images is returned and regenerated until the start time in which the operator detects the display screen with his / her eyes detected by the detection unit.
　また、本発明にかかる画像再生制御装置は、前記検出部は、前記操作者の瞬きの開始時間と終了時間とを検出する瞬き検出部であり、前記表示制御部は、前記検出部が検出した瞬きの開始時間と終了時間との間の期間と、前記一連の画像の再生処理におけるフレームレートとをもとに、前記操作者の目が瞬きを行っていた間に前記表示装置で再生された画像のフレーム数を求め、前記検出部が検出した終了時間から、少なくとも前記求めたフレーム数分、再生位置を戻して前記一連の画像を再々生させることを特徴とする。 In the image reproduction control device according to the present invention, the detection unit is a blink detection unit that detects a blink start time and an end time of the operator, and the display control unit is detected by the detection unit. Based on the period between the blink start time and end time and the frame rate in the series of image reproduction processing, the image was reproduced on the display device while the operator's eyes were blinking. The number of frames of the image is obtained, and the reproduction position is returned at least by the obtained number of frames from the end time detected by the detection unit, and the series of images is regenerated again.
　また、本発明にかかる画像再生制御装置は、前記表示制御部は、前記表示装置で再生されている画像が、前記少なくとも操作者が目で表示画面を見なくなったと推定される時点まで再生位置を戻して再々生された画像であることを報知する報知情報を含ませた画面で前記表示装置に前記一連の画像を再々生させることを特徴とする。 Further, in the image reproduction control device according to the present invention, the display control unit sets the reproduction position until the image reproduced on the display device is estimated to be at least when the operator stops viewing the display screen with eyes. The display device is made to regenerate the series of images on a screen including notification information for notifying that the image has been regenerated and regenerated again.
　また、本発明にかかる画像再生制御装置は、前記一連の画像は、カプセル型内視鏡によって撮像された画像であって、前記表示制御部は、前記表示装置に、前記カプセル型内視鏡によって撮像された画像を再生させることを特徴とする。 In the image reproduction control device according to the present invention, the series of images are images captured by a capsule endoscope, and the display control unit is connected to the display device by the capsule endoscope. The captured image is played back.
　また、本発明にかかる画像再生制御方法は、接続する表示装置に画像を再生させる画像再生制御装置が行う画像再生制御方法であって、前記表示装置の表示画面に、一連の画像を順次再生させる画像再生ステップと、前記一連の画像の再生中に、当該画像再生制御装置の操作者が目で前記表示画面を見ているか否かを検出する検出ステップと、前記検出ステップにおいて、前記操作者が目で前記表示画面を見ていないと検出された場合に、前記一連の画像の再生位置を、少なくとも前記操作者が目で前記表示画面を見なくなったと推定される時点まで戻して前記表示装置に前記一連の画像を再々生させる画像再々生ステップと、を含むことを特徴とする。 The image reproduction control method according to the present invention is an image reproduction control method performed by an image reproduction control device that reproduces an image on a connected display device, and sequentially reproduces a series of images on the display screen of the display device. In the image reproduction step, the detection step of detecting whether or not the operator of the image reproduction control device is viewing the display screen with eyes during the reproduction of the series of images, and in the detection step, the operator When it is detected that the display screen is not seen with the eyes, the playback position of the series of images is returned to at least the time when it is estimated that the operator no longer sees the display screen with the eyes. An image regeneration step for reproducing the series of images again.
　本発明によれば、操作者が目で表示画面を見ていないと検出された場合に、一連の画像の再生位置を、少なくとも操作者が目で表示画面を見なくなったと推定される時点まで戻して表示装置に一連の画像を自動的に再々生させるため、再生画像の見落とし防止と、操作者の操作負担の軽減との双方の実現が可能になる。 According to the present invention, when it is detected that the operator is not looking at the display screen with the eyes, the playback position of the series of images is returned to at least the time when it is estimated that the operator has stopped looking at the display screen with the eyes. Since the display device automatically reproduces a series of images, it is possible to realize both prevention of overlooking of the reproduced image and reduction of the operation burden on the operator.
図１は、本発明の実施の形態１にかかるカプセル型内視鏡システムの概略構成を示す模式図である。FIG. 1 is a schematic diagram illustrating a schematic configuration of a capsule endoscope system according to the first embodiment of the present invention. 図２は、図１に示す画像再生制御装置の構成を模式的に示すブロック図である。FIG. 2 is a block diagram schematically showing the configuration of the image reproduction control apparatus shown in FIG. 図３は、図１に示す画像再生制御装置における画像再生処理の処理手順を示すフローチャートである。FIG. 3 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG. 図４は、図３に示す戻り位置設定処理を説明するための図である。FIG. 4 is a diagram for explaining the return position setting process shown in FIG. 図５は、実施の形態１における表示装置の表示画面の一例を示す図である。FIG. 5 is a diagram illustrating an example of a display screen of the display device in the first embodiment. 図６は、実施の形態１における表示装置の表示画面の他の例を示す図である。FIG. 6 is a diagram illustrating another example of the display screen of the display device according to the first embodiment. 図７は、実施の形態２にかかる画像再生制御装置の構成を示すブロック図である。FIG. 7 is a block diagram of the configuration of the image reproduction control apparatus according to the second embodiment. 図８は、図７に示す画像再生制御装置における画像再生処理の処理手順を示すフローチャートである。FIG. 8 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG. 図９は、図８に示す戻り位置設定処理を説明するための図である。FIG. 9 is a diagram for explaining the return position setting process shown in FIG. 図１０は、実施の形態２にかかる画像再生制御装置の他の構成を示すブロック図である。FIG. 10 is a block diagram of another configuration of the image reproduction control apparatus according to the second embodiment.
　以下の説明では、本発明を実施するための形態（以下、「実施の形態」という）として、カプセル型内視鏡によって撮像された画像を表示装置に再生させる画像再生制御装置について説明する。また、この実施の形態により、この発明が限定されるものではない。さらに、図面の記載において、同一部分には同一の符号を付している。 In the following description, an image reproduction control device that causes a display device to reproduce an image captured by a capsule endoscope will be described as a mode for carrying out the present invention (hereinafter referred to as “embodiment”). Moreover, this invention is not limited by this embodiment. Furthermore, the same code | symbol is attached | subjected to the same part in description of drawing.
（実施の形態１）
　図１は、本発明の実施の形態１におけるカプセル型内視鏡システムの概略構成を示す模式図である。図１に示すように、実施の形態１におけるカプセル型内視鏡システム１は、カプセル型内視鏡２、受信アンテナユニット３、受信装置４、クレードル５、画像再生制御装置１０、および、表示装置２０を備える。 (Embodiment 1)
FIG. 1 is a schematic diagram showing a schematic configuration of a capsule endoscope system according to Embodiment 1 of the present invention. As shown in FIG. 1, a capsule endoscope system 1 according to Embodiment 1 includes a capsule endoscope 2, a receiving antenna unit 3, a receiving device 4, a cradle 5, an image reproduction control device 10, and a display device. 20.
　カプセル型内視鏡２は、被検体Ｈ内に導入された後、臓器の蠕動運動等によって被検体Ｈの消化管内を移動しつつ、生体部位（食道、胃、小腸、及び大腸等）を所定の周期（例えば０．５秒周期）で順次撮像して画像データを取得し、無線信号に重畳して送信する。受信アンテナユニット３は、カプセル型内視鏡２から送信された無線信号を、被検体Ｈに装着された複数の受信アンテナ３ａ～３ｈを介して受信する。受信装置４は、受信アンテナユニット３を介して受信した画像データに所定の信号処理を施し、メモリ（不図示）に記録する。画像再生制御装置１０は、ワークステーションを用いて構成され、カプセル型内視鏡２が取得した画像データを、クレードル５を介して、受信装置４のメモリから取り込み、該画像データを用いて被検体Ｈ内の画像を作成する。画像再生制御装置１０は、作成した被検体Ｈ内の画像を、記憶装置（不図示）に記憶させるほか、画像再生制御装置１０に接続する液晶ディスプレイ等の表示装置２０に再生させる。また、表示装置２０には、表示装置２０の表示画面に向かい合って、画像再生制御装置１０の操作を行う操作者の顔を撮像する撮像装置１４ａが取り付けられており、撮像装置１４ａが撮像した操作者の顔が映る画像は、画像再生制御装置１０に出力される。なお、画像再生制御装置１０はワークステーションに限らず、ラップトップＰＣやタブレット端末などのモバイル機器や、メガネ型や時計型などのウェアラブル端末などで構成されていてもよい。 After the capsule endoscope 2 is introduced into the subject H, the capsule endoscope 2 moves through the digestive tract of the subject H by peristaltic movement of the organ and the like, and moves a predetermined part of the living body (esophagus, stomach, small intestine, large intestine, etc.) The image data is acquired in sequence with a period of (for example, a 0.5 second period) to acquire image data, and is superimposed on the radio signal and transmitted. The receiving antenna unit 3 receives the radio signal transmitted from the capsule endoscope 2 via the plurality of receiving antennas 3a to 3h attached to the subject H. The receiving device 4 performs predetermined signal processing on the image data received via the receiving antenna unit 3 and records it in a memory (not shown). The image reproduction control device 10 is configured using a workstation, takes in the image data acquired by the capsule endoscope 2 from the memory of the receiving device 4 via the cradle 5, and uses the image data for the subject. Create an image in H. The image reproduction control device 10 causes the created image in the subject H to be stored in a storage device (not shown), and is also reproduced on a display device 20 such as a liquid crystal display connected to the image reproduction control device 10. The display device 20 is attached with an imaging device 14a that faces the display screen of the display device 20 and captures the face of the operator who operates the image reproduction control device 10, and the operation captured by the imaging device 14a. The image showing the person's face is output to the image reproduction control device 10. The image reproduction control apparatus 10 is not limited to a workstation, and may be configured by a mobile device such as a laptop PC or a tablet terminal, or a wearable terminal such as a glasses type or a clock type.
　図２は、図１に示す画像再生制御装置１０の構成を示すブロック図である。図２においては、説明のために、画像再生制御装置１０が接続する表示装置２０および記憶装置３０についても示している。記憶装置３０は、画像群Ｇｇを格納するフォルダＦａを複数記憶する。画像群Ｇｇは、カプセル型内視鏡２によって撮像された画像であって、時系列に連続する一連の画像によって構成される。 FIG. 2 is a block diagram showing a configuration of the image reproduction control apparatus 10 shown in FIG. In FIG. 2, the display device 20 and the storage device 30 to which the image reproduction control device 10 is connected are also shown for explanation. The storage device 30 stores a plurality of folders Fa for storing the image group Gg. The image group Gg is an image captured by the capsule endoscope 2 and is composed of a series of images that are continuous in time series.
　図２に示すように、画像再生制御装置１０は、入力部１１、データ送受信部１２、制御部１３、瞬き検出部１４、画像処理部１５、表示制御部１６、出力部１８、および、記憶部１９を有する。 As shown in FIG. 2, the image reproduction control device 10 includes an input unit 11, a data transmission / reception unit 12, a control unit 13, a blink detection unit 14, an image processing unit 15, a display control unit 16, an output unit 18, and a storage unit. 19
　入力部１１は、例えばキーボードやマウス、タッチパネル、各種スイッチ等の入力デバイスによって実現される。入力部１１は、操作者の操作に応じた情報や命令の入力を受け付ける。 The input unit 11 is realized by an input device such as a keyboard, a mouse, a touch panel, and various switches. The input unit 11 receives input of information and commands according to the operation of the operator.
　データ送受信部１２は、記憶媒体、又は有線ＬＡＮや無線ＬＡＮ等の通信回線と接続可能なインタフェースである。たとえば、データ送受信部１２は、制御部１３の制御に基づいて記憶装置３０に接続し、記憶装置３０が記憶するカプセル型内視鏡２によって撮像された画像群Ｇｇの画像データを受信する。 The data transmission / reception unit 12 is an interface that can be connected to a storage medium or a communication line such as a wired LAN or a wireless LAN. For example, the data transmission / reception unit 12 is connected to the storage device 30 based on the control of the control unit 13 and receives image data of the image group Gg captured by the capsule endoscope 2 stored in the storage device 30.
　制御部１３は、ＣＰＵ等のハードウェアによって実現され、記憶部１９に記憶された各種プログラムを読み込むことにより、入力部１１を介して入力された信号や、データ送受信部１２から入力された画像データ等に基づいて、画像再生制御装置１０を構成する各部への指示やデータの転送等を行い、画像再生制御装置１０全体の動作を統括的に制御する。 The control unit 13 is realized by hardware such as a CPU, and reads various programs stored in the storage unit 19, so that signals input via the input unit 11 and image data input from the data transmission / reception unit 12 are read. Based on the above, instructions to each unit constituting the image reproduction control device 10 and data transfer are performed to control the overall operation of the image reproduction control device 10 in an integrated manner.
　瞬き検出部１４は、瞼の動きや瞳の動き、焦点位置の変化など、目の動きを検出する検出部の１つであり、画像再生制御装置１０の操作者が目で表示装置２０の表示画面を見ているか否かを検出する。具体的には、瞬き検出部１４は、操作者の瞬きの終了時間を検出して、検出結果を制御部１３に出力する。 The blink detection unit 14 is one of detection units that detect eye movement such as eyelid movement, pupil movement, and focal position change. The operator of the image reproduction control device 10 displays the display device 20 with the eyes. Detect if you are looking at the screen. Specifically, the blink detection unit 14 detects the end time of the operator's blink and outputs the detection result to the control unit 13.
　瞬き検出部１４は、撮像装置１４ａが撮像した操作者の顔の画像を処理して目の瞬きの有無を検出し、目の瞬きを検出した場合には、目の瞬きの終了時間を示す瞬き検出信号を出力する。瞬き検出部１４は、たとえば、撮像装置１４ａによって撮像された画像から目が映る領域を抽出し、標準の目の型との間でパターンマッチングを行うことによって、目の瞬きの有無を検出する。あるいは、瞬き検出部１４は、撮像装置１４ａによって撮像された画像のうちの目が映る領域に対して、複数の横方向ラインの濃淡データを取得し、濃淡データの二値化によって白（白目）の部分と黒（黒目）の部分とを分離し、黒の部分の有無や黒の部分の大きさをもとに目の瞬きの有無を検出する。 The blink detection unit 14 processes the image of the face of the operator imaged by the imaging device 14a to detect the presence or absence of blinking eyes. When blinking eyes are detected, blinking indicating the end time of blinking eyes is detected. A detection signal is output. The blink detection unit 14 detects, for example, the presence or absence of blinks of the eyes by extracting a region where the eyes appear from an image captured by the imaging device 14a and performing pattern matching with a standard eye pattern. Alternatively, the blink detection unit 14 acquires grayscale data of a plurality of horizontal lines for a region in which an eye is captured in an image captured by the imaging device 14a, and white (white eye) is obtained by binarizing the grayscale data. And the black (black eye) part are separated, and the presence or absence of blinking eyes is detected based on the presence or absence of the black part and the size of the black part.
　画像処理部１５は、データ送受信部１２が受信した画像群Ｇｇの画像データに対して、オプティカルブラック減算処理、ゲイン調整処理、画像データの同時化処理、ガンマ補正処理、ホワイトバランス（ＷＢ）調整処理、カラーマトリクス演算処理、色再現処理およびエッジ強調処理等の画像処理を行う。 The image processing unit 15 performs optical black subtraction processing, gain adjustment processing, image data synchronization processing, gamma correction processing, white balance (WB) adjustment processing on the image data of the image group Gg received by the data transmitting / receiving unit 12. Image processing such as color matrix calculation processing, color reproduction processing, and edge enhancement processing is performed.
　表示制御部１６は、画像処理部１５が処理した画像群の画像データ、あるいは、記憶装置３０から受信した画像群の画像データから、表示装置２０に表示させるための各表示用画像データを生成する。表示制御部１６は、表示用画像データを、デジタル信号からアナログ信号に変換し、変換したアナログ信号の画像データをハイビジョン方式等のフォーマットに変更後、表示装置２０へ出力する。表示制御部１６は、これによって、表示装置の表示画面に、一連の画像を時系列に沿って順次再生させる。なお、一連の画像を撮像したときのカプセル型内視鏡２の位置に基づいて、一連の画像を並べ替えて順次再生してもよい。 The display control unit 16 generates display image data to be displayed on the display device 20 from the image data of the image group processed by the image processing unit 15 or the image data of the image group received from the storage device 30. . The display control unit 16 converts display image data from a digital signal to an analog signal, changes the converted analog signal image data to a format such as a high-definition method, and then outputs the converted image data to the display device 20. Thereby, the display control unit 16 causes the display screen of the display device to sequentially reproduce a series of images in time series. Note that the series of images may be rearranged and sequentially reproduced based on the position of the capsule endoscope 2 when the series of images are captured.
　表示制御部１６は、瞬き検出部１４が、操作者が目で表示画面を見ていないと検出した場合に、表示装置２０に表示されている一連の画像に対する再生位置を、少なくとも操作者が目で表示画面を見なくなったと推定される時点まで戻して、表示装置２０に一連の画像を再々生させる。表示制御部１６は、瞬き検出部１４が、操作者が目で表示画面を見ていないと検出した場合に、一連の画像の再生位置の戻り位置を設定する戻り位置設定部１７を備える。 When the blink detection unit 14 detects that the operator is not looking at the display screen with his / her eyes, the display control unit 16 determines at least the playback position for the series of images displayed on the display device 20 when the operator The display device 20 is returned to the time point when it is estimated that the display screen is no longer seen, and a series of images are regenerated on the display device 20. The display control unit 16 includes a return position setting unit 17 that sets the return position of the reproduction position of a series of images when the blink detection unit 14 detects that the operator is not looking at the display screen with eyes.
　出力部１８は、スピーカーやプリンタ等を用いて実現され、制御部１３の制御にしたがい、カプセル型内視鏡２が撮像した画像データの再生処理に関する情報を出力する。 The output unit 18 is realized using a speaker, a printer, or the like, and outputs information related to the reproduction processing of the image data captured by the capsule endoscope 2 under the control of the control unit 13.
　記憶部１９は、揮発性メモリや不揮発性メモリを用いて実現され、画像再生制御装置１０を動作させるための各種プログラムを記憶する。記憶部１９は、画像再生制御装置１０の処理中の情報を一時的に記憶する。記憶部１９は、画像再生制御装置１０の外部から装着されるメモリカード等を用いて構成されてもよい。 The storage unit 19 is realized using a volatile memory or a non-volatile memory, and stores various programs for operating the image reproduction control device 10. The storage unit 19 temporarily stores information being processed by the image reproduction control device 10. The storage unit 19 may be configured using a memory card or the like attached from the outside of the image reproduction control device 10.
　図３は、図１に示す画像再生制御装置１０における画像再生処理の処理手順を示すフローチャートである。再生対象の画像群の画像データとともに、画像再生の開始を指示する情報が入力されると（ステップＳ１）、表示制御部１６は、再生対象の画像データに対する表示用画像データを生成し、表示装置２０の表示画面に再生させる再生処理（ステップＳ２）を行う。 FIG. 3 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus 10 shown in FIG. When information for instructing the start of image reproduction is input together with the image data of the image group to be reproduced (step S1), the display control unit 16 generates display image data for the image data to be reproduced, and displays the display device. A reproduction process (step S2) for reproducing on the 20 display screens is performed.
　瞬き検出部１４は、制御部１３の制御のもと、画像再生制御装置１０の操作者の目の瞬きを検出する瞬き検出処理を行い（ステップＳ３）、画像再生制御装置１０の操作者が目で表示装置２０の表示画面を見ているか否かを検出する。瞬き検出部１４は、操作者の目の瞬きを検出した場合には、目の瞬きの終了時間を示す瞬き検出信号を、制御部１３を介して、表示制御部１６に出力する。 Under the control of the control unit 13, the blink detection unit 14 performs blink detection processing for detecting blinks in the eyes of the operator of the image reproduction control device 10 (step S <b> 3), and the operator of the image reproduction control device 10 observes the eyes. It is detected whether or not the display screen of the display device 20 is being viewed. When the blink detection unit 14 detects the blink of the eye of the operator, the blink detection unit 14 outputs a blink detection signal indicating the end time of the blink of the eye to the display control unit 16 via the control unit 13.
　表示制御部１６は、瞬き検出信号の入力の有無をもとに、瞬き検出部１４が、画像再生制御装置１０の操作者の目の瞬きを検出したか否かを判断する（ステップＳ４）。 The display control unit 16 determines whether or not the blink detection unit 14 has detected the blink of the eyes of the operator of the image reproduction control device 10 based on whether or not the blink detection signal is input (step S4).
　表示制御部１６は、瞬き検出部１４が、画像再生制御装置１０の操作者の目の瞬きを検出したと判断した場合には（ステップＳ４：Ｙｅｓ）、戻り位置設定部１７が、入力された瞬き検出信号をもとに、画像群の再生位置の戻り位置を設定する戻り位置設定処理を行う（ステップＳ５）。戻り位置設定部１７は、画像群の再生位置の戻り位置を、少なくとも操作者が目で表示画面を見なくなったと推定される時点に設定する。 When the blink control unit 14 determines that the blink detection unit 14 has detected the blink of the eyes of the operator of the image reproduction control device 10 (step S4: Yes), the display position setting unit 17 is input. Based on the blink detection signal, return position setting processing for setting the return position of the reproduction position of the image group is performed (step S5). The return position setting unit 17 sets the return position of the playback position of the image group at least when it is estimated that the operator has stopped viewing the display screen with his eyes.
　図４は、図３に示す戻り位置設定処理を説明するための図である。図４の（１）は、瞬き検出部１４による瞬き検出処理に関するタイミングチャートである。図４の（２）は、現に表示装置２０に再生されている画像の画像群における再生位置を示すタイムバーＢの模式図である。 FIG. 4 is a diagram for explaining the return position setting process shown in FIG. (1) of FIG. 4 is a timing chart regarding blink detection processing by the blink detection unit 14. (2) of FIG. 4 is a schematic diagram of the time bar B indicating the reproduction position in the image group of the image that is actually reproduced on the display device 20.
　瞬き検出部１４は、操作者の目の瞬きを検出した場合には、目の瞬きの終了時間ｔａを示す瞬き検出信号Ｓａを、制御部１３を介して表示制御部１６に出力する。表示制御部１６では、戻り位置設定部１７は、入力された瞬き検出信号Ｓａに含まれる瞬きの終了時間ｔａに対応する再生位置Ｄａから、矢印Ｙｂのように、一回の瞬きに要する時間に対応する所定の時間Ｔｂ分戻した位置Ｄｂに再生位置を戻すように、戻り位置を設定する。人間の瞬きの時間は、０．１０～０．１５（秒）程度であるため、戻り位置設定部１７は、入力された瞬き検出信号に含まれる瞬きの終了時間に対応する再生位置Ｄａから、０．１５（秒）分、再生位置を戻す。もちろん、戻り位置設定部１７は、矢印Ｙｃに示すように、再生位置Ｄａから、一回の瞬きに要する時間よりも十分に長い一定の時間Ｔｃ（＞Ｔｂ）分戻した位置Ｄｃに再生位置を戻すように設定してもよい。 When the blink detection unit 14 detects the blink of the eyes of the operator, the blink detection unit 14 outputs a blink detection signal Sa indicating the blinking end time ta of the eyes to the display control unit 16 via the control unit 13. In the display control unit 16, the return position setting unit 17 sets the time required for one blink as indicated by an arrow Yb from the reproduction position Da corresponding to the blink end time ta included in the input blink detection signal Sa. The return position is set so that the reproduction position is returned to the position Db returned by the corresponding predetermined time Tb. Since the human blinking time is about 0.10 to 0.15 (seconds), the return position setting unit 17 determines from the reproduction position Da corresponding to the blinking end time included in the input blink detection signal. Return the playback position by 0.15 (seconds). Of course, as shown by the arrow Yc, the return position setting unit 17 sets the playback position to a position Dc that is returned from the playback position Da by a fixed time Tc (> Tb) that is sufficiently longer than the time required for one blink. It may be set to return.
　また、戻り位置設定部１７は、一回の瞬きに要する時間に対応する所定の時間と、画像群の再生処理におけるフレームレートとをもとに、一回の瞬きに要する時間に対応する所定の時間の間に表示装置２０で再生された画像のフレーム数を求め、瞬き検出部１４が検出した瞬きの終了時間から、少なくとも、求めたフレーム数分、再生位置を戻す。たとえば、人間の瞬きの時間を０．１０（秒）と仮定し、画像群の再生処理におけるフレームレートをＦ（フレーム／秒）とした場合には、下記の（１）式を用いて、再生位置を戻すフレーム数Ｊを演算する。
Ｊ＝Ｆ×０．１０　　　　　　　　　　　　　　　　　・・・（１） The return position setting unit 17 also determines a predetermined time corresponding to the time required for one blink based on the predetermined time corresponding to the time required for one blink and the frame rate in the reproduction processing of the image group. The number of frames of the image reproduced on the display device 20 during the time is obtained, and the reproduction position is returned by at least the obtained number of frames from the blink end time detected by the blink detection unit 14. For example, assuming that the human blinking time is 0.10 (seconds) and the frame rate in the image group reproduction processing is F (frames / second), reproduction using the following equation (1): The number J of frames for returning the position is calculated.
J = F × 0.10 (1)
　たとえば、画像群の再生処理におけるフレームレートＦが６０（フレーム／秒）である場合には、（１）式によって再生位置を戻すフレーム数が６フレームと求められるため、戻り位置設定部１７は、６フレーム以上、再生位置を戻す。 For example, when the frame rate F in the reproduction processing of the image group is 60 (frames / second), the number of frames for returning the reproduction position is obtained as 6 frames according to the equation (1). Return the playback position for 6 frames or more.
　表示制御部１６は、戻り位置設定部１７が設定した位置まで、画像群の再生位置を戻して、表示装置２０に画像群の画像を再度再生させる再々生処理を行う（ステップＳ６）。この場合、表示制御部１６は、再々生させる画像とともに、表示装置２０の表示画面に再生されている画像が、少なくとも操作者が目で表示画面を見なくなったと推定される時点まで再生位置を戻して再々生された画像であることを報知する報知情報を含ませた画面で、表示装置２０に画像群を再々生させる。 The display control unit 16 returns the reproduction position of the image group to the position set by the return position setting unit 17, and performs re-generation processing for causing the display device 20 to reproduce the image of the image group again (step S6). In this case, the display control unit 16 returns the reproduction position to the time when it is estimated that the image reproduced on the display screen of the display device 20 together with the image to be regenerated is at least the point where the operator can no longer see the display screen with eyes. Then, the image group is reproduced again on the display device 20 on the screen including the notification information for notifying that the image is reproduced again.
　図５は、再々生画像表示時における表示装置２０の表示画面の一例を示す図である。図５に示すように、表示装置２０の表示画面には、カプセル型内視鏡画像再生用の再生メニューＭが表示されている。再生メニューＭにおいて、左側上部には領域Ｒａが設けられ、左側下部においては領域Ｒｂが設けられ、下部には領域Ｒｃが設けられ、右側には領域Ｒｄが設けられる。領域Ｒａには、被検体（患者）情報（たとえば名前、ＩＤ、生年月日等）が表示され、領域Ｒｂには、画像表示を制御するためのアイコン画像が表示され、領域Ｒｃには、現に再生されている画像の画像群全体における位置を示すスライダーＢｎを有するタイムバーＢが表示され、領域Ｒｄには、カプセル型内視鏡２によって撮像された画像Ｇが再生される。表示制御部１６は、再々生時には、この領域Ｒｄに表示される画像Ｇを、再生時とは異なる色、または、再生時とは異なる太さの枠Ｗで囲うことによって、再生されている画像が、少なくとも操作者が目で表示画面を見なくなったと推定される時点まで再生位置を戻して再々生された画像であることを示す。 FIG. 5 is a diagram illustrating an example of a display screen of the display device 20 when a re-live image is displayed. As shown in FIG. 5, a playback menu M for playing back a capsule endoscope image is displayed on the display screen of the display device 20. In the reproduction menu M, an area Ra is provided in the upper left part, an area Rb is provided in the lower left part, an area Rc is provided in the lower part, and an area Rd is provided in the right part. Subject (patient) information (for example, name, ID, date of birth, etc.) is displayed in the region Ra, an icon image for controlling image display is displayed in the region Rb, and the region Rc actually A time bar B having a slider Bn indicating the position of the image being reproduced in the entire image group is displayed, and an image G captured by the capsule endoscope 2 is reproduced in the region Rd. The display control unit 16 regenerates the image G displayed in the region Rd by surrounding it with a frame W having a color different from that at the time of reproduction or a thickness W different from that at the time of reproduction. Indicates that the image has been reproduced again with the playback position returned to at least the time when it is estimated that the operator has stopped viewing the display screen with his eyes.
　また、図６に示すように、表示制御部１６は、この領域Ｒｄに表示される画像Ｇの近傍または周縁に、再生されている画像が、少なくとも操作者が目で表示画面を見なくなったと推定される時点まで再生位置を戻して再々生された画像であることを示すアイコン画像Ａを表示させてもよい。また、表示制御部１６は、領域ＲｃのタイムバーＢにおける再々生区間に対応する部分を、拡大表示させてもよく、他とは異なる色を付して表示させてもよい。 Further, as shown in FIG. 6, the display control unit 16 estimates that the image being reproduced is at least near the periphery of the image G displayed in the region Rd, and at least the operator does not see the display screen with eyes. The icon image A may be displayed indicating that the image has been reproduced again after returning to the reproduction position. Further, the display control unit 16 may enlarge and display a portion corresponding to the re-occurring section in the time bar B of the region Rc, or may display it with a color different from the others.
　表示制御部１６は、表示装置２０に再々生させた画像が、再々生対象の画像の最終の画像であるか否かを判断する（ステップＳ７）。表示制御部１６は、表示装置２０に再々生させた画像が、再々生対象の画像の最終の画像でないと判断した場合には（ステップＳ７：Ｎｏ）、ステップＳ６に戻り、画像の再々生処理を継続する。一方、表示制御部１６は、表示装置２０に再々生させた画像が、再々生対象の画像の最終の画像であると判断した場合には（ステップＳ７：Ｙｅｓ）、ステップＳ２に戻り、画像群における次の画像データに対する画像再生処理を行う。すなわち、表示制御部１６は、瞬き検出部１４が検出した瞬きの終了時間に再生されていた画像の次の画像データに対して、通常の画像再生処理を行う。 The display control unit 16 determines whether or not the image regenerated on the display device 20 is the final image of the image to be regenerated (step S7). When the display control unit 16 determines that the image regenerated on the display device 20 is not the final image of the image to be regenerated (step S7: No), the display control unit 16 returns to step S6 to regenerate the image. Continue. On the other hand, when the display control unit 16 determines that the image regenerated on the display device 20 is the final image of the image to be regenerated (step S7: Yes), the display control unit 16 returns to step S2 to display the image group. The image reproduction process is performed on the next image data in. That is, the display control unit 16 performs a normal image reproduction process on the image data next to the image reproduced at the blink end time detected by the blink detection unit 14.
　また、表示制御部１６は、瞬き検出部１４が画像再生制御装置１０の操作者の目の瞬きを検出していないと判断した場合には（ステップＳ４：Ｎｏ）、再生した画像が、再生を指示された画像群の最終画像である否かを判断する（ステップＳ８）。表示制御部１６は、再生した画像が、再生を指示された画像群の最終画像でないと判断した場合には（ステップＳ８：Ｎｏ）、ステップＳ２に戻り、画像群における次の画像データに対する画像再生処理を行う。また、表示制御部１６は、再生した画像が、再生を指示された画像群の最終画像であると判断した場合には（ステップＳ８：Ｙｅｓ）、この画像群に対する再生処理を終了する。 When the display control unit 16 determines that the blink detection unit 14 has not detected the blink of the eyes of the operator of the image reproduction control device 10 (step S4: No), the reproduced image is reproduced. It is determined whether or not it is the final image of the instructed image group (step S8). If the display control unit 16 determines that the reproduced image is not the final image of the image group instructed to be reproduced (step S8: No), the display control unit 16 returns to step S2 and reproduces the next image data in the image group. Process. Further, when the display control unit 16 determines that the reproduced image is the final image of the image group instructed to be reproduced (step S8: Yes), the reproduction process for the image group is ended.
　人の瞬きの時間は、０．１０～０．１５（秒）であるため、１０～１５（フレーム／秒）以上のフレームレートで画像がコマ再生される場合には、瞬きをしている間に１フレーム以上の画像を見落とす可能性がある。そこで、実施の形態１では、表示制御部１６は、瞬き検出部１４が操作者の目の瞬きを検出した場合、瞬き検出部１４が検出した瞬きの終了時間から、少なくとも、一回の瞬きに要する時間に対応する所定の時間分、画像群における再生位置を戻して表示装置２０に画像群を再度再生させる。 Since the blinking time of a person is 0.10 to 0.15 (seconds), when an image is played back at a frame rate of 10 to 15 (frames / second) or more, while blinking, There is a possibility that an image of one frame or more may be overlooked. Therefore, in the first embodiment, when the blink detection unit 14 detects the blink of the eyes of the operator, the display control unit 16 performs at least one blink from the blink end time detected by the blink detection unit 14. The reproduction position in the image group is returned for a predetermined time corresponding to the time required, and the display device 20 is made to reproduce the image group again.
　このため、実施の形態１によれば、操作者が瞬きによって再生画像を見落とした場合であっても、自動的に、瞬き前の画像まで戻って画像が再々生されるため、再生画像の見落としを防止できるとともに、操作者の操作負担を格段に軽減することが可能になる。特に、瞬きは、無自覚な生理反応であるため、操作者が、自身の瞬きに気づいて、瞬きによって見逃した画像まで再生位置を戻すこと自体が難しいが、実施の形態１によれば、瞬きによって操作者が無自覚に見逃した画像があっても、自動的に、瞬き前の画像まで再生位置を戻すことができるため、再生画像の見落としを確実に防止することができる。 For this reason, according to the first embodiment, even if the operator overlooks the reproduced image by blinking, the image is automatically regenerated by returning to the image before blinking. Can be prevented, and the operation burden on the operator can be significantly reduced. In particular, since blinking is an unconscious physiological reaction, it is difficult for the operator to notice the blink and return the playback position to the image that was missed by blinking. Even if there is an image that the operator has unknowingly missed, the reproduction position can be automatically returned to the image before blinking, so that it is possible to reliably prevent the reproduction image from being overlooked.
　また、再々生処理の場合には、枠Ｗやアイコン画像Ａを表示させて、再々生画像であることを報知するため、操作者は、現に再生されている画像が、見逃したおそれがある画像であることを認識することができる。 In the case of re-reproduction processing, the frame W and the icon image A are displayed to notify that the image is a re-reproduction image, so that the operator may have missed the image that is actually being reproduced. Can be recognized.
（実施の形態２）
　次に、実施の形態２について説明する。図７は、実施の形態２にかかる画像再生制御装置の構成を示すブロック図である。 (Embodiment 2)
Next, a second embodiment will be described. FIG. 7 is a block diagram of the configuration of the image reproduction control apparatus according to the second embodiment.
　図７に示すように、実施の形態２にかかる画像再生制御装置２１０は、画像再生制御装置１０と比して、制御部１３、瞬き検出部１４および表示制御部１６に代えて、制御部１３と同様の機能を有する制御部２１３、瞬き検出部２１４および表示制御部２１６を備える。 As illustrated in FIG. 7, the image reproduction control device 210 according to the second embodiment is different from the image reproduction control device 10 in that the control unit 13, the blink detection unit 14, and the display control unit 16 are replaced with the control unit 13. Are provided with a control unit 213, a blink detection unit 214, and a display control unit 216 having the same functions.
　瞬き検出部２１４は、瞬きによって操作者が目で表示画面を見なくなった状態の開始時間と終了時間とを検出し、目の瞬きを検出した場合には、目の瞬きの開始時間と終了時間とを示す瞬き検出信号を出力する。 The blink detection unit 214 detects a start time and an end time in a state where the operator cannot see the display screen with the eyes due to the blink, and when a blink of the eyes is detected, the blink start time and the end time A blink detection signal indicating that is output.
　表示制御部２１６は、少なくとも瞬き検出部２１４が検出した操作者が目で表示画面を見なくなった状態の開始時間、すなわち瞬きの開始時間まで、一連の画像の再生位置を戻す。表示制御部２１６は、画像群の再生位置の戻り位置を、瞬きの開始時間に設定する戻り位置設定部２１７を有する。 The display control unit 216 returns the playback position of a series of images until at least the start time when the operator detected by the blink detection unit 214 stops looking at the display screen with eyes, that is, the blink start time. The display control unit 216 includes a return position setting unit 217 that sets the return position of the reproduction position of the image group to the blink start time.
　図８は、図７に示す画像再生制御装置における画像再生処理の処理手順を示すフローチャートである。ステップＳ１１およびステップＳ１２は、図３に示すステップＳ１およびステップＳ２である。 FIG. 8 is a flowchart showing a processing procedure of image reproduction processing in the image reproduction control apparatus shown in FIG. Step S11 and step S12 are step S1 and step S2 shown in FIG.
　瞬き検出部２１４は、制御部２１３の制御のもと、画像再生制御装置２１０の操作者の目の瞬きの有無を検出する瞬き検出処理を行う（ステップＳ１３）。瞬き検出部２１４は、操作者の目の瞬きを検出した場合には目の瞬きの開始時間と終了時間とを示す瞬き検出信号を、制御部２１３を介して、表示制御部２１６に出力する。ステップＳ１４は、図３に示すステップＳ４である。 The blink detection unit 214 performs blink detection processing for detecting whether or not the operator of the image reproduction control device 210 blinks under the control of the control unit 213 (step S13). When the blink detection unit 214 detects the blink of the eyes of the operator, the blink detection signal is output to the display control unit 216 via the control unit 213. The blink detection signal indicates the blink start time and end time. Step S14 is step S4 shown in FIG.
　表示制御部２１６は、瞬き検出部２１４が画像再生制御装置２１０の操作者の目の瞬きを検出したと判断した場合には（ステップＳ１４：Ｙｅｓ）、戻り位置設定部２１７が、入力された瞬き検出信号をもとに、画像群の再生位置の戻り位置を設定する戻り位置設定処理を行う（ステップＳ１５）。戻り位置設定部２１７は、画像群の再生位置の戻り位置を、瞬きの開始時間に対応する位置に設定する。 When the display control unit 216 determines that the blink detection unit 214 has detected the blink of the eyes of the operator of the image reproduction control device 210 (step S14: Yes), the return position setting unit 217 receives the input blink. Based on the detection signal, return position setting processing for setting the return position of the reproduction position of the image group is performed (step S15). The return position setting unit 217 sets the return position of the reproduction position of the image group to a position corresponding to the blink start time.
　図９は、図８に示す戻り位置設定処理を説明するための図である。図９の（１）は、瞬き検出部２１４による瞬き検出処理に関するタイミングチャートである。図９の（２）は、現に表示装置２０に表示されている画像の画像群における再生位置を示すタイムバーＢの模式図である。 FIG. 9 is a diagram for explaining the return position setting process shown in FIG. (1) of FIG. 9 is a timing chart regarding blink detection processing by the blink detection unit 214. (2) of FIG. 9 is a schematic diagram of the time bar B indicating the reproduction position in the image group of the image currently displayed on the display device 20.
　瞬き検出部２１４は、操作者の目の瞬きを検出した場合には、目の瞬きの開始時間ｔｄと終了時間ｔｅとを示す瞬き検出信号Ｓｄ，Ｓｅを、制御部２１３を介して、表示制御部２１６に出力する。表示制御部２１６では、戻り位置設定部２１７は、入力された瞬き検出信号Ｓｄ，Ｓｅに含まれる瞬きの終了時間ｔｅに対応する再生位置Ｄｅから、矢印Ｙｄのように、瞬きの開始時間ｔｄに対応する位置Ｄｄに再生位置を戻すように戻り位置を設定する。ステップＳ１６は、図３に示すステップＳ６である。この結果、表示装置２０では、瞬きがあった時間Ｔｅ（図８参照）の間に再生されていた画像が、再度再生される。 When the blink detection unit 214 detects the blink of the eye of the operator, the blink detection signals Sd and Se indicating the blink start time td and the end time te are displayed via the control unit 213. To the unit 216. In the display control unit 216, the return position setting unit 217 performs the blink start time td as indicated by the arrow Yd from the reproduction position De corresponding to the blink end time te included in the input blink detection signals Sd and Se. The return position is set so that the playback position is returned to the corresponding position Dd. Step S16 is step S6 shown in FIG. As a result, on the display device 20, the image that was reproduced during the blinking time Te (see FIG. 8) is reproduced again.
　また、戻り位置設定部２１７は、瞬き検出部２１４が検出した瞬きの開始時間と終了時間との期間と、画像群の再生処理におけるフレームレートとをもとに、操作者の目が瞬きを行っていた間に表示装置２０で再生された画像のフレーム数を求め、瞬き検出部２１４が検出した終了時間から、少なくとも、求めたフレーム数分、再生位置を戻すように戻り位置を設定してもよい。たとえば、瞬き検出部２１４が検出した瞬きの開始時間と終了時間との間の時間がＥ（秒）である場合には、画像群の再生処理におけるフレームレートＦ（フレーム／秒）と、時間Ｅから、下記の（２）式を用いて、再生位置を戻すフレーム数Ｊを演算する。
Ｊ＝Ｆ×Ｅ　　　　　　　　　　　　　　　　　　　　・・・（２） Also, the return position setting unit 217 blinks the eyes of the operator based on the blink start time and end time detected by the blink detection unit 214 and the frame rate in the image group reproduction processing. Even if the number of frames of the image reproduced on the display device 20 is obtained during the period, the return position is set so that the reproduction position is returned at least by the obtained number of frames from the end time detected by the blink detection unit 214. Good. For example, when the time between the blink start time and the blink end time detected by the blink detection unit 214 is E (seconds), the frame rate F (frame / second) in the image group reproduction processing and the time E Then, the number J of frames for returning the reproduction position is calculated using the following equation (2).
J = F × E (2)
　なお、ステップＳ１７およびステップＳ１８は、図３に示すステップＳ７およびステップＳ８である。 Note that steps S17 and S18 are steps S7 and S8 shown in FIG.
　この実施の形態２によれば、瞬き検出部２１４が検出した瞬きの終了時間に対応する再生位置から、少なくとも瞬きの開始時間に対応する位置まで画像群の再生位置を戻して、表示装置に画像群を自動的に再々生させるため、実施の形態１と同様に、再生画像の見落としと、操作者の操作負担の軽減との双方を実現することができる。また、しばらく目を閉じていたなど、瞬きよりも長い時間画面を見ていなかった場合でも、目を閉じた時間に対応する位置まで画像群の再生位置を戻して、表示装置に画像群を自動的に再々生させるため、実施の形態１と同様に、再生画像の見落としと、操作者の操作負担の軽減との双方を実現することができる。 According to the second embodiment, the reproduction position of the image group is returned from the reproduction position corresponding to the blink end time detected by the blink detection unit 214 to at least the position corresponding to the blink start time, and the image is displayed on the display device. Since the group is automatically regenerated, both the oversight of the reproduced image and the reduction of the operation burden on the operator can be realized as in the first embodiment. Even if you have not seen the screen for a longer time than blinking, such as when you closed your eyes for a while, the playback position of the image group is returned to the position corresponding to the time when you closed your eyes, and the image group is automatically displayed on the display device. Therefore, similar to the first embodiment, both oversight of the reproduced image and reduction of the operation burden on the operator can be realized.
　なお、本実施の形態２では、瞬き検出部２１４を備えた例を説明したが、もちろん、図１０に示す画像再生制御装置２１０Ａのように、操作者の視線の向きを検出する視線検出部２１４Ａ（前述した検出部の１つ）によって、操作者が目で表示画面を見ているか否かを検出してもよい。視線検出部２１４Ａは、撮像装置１４ａが撮像した操作者の顔の画像を処理して、操作者の視線の向きを検出する。視線検出部２１４Ａは、操作者の視線が表示画面に再生されている画像に向いているかを検出し、表示制御部２１６は、表示画面に再生されている画像に操作者の視線が向いていないことを視線検出部２１４Ａが検出した場合には、表示画面に再生されている画像に操作者の視線が向かなくなった時点まで再生位置を戻して、画像群を再々生してもよい。また、視線検出部２１４Ａに、さらに操作者の焦点が検出できる機構を設け、操作者が表示画面に再生されている画像に焦点を合わせているかを検出して、表示画面に再生されている画像に操作者の焦点が合っていない場合には、焦点が合わなくなった時点まで再生位置を戻して、画像群を再々生してもよい。 In the second embodiment, an example in which the blink detection unit 214 is provided has been described. Of course, as in the image reproduction control device 210A illustrated in FIG. 10, the line-of-sight detection unit 214A that detects the direction of the operator's line of sight. (One of the detection units described above) may detect whether the operator is viewing the display screen with his / her eyes. The line-of-sight detection unit 214A processes the image of the operator's face captured by the imaging device 14a, and detects the direction of the line of sight of the operator. The line-of-sight detection unit 214A detects whether the operator's line of sight is directed to the image reproduced on the display screen, and the display control unit 216 does not point the operator's line of sight to the image reproduced on the display screen. When the line-of-sight detection unit 214A detects this, the reproduction position may be returned to the point where the operator's line of sight no longer turns to the image reproduced on the display screen, and the image group may be regenerated. Further, the line-of-sight detection unit 214A is further provided with a mechanism capable of detecting the focus of the operator, detects whether the operator is focused on the image reproduced on the display screen, and is reproduced on the display screen. If the operator is out of focus, the playback position may be returned to the point in time when the focus is lost, and the image group may be regenerated.
　さらに、画像再生制御装置１０は瞬き検出部１４と視線検出部２１４Ａの両方を備えていてもよく、これらの検出結果を組み合わせて操作者が目で表示画面を見ているか否かを検出してもよい。 Furthermore, the image reproduction control device 10 may include both the blink detection unit 14 and the line-of-sight detection unit 214A, and detects whether the operator is viewing the display screen with his eyes by combining these detection results. Also good.
　また、本実施の形態１，２にかかる画像再生制御装置１０，２１０，２１０Ａで実行される各処理に対する実行プログラムは、インストール可能な形式又は実行可能な形式のファイルでＣＤ－ＲＯＭ、フレキシブルディスク、ＣＤ－Ｒ、ＤＶＤ等のコンピュータで読み取り可能な記録媒体に記録して提供するように構成してもよく、インターネット等のネットワークに接続されたコンピュータ上に格納し、ネットワーク経由でダウンロードさせることにより提供するように構成してもよい。また、インターネット等のネットワーク経由で提供または配布するように構成してもよい。 An execution program for each process executed by the image reproduction control apparatuses 10, 210, and 210A according to the first and second embodiments is an installable format or an executable format file, such as a CD-ROM, a flexible disk, It may be configured to be recorded on a computer-readable recording medium such as a CD-R or DVD, and provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. You may comprise. Further, it may be configured to be provided or distributed via a network such as the Internet.
　１　カプセル型内視鏡システム
　２　カプセル型内視鏡
　３　受信アンテナユニット
　３ａ～３ｈ　受信アンテナ
　４　受信装置
　５　クレードル
　１０，２１０，２１０Ａ　画像再生制御装置
　１１　入力部
　１２　データ送受信部
　１３，２１３　制御部
　１４，２１４　瞬き検出部
　１５　画像処理部
　１６，２１６　表示制御部
　１７，２１７　戻り位置設定部
　１８　出力部
　１９　記憶部
２０　表示装置
３０　記憶装置
　２１４Ａ　視線検出部
　Ｈ　被検体 DESCRIPTION OF SYMBOLS 1 Capsule type | mold endoscope system 2 Capsule type | mold endoscope 3 Receiving antenna unit 3a-3h Receiving antenna 4 Receiving device 5 Cradle 10,210,210A Image reproduction control apparatus 11 Input part 12 Data transmission / reception part 13,213 Control part 14, 214 blink detection unit 15 image processing unit 16,216 display control unit 17,217 return position setting unit 18 output unit 19 storage unit 20 display device 30 storage device 214A gaze detection unit H subject

Claims (9)

  1. 　接続する表示装置に画像を再生させる画像再生制御装置であって、
    　前記表示装置の表示画面に、一連の画像を順次再生させる表示制御部と、
    　当該画像再生制御装置の操作者が目で前記表示画面を見ているか否かを検出する検出部と、
    　を備え、
    　前記表示制御部は、前記一連の画像の再生中に、前記検出部が、前記操作者が目で前記表示画面を見ていないと検出した場合に、前記一連の画像の再生位置を、少なくとも前記操作者が目で前記表示画面を見なくなったと推定される時点まで戻して前記表示装置に前記一連の画像を再々生させることを特徴とする画像再生制御装置。     An image reproduction control device for reproducing an image on a connected display device,
    A display control unit that sequentially reproduces a series of images on the display screen of the display device;
    A detection unit for detecting whether an operator of the image reproduction control device is viewing the display screen with eyes;
    With
    When the detection unit detects that the operator is not looking at the display screen with the eyes during reproduction of the series of images, the display control unit sets at least the reproduction position of the series of images. An image reproduction control device, wherein the display device is returned to a point in time when it is estimated that the display screen is no longer seen by an operator, and the series of images is regenerated on the display device.
  2. 　前記検出部は、前記操作者の瞬きの終了時間を検出する瞬き検出部であり、
    　前記表示制御部は、前記検出部が検出した瞬きの終了時間から、少なくとも、一回の瞬きに要する時間に対応する所定の時間分、再生位置を戻して前記一連の画像を再々生させることを特徴とする請求項１に記載の画像再生制御装置。     The detection unit is a blink detection unit that detects an end time of the operator's blink,
    The display control unit returns the reproduction position by a predetermined time corresponding to the time required for one blink from the end time of the blink detected by the detection unit, and reproduces the series of images again. The image reproduction control device according to claim 1, wherein:
  3. 　前記表示制御部は、前記一回の瞬きに要する時間に対応する所定の時間と、前記一連の画像の再生処理におけるフレームレートとをもとに、前記所定の時間の間に前記表示装置で再生された画像のフレーム数を求め、前記検出部が検出した瞬きの終了時間から、少なくとも前記求めたフレーム数分、再生位置を戻して前記一連の画像を再々生させることを特徴とする請求項２に記載の画像再生制御装置。 The display control unit reproduces on the display device during the predetermined time based on a predetermined time corresponding to the time required for one blink and a frame rate in the series of image reproduction processing. The number of frames of the obtained image is obtained, and the reproduction position is returned at least by the obtained number of frames from the blink end time detected by the detection unit, and the series of images is regenerated again. The image reproduction control apparatus according to 1.
  4. 　前記表示制御部は、前記検出部が検出した瞬きの終了時間から、前記一回の瞬きに要する時間に対応する所定の時間よりも長い一定の時間分、再生位置を戻して前記一連の画像を再々生させることを特徴とする請求項２に記載の画像再生制御装置。 The display control unit returns the playback position by a predetermined time longer than a predetermined time corresponding to the time required for one blink from the end time of the blink detected by the detection unit, and displays the series of images. The image reproduction control device according to claim 2, wherein the image reproduction control device is regenerated again.
  5. 　前記検出部は、前記操作者が目で前記表示画面を見なくなった状態の開始時間と終了時間とを検出し、
    　前記表示制御部は、少なくとも前記検出部が検出した前記操作者が目で前記表示画面を見なくなった状態の開始時間まで、再生位置を戻して前記一連の画像を再々生させることを特徴とする請求項１に記載の画像再生制御装置。     The detection unit detects a start time and an end time when the operator stops viewing the display screen with eyes,
    The display control unit returns the reproduction position and reproduces the series of images again at least until the start time when the operator detects the display screen with the eyes detected by the detection unit. The image reproduction control device according to claim 1.
  6. 　前記検出部は、前記操作者の瞬きの開始時間と終了時間とを検出する瞬き検出部であり、
    　前記表示制御部は、前記検出部が検出した瞬きの開始時間と終了時間との間の期間と、前記一連の画像の再生処理におけるフレームレートとをもとに、前記操作者の目が瞬きを行っていた間に前記表示装置で再生された画像のフレーム数を求め、前記検出部が検出した終了時間から、少なくとも前記求めたフレーム数分、再生位置を戻して前記一連の画像を再々生させることを特徴とする請求項５に記載の画像再生制御装置。     The detection unit is a blink detection unit that detects a start time and an end time of the operator's blink,
    The display control unit blinks the eyes of the operator based on a period between a blink start time and an end time detected by the detection unit and a frame rate in the series of image reproduction processes. The number of frames of the image reproduced on the display device while being performed is obtained, and the reproduction position is returned by at least the obtained number of frames from the end time detected by the detection unit, and the series of images is reproduced again. The image reproduction control apparatus according to claim 5, wherein
  7. 　前記表示制御部は、前記表示装置で再生されている画像が、前記少なくとも操作者が目で表示画面を見なくなったと推定される時点まで再生位置を戻して再々生された画像であることを報知する報知情報を含ませた画面で前記表示装置に前記一連の画像を再々生させることを特徴とする請求項１に記載の画像再生制御装置。 The display control unit notifies that an image being reproduced on the display device is an image regenerated by returning the reproduction position until at least the time when it is estimated that the operator no longer sees the display screen with eyes. The image reproduction control device according to claim 1, wherein the series of images are regenerated on the display device on a screen including notification information to be reproduced.
  8. 　前記一連の画像は、カプセル型内視鏡によって撮像された画像であって、
    　前記表示制御部は、前記表示装置に、前記カプセル型内視鏡によって撮像された画像を順次再生させることを特徴とする請求項１に記載の画像再生制御装置。     The series of images are images taken by a capsule endoscope,
    The image display control device according to claim 1, wherein the display control unit causes the display device to sequentially reproduce images picked up by the capsule endoscope.
  9. 　接続する表示装置に画像を再生させる画像再生制御装置が行う画像再生制御方法であって、
    　前記表示装置の表示画面に、一連の画像を順次再生させる画像再生ステップと、
    　前記一連の画像の再生中に、当該画像再生制御装置の操作者が目で前記表示画面を見ているか否かを検出する検出ステップと、
    　前記検出ステップにおいて、前記操作者が目で前記表示画面を見ていないと検出された場合に、前記一連の画像の再生位置を、少なくとも前記操作者が目で前記表示画面を見なくなったと推定される時点まで戻して前記表示装置に前記一連の画像を再々生させる画像再々生ステップと、
    　を含むことを特徴とする画像再生制御方法。     An image reproduction control method performed by an image reproduction control device for reproducing an image on a connected display device,
    An image reproduction step of sequentially reproducing a series of images on the display screen of the display device;
    A detection step of detecting whether or not an operator of the image reproduction control device is viewing the display screen with eyes during reproduction of the series of images;
    In the detection step, when it is detected that the operator does not look at the display screen with eyes, it is estimated that at least the operator has stopped looking at the display screen with eyes at the reproduction position of the series of images. An image regeneration step for causing the display device to reproduce the series of images by returning to a certain time point;
    An image reproduction control method comprising:
PCT/JP2015/059361 2014-07-09 2015-03-26 Image playback control device and image playback control method WO2016006284A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014141051 2014-07-09
JP2014-141051 2014-07-09

Publications (1)

Publication Number Publication Date
WO2016006284A1 true WO2016006284A1 (en) 2016-01-14

Family

ID=55063930

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/059361 WO2016006284A1 (en) 2014-07-09 2015-03-26 Image playback control device and image playback control method

Country Status (1)

Country Link
WO (1) WO2016006284A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019078237A1 (en) * 2017-10-18 2019-04-25 富士フイルム株式会社 Medical image processing device, endoscope system, diagnosis supporting device, and medical business supporting system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005317151A (en) * 2004-04-30 2005-11-10 Nippon Telegr & Teleph Corp <Ntt> Method, device and program for multimedia data synchronizing reproduction
JP2006247371A (en) * 2005-02-09 2006-09-21 Pentax Corp Electronic endoscope system
JP2008237369A (en) * 2007-03-26 2008-10-09 Canon Inc Image processing device and image processing method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005317151A (en) * 2004-04-30 2005-11-10 Nippon Telegr & Teleph Corp <Ntt> Method, device and program for multimedia data synchronizing reproduction
JP2006247371A (en) * 2005-02-09 2006-09-21 Pentax Corp Electronic endoscope system
JP2008237369A (en) * 2007-03-26 2008-10-09 Canon Inc Image processing device and image processing method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019078237A1 (en) * 2017-10-18 2019-04-25 富士フイルム株式会社 Medical image processing device, endoscope system, diagnosis supporting device, and medical business supporting system
US11426054B2 (en) 2017-10-18 2022-08-30 Fujifilm Corporation Medical image processing system, endoscope system, diagnosis support apparatus, and medical service support apparatus

Similar Documents

Publication Publication Date Title
JP2010035756A (en) Diagnosis support apparatus and diagnosis support method
EP2682050B1 (en) Medical information recording apparatus
JP4823659B2 (en) In vivo image display device
US8732546B2 (en) Radio receiver with an error correction code detector and with a correction unit
US20090135249A1 (en) Image display apparatus, image display method, and image display program
JP2008119145A (en) Image display method and image display apparatus
JP5914765B2 (en) Wireless transmission system
US20160323514A1 (en) Medical video recording and reproducing system and medical video recording and reproducing device
US20190133425A1 (en) Endoscope system, terminal device, server, and transmission method
JPWO2016072288A1 (en) Endoscope apparatus, operation method of endoscope apparatus, and program
US10932648B2 (en) Image processing apparatus, image processing method, and computer-readable recording medium
WO2016006284A1 (en) Image playback control device and image playback control method
JP2006296882A (en) Image display device
JP4815949B2 (en) Multi-display device and display device
JP4804739B2 (en) Image display device, image display method, and image display program
WO2012042966A1 (en) Image display device, image display method, and image display program
CN103927011B (en) Realize the operating system device and application program module of user&#39;s attitude detection
JP4891566B2 (en) Image display device
JP6129458B1 (en) Endoscope processor
JP5927369B2 (en) Endoscopic video system and video processor
WO2024070917A1 (en) Terminal device and measurement method
CN116801815A (en) Symptom recording device, symptom recording method, and program
CN115695715A (en) Monitoring equipment and medical monitoring system
JP2021117664A (en) Information processing device and program
KR101630364B1 (en) medical imaging system using endoscope and method for controlling the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15819583

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: JP

122 Ep: pct application non-entry in european phase

Ref document number: 15819583

Country of ref document: EP

Kind code of ref document: A1