WO2018216618A1 - 情報処理装置、制御方法、及びプログラム - Google Patents
情報処理装置、制御方法、及びプログラム Download PDFInfo
- Publication number
- WO2018216618A1 WO2018216618A1 PCT/JP2018/019310 JP2018019310W WO2018216618A1 WO 2018216618 A1 WO2018216618 A1 WO 2018216618A1 JP 2018019310 W JP2018019310 W JP 2018019310W WO 2018216618 A1 WO2018216618 A1 WO 2018216618A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- moving image
- image frame
- notification
- abnormal region
- predetermined range
- Prior art date
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00055—Operational features of endoscopes provided with output arrangements for alerting the user
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/045—Control thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Definitions
- Patent Documents 1 to 3 disclose a technique for displaying an image (CT image or MRI ⁇ image) obtained in the past (for example, one year ago) and an image obtained in this examination side by side.
- Patent Documents 1 and 4 disclose a technique for detecting a lesion from an image and marking the detected lesion.
- One method for inspecting the inside of a body is to use an endoscope system or the like to inspect the state of the inside of the body while watching a moving image drawn on a display device.
- a doctor inserts a scope having a camera at the tip thereof from the subject's nose, mouth, anus, etc., and moves the scope within the body. By doing so, the inside of the body is imaged by the camera.
- the doctor checks whether there is an abnormal part in the body of the subject while watching the moving image drawn on the display device showing the state of the body imaged by the camera.
- the part that can be observed by the doctor changes with time because the camera is moved in the body. Therefore, there is a possibility that a doctor may miss an abnormal part, and in fact, there is a difference in the lesion detection rate depending on the doctor in charge of the examination. In each of the above-described prior art documents, it is not assumed that the part that can be observed by the doctor changes with time.
- the control method of the present invention is executed by a computer.
- the control method includes: 1) a detection step of detecting an abnormal area in the body from a moving image taken inside the body; 2) the abnormal area is detected from a predetermined range of a first moving image frame of the moving image; A determination step of determining whether or not a predetermined condition is satisfied when the abnormal region is not detected from within the predetermined range of the second moving image frame of the moving image generated after one moving image frame; and 3) A notification step of performing a first notification when it is determined that the predetermined condition is not satisfied.
- the program of the present invention causes a computer to execute each step of the control method of the present invention.
- FIG. 1 is a diagram conceptually illustrating an information processing apparatus according to a first embodiment. It is a block diagram which illustrates the functional composition of an information processor. It is a figure which illustrates the computer for implement
- 3 is a flowchart illustrating a flow of processing executed by the information processing apparatus according to the first embodiment. It is a figure which illustrates the animation frame in which the 2nd predetermined range is set. It is a figure which illustrates the motion of the abnormal area
- FIG. 6 is a block diagram illustrating an information processing apparatus according to a second embodiment.
- FIG. 10 is a diagram illustrating a flow of processing executed by the information processing apparatus according to the second embodiment. It is a figure which illustrates the 2nd notification which highlights and displays the animation frame currently displayed on the 1st field. It is a figure which illustrates history information in a table format.
- each block diagram unless otherwise specified, each block represents a functional unit configuration, not a hardware unit configuration.
- FIG. 1 is a diagram conceptually illustrating the information processing apparatus 2000 according to the first embodiment.
- FIG. 1A conceptually shows the usage environment of the information processing apparatus 2000.
- FIG. 1B conceptually shows the operation of the information processing apparatus 2000.
- FIG. 1 merely shows an example of the operation for facilitating the understanding of the information processing apparatus 2000, and does not limit the functions of the information processing apparatus 2000 at all.
- the camera 10 is used for testing humans and other animals.
- the person to be examined is referred to as a subject.
- the camera 10 is an arbitrary camera capable of imaging the inside of the subject, and generates a moving image frame 14 representing the imaging result.
- the camera 10 is an endoscopic camera.
- the moving image data 12 is composed of a plurality of moving image frames 14 generated at different times.
- the moving image data 12 generated by the camera 10 is displayed on the display device 20.
- the display device 20 is an arbitrary display device that can display the moving image data 12. Note that the display of the moving image data 12 on the display device 20 means that the plurality of moving image frames 14 constituting the moving image data 12 are displayed on the display device 20 in order.
- the user grasps the state of the subject's body by viewing the moving image data 12 displayed on the display apparatus 20. More specifically, the user grasps whether or not there is an abnormal part in the body of the subject and the degree of the abnormality.
- the “abnormal part in the body” is, for example, a part having a lesion, a part having a wound, a part having a foreign object, or the like.
- a lesion is a change in a living body caused by a disease, such as a tumor.
- the doctor may miss it even though the abnormal part was imaged by the camera is there. For this reason, it is preferable to provide support so that the doctor can easily grasp the abnormal part to prevent the abnormal part from being overlooked.
- the information processing apparatus 2000 of the present embodiment performs the following operation.
- the information processing apparatus 2000 acquires the moving image data 12 and performs image analysis on the moving image frame 14 constituting the moving image data 12.
- the information processing apparatus 2000 detects the abnormal region 30 from the predetermined range 16 of the moving image frame 14.
- the abnormal region 30 is a region presumed to represent an abnormal part in the body of the subject.
- the abnormal region 30 in FIG. 1 is a region including a tumor (region representing a lesion).
- the predetermined range 16 may be the entire moving image frame 14 or an image area smaller than the entire moving image frame 14 (part of the moving image frame 14).
- the abnormal region 30 is detected from a predetermined range 16 of a certain moving image frame 14 and the abnormal region 30 is not detected from the predetermined range 16 of the moving image frame 14 generated after the moving image frame 14.
- the abnormal region 30 enters (frames in) the whole or part of the imaging range of the camera 10 and then moves out (out of the frame) from the whole or part of the imaging range of the camera 10.
- the information processing apparatus 2000 determines whether or not a predetermined condition is satisfied, and performs a predetermined notification (hereinafter, first notification) when the predetermined condition is not satisfied.
- This predetermined condition is a condition that is satisfied when the user of the information processing apparatus 2000 is assumed to recognize the abnormal region 30 included in the moving image data 12.
- the predetermined condition is a condition that is not satisfied when it is estimated that the user of the information processing apparatus 2000 does not recognize the abnormal region 30 included in the moving image data 12.
- the predetermined range 16 is displayed with a dotted line for the sake of explanation, but a frame line representing the predetermined range 16 is not necessarily displayed on the display device 20. The same applies to the subsequent drawings.
- FIG. 1B shows a screen displayed on the display device 20 at a time point “t” and a screen displayed on the display device 20 at a subsequent time point “t + a”.
- An abnormal region 30 is included in the predetermined range 16 of the moving image frame 14 displayed on the display device 20 at the time point t.
- the abnormal area 30 is not included in the predetermined range 16 of the moving image frame 14 displayed on the display device 20 at the time point t + a.
- the information processing apparatus 2000 determines whether or not the predetermined condition described above is satisfied.
- the predetermined condition is not satisfied in the example of FIG. That is, it is assumed that the user has not recognized the abnormal region 30. Therefore, the information processing apparatus 2000 performs the first notification described above. For example, in FIG. 1, a first notification is made that the frame of the moving image frame 14 is thickened and emphasized.
- the information processing apparatus 2000 when the abnormal area 30 is not included in the predetermined range of the moving image data 12, and it is estimated that the user does not recognize the abnormal area 30, a notification is issued. To be done. By doing so, the user can be made aware of the presence of the abnormal region 30 that has not been recognized, and thus the user can be prevented from overlooking the abnormal region 30. Therefore, the accuracy of the in-vivo examination performed using the camera 10 can be improved.
- FIG. 2 is a block diagram illustrating the functional configuration of the information processing apparatus 2000.
- the information processing apparatus 2000 includes a detection unit 2020, a determination unit 2040, and a notification unit 2060.
- the detection unit 2020 detects the abnormal region 30 from the moving image data 12.
- the determination unit 2040 detects an abnormal region 30 from a predetermined range 16 of a certain moving image frame 14 (referred to as a first moving image frame) of the moving image data 12, and the moving image frame 14 (second image) generated after the moving image frame 14 is detected.
- a predetermined range 16 referred to as a first moving image frame
- the notification unit 2060 performs the first notification described above when it is determined that the predetermined condition is not satisfied.
- Each functional component of the information processing apparatus 2000 may be realized by hardware (eg, a hard-wired electronic circuit) that implements each functional component, or a combination of hardware and software (eg: It may be realized by a combination of an electronic circuit and a program for controlling it).
- hardware eg, a hard-wired electronic circuit
- software eg: It may be realized by a combination of an electronic circuit and a program for controlling it.
- FIG. 3 is a diagram illustrating a computer 1000 for realizing the information processing apparatus 2000.
- the computer 1000 is a variety of computers.
- the computer 1000 is a Personal Computer (PC), a server machine, a tablet terminal, or a smartphone.
- the computer 1000 may be a dedicated computer designed for realizing the information processing apparatus 2000 or a general-purpose computer.
- the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120.
- the bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit / receive data to / from each other.
- the processor 1040 is an arithmetic processing unit such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).
- the memory 1060 is a main storage device realized by RAM (Random Access Memory) or the like.
- the storage device 1080 is an auxiliary storage device realized by a hard disk, an SSD (Solid State Drive), a ROM, a memory card, or the like. However, the storage device 1080 may be realized by hardware similar to the hardware used for realizing the main storage device, such as a RAM.
- the network interface 1120 is an interface for connecting to a communication network such as a WAN (Wide Area Network) or a LAN (Local Area Network).
- a communication network such as a WAN (Wide Area Network) or a LAN (Local Area Network).
- FIG. 4 is a diagram illustrating a specific example of the usage environment of the information processing apparatus 2000.
- the information processing apparatus 2000 is used together with the scope 40 and the endoscope system 50.
- the scope 40 is connected to the endoscope system 50.
- the scope 40 is provided with a camera 10.
- the moving image data 12 includes a plurality of moving image frames 14 generated by the camera 10 provided in the scope 40.
- the endoscope system 50 outputs the moving image data 12 to the information processing apparatus 2000.
- the moving image data 12 is sent from a video output interface (for example, a High-Definition Multimedia Interface (HDMI) (registered trademark) interface) provided in the endoscope system 50 to a video input interface of the information processing apparatus 2000. Is output. Then, the information processing apparatus 2000 processes the moving image data 12 acquired from the endoscope system 50 and performs a first notification.
- a video output interface for example, a High-Definition Multimedia Interface (HDMI) (registered trademark) interface
- HDMI High-Definition Multimedia Interface
- the first notification may be a display performed using the display device 20, or may be other than that.
- the information processing device 2000 controls the display device 20 to display the moving image data 12 including the first notification on the display device 20.
- the process of displaying the moving image data 12 on the display device 20 may be performed by the information processing device 2000 or may be performed by another device (for example, the endoscope system 50).
- the display device 20 does not need to be connected to the information processing device 2000.
- the configuration illustrated in FIG. 4 is merely an example, and the usage environment of the information processing apparatus 2000 is not limited to the configuration illustrated in FIG.
- the moving image data 12 may be output from the camera 10 to the information processing apparatus 2000.
- the information processing apparatus 2000 may not be connected to the endoscope system 50.
- FIG. 5 is a flowchart illustrating the flow of processing executed by the information processing apparatus 2000 according to the first embodiment.
- S ⁇ b> 102 to S ⁇ b> 114 are loop processes executed for each moving image frame 14 acquired from the camera 10.
- the information processing apparatus 2000 selects the moving image frame 14 with the earliest generation time among the moving image frames 14 not yet subjected to the loop process A.
- the moving image frame 14 selected here is referred to as a moving image frame i. Note that, when the loop process A has already been executed for all the moving image frames 14, for example, the information processing apparatus 2000 waits until a new moving image frame 14 is generated. In addition, in this case, the process of FIG. 5 may be terminated.
- the detection unit 2020 detects the abnormal region 30 from the predetermined range 16 of the moving image frame i (S104). When the abnormal region 30 is detected from the predetermined range 16 of the moving image frame i (S106: YES), the process of FIG. 5 proceeds to S114. When the abnormal region 30 is not detected from the predetermined range 16 of the moving image frame i (S106: NO), the determination unit 2040 detects the abnormal region 30 from the predetermined range 16 of the moving image frame that is the target of S104 before the moving image frame i. It is determined whether or not it has been detected (S108). When the abnormal region 30 is not detected from the predetermined range 16 of the moving image frame that is the target of S104 before the moving image frame i (S108: NO), the process of FIG. 5 proceeds to S114.
- the determination unit 2040 determines whether or not the predetermined condition is satisfied. (S110). When the predetermined condition is not satisfied (S110: NO), the notification unit 2060 performs the first notification (S112). On the other hand, when the predetermined condition is satisfied (S110: YES), the process of FIG. 5 proceeds to S114.
- S114 is the end of loop processing A. Therefore, the process in FIG. 5 proceeds to S102 after S114.
- the moving image frame 14 that is the target of the process of detecting the abnormal region 30 may be all the moving image frames 14 included in the moving image data 12, or may be a part of the moving image frames 14. Good. In the latter case, for example, the detection unit 2020 executes S104 only for one moving image frame 14 for every predetermined number (for example, one for every 10 frames).
- the detection unit 2020 detects the abnormal region 30 from each moving image frame 14 constituting the moving image data 12.
- an existing technique can be used as a technique for analyzing an image of the inside of the body and detecting an abnormal part.
- techniques such as feature amount matching and template matching can be used.
- feature amount matching when a tumor is detected by feature amount matching, one or more values (feature amounts) representing features of the appearance (color, pattern, shape, etc.) of the tumor are defined in advance.
- the detection unit 2020 detects, from the moving image frame 14, an image region having a high similarity to a predetermined tumor feature amount from the image regions of the moving image frame 14. Then, the detection unit 2020 treats the detected image area as an image area representing the abnormal area 30.
- a similar method can be adopted for cases in which scratches or foreign objects are detected.
- the determination unit 2040 specifies that the abnormal region 30 detected from the predetermined range 16 of the first moving image frame is not detected from the predetermined range 16 of the moving image frame 14 generated thereafter (S106). In other words, the determination unit 2040 detects, from the predetermined range 16 among the moving image frames 14 generated after the first moving image frame, that the abnormal region 30 detected from the predetermined range 16 of the first moving image frame is not detected.
- the second moving image frame is specified.
- the determination unit 2040 performs tracking of the abnormal region 30. Specifically, the determination unit 2040 performs tracking on each of the moving image frames 14 generated thereafter for the abnormal region 30 detected from the predetermined range 16 of the first moving image frame. Then, the determination unit 2040 identifies the moving image frame 14 in which the tracking target abnormal region 30 is not included in the predetermined range 16 as the second moving image frame. Note that the determination unit 2040 treats the moving image frame 14 as the first moving image frame when the abnormal region 30 is detected from a certain moving image frame 14 and the abnormal region 30 has not been a tracking target.
- An existing technique can be used as a technique for tracking an object detected from moving image data.
- the determination unit 2040 may track an image area representing the abnormal area 30 or may track an image area having a predetermined size or shape including the periphery of the abnormal area 30. Further, the determination unit 2040 may track an image area around the abnormal area 30 instead of the abnormal area 30.
- the method by which the determination unit 2040 identifies the second frame is not limited to tracking.
- the determination unit 2040 searches for the feature amount of the image region representing the abnormal region 30 detected from the predetermined range 16 of the first moving image frame within the predetermined range 16 of each moving image frame 14 generated thereafter, and the feature.
- One of the moving image frames 14 that do not include the image area having the amount may be a second moving image frame.
- the determination unit 2040 After the abnormal region 30 is detected from the predetermined range 16 of a certain moving image frame 14 (S106: YES), the determination unit 2040 detects the abnormality from the predetermined range 16 of the moving image frame 14 generated after that moving image frame 14. When the region 30 is not detected (S108: NO), it is determined whether or not the predetermined condition described above is satisfied (S110).
- Various conditions can be adopted as the predetermined condition. Hereinafter, this predetermined condition will be exemplified.
- the predetermined range 16 can be an arbitrary range included in the image area of the moving image frame 14, or can be the entire screen area of the moving image frame 14. Which range in the moving image frame 14 the predetermined range 16 represents may be preset in the information processing apparatus 2000 or may be stored in a storage device accessible from the information processing apparatus 2000.
- Example 1 of predetermined conditions When the abnormal area 30 included in the moving image data 12 is recognized by the user, the user tries to observe the abnormal area 30 in detail by stopping the movement of the camera 10 or slowing the moving speed of the camera 10. The probability is high. Therefore, when the abnormal area 30 is recognized by the user, the moving speed of the abnormal area 30 in the moving image data 12 is considered to be relatively slow. In other words, when the abnormal area 30 is not recognized by the user, the moving speed of the abnormal area 30 in the moving image data 12 is considered to be relatively fast.
- a predetermined condition is “a moving image data 12 from the first moving image frame to the second moving image frame has a statistical value indicating the magnitude of the moving speed of the abnormal region 30 included in the predetermined range 16 being a predetermined value or less.
- the statistical value indicating the magnitude of the moving speed is, for example, a minimum value, a maximum value, or an average value.
- the predetermined value may be set in the determination unit 2040 in advance, or may be stored in a storage device accessible from the determination unit 2040.
- the first notification by the notification unit 2060 is not performed. Therefore, for example, when the user recognizes the abnormal region 30 and slows down the moving speed of the camera 10 or temporarily stops moving the camera 10, the first notification by the notification unit 2060 is not performed.
- the notification unit 2060 performs the first notification. Therefore, for example, when the user keeps moving the camera 10 quickly without recognizing the abnormal region 30, the first notification by the notification unit 2060 is performed. Therefore, it is possible to prevent the user from overlooking the abnormal area 30.
- an existing technique can be used as a technique for calculating moving speed of an object (abnormal region 30 in the present invention) detected from the moving picture data by analyzing the moving picture data.
- the determination unit 2040 determines that the predetermined condition is satisfied. Therefore, the first notification by the notification unit 2060 is not performed. On the other hand, when the predetermined action by the user is not detected during the above period, the determination unit 2040 determines that the predetermined condition is not satisfied. Thus, the first notification by the notification unit 2060 is performed.
- the predetermined action described above a user who has recognized the abnormal area 30 can perform an input operation such as keyboard input or pressing a shutter button in order to store the moving image frame 14 including the abnormal area 30 in the storage device. Therefore, the input operation for storing the moving image frame 14 in the storage device is defined as the predetermined action.
- the determination unit 2040 treats all the abnormal areas 30 included in the predetermined range 16 of the moving image frame 14 stored in the storage device as the target of the input operation.
- a user who has recognized the abnormal area 30 can perform a character input operation in order to record findings regarding the abnormal area 30. Therefore, such a character input operation is defined as the predetermined action.
- An existing technique can be used as a technique for recognizing such a character input operation.
- the determination unit 2040 uses the designated abnormal area 30 as the target of the user's predetermined action. Treat as.
- the determination unit 2040 when the character input operation is performed without designating the abnormal region 30, for example, the determination unit 2040, when the character input operation is performed, performs all the operations included in the predetermined range 16 of the moving image frame 14.
- the abnormal region 30 is treated as the abnormal region 30 that is the target of the user's predetermined action.
- the determination unit 2040 treats the abnormal area 30 pointed by the pointing operation as the abnormal area 30 that is the target of the user's predetermined action.
- a user who has recognized the abnormal region 30 may 1) act to change the color or intensity of the light irradiated to the abnormal region 30 or its surroundings, and 2) to disperse or stain pigment in the abnormal region 30 or its peripheral region. 3) An act of administering a drug to the abnormal region 30 and its surroundings, or 4) An act of collecting the abnormal region 30 and its surrounding tissues. Therefore, for example, such an action is set as the predetermined action.
- the above actions 1) to 4) are performed when the user performs a predetermined input operation on the endoscope system or the like.
- a mechanism light source or the like
- a mechanism that irradiates light to a scope provided with a camera a mechanism that ejects pigment or staining liquid, a mechanism that administers water or medicine, and a tissue are collected.
- a mechanism for this is provided.
- These mechanisms operate according to a predetermined input operation performed on the endoscope system by the user. In other words, when the various actions described above are performed, an input operation for operating a mechanism that realizes the actions is performed.
- the determination unit 2040 sets the abnormal region 30 included in the predetermined range 16 of the moving image frame 14 determined to have changed in brightness and color as a target of the user's predetermined action. Treated as an abnormal region 30.
- the area within the predetermined range 16 and outside the second predetermined range 18 is a portion deviating from the center of the imaging range of the camera 10.
- the region closer to the center of the display screen is more easily recognized by the user. Therefore, it can be said that the region within the predetermined range 16 and outside the second predetermined range 18 is a region that is difficult for the user to recognize. Therefore, there is a high probability that the abnormal region 30 that has not entered the predetermined range 16 is not recognized by the user.
- the predetermined condition is “the abnormal region 30 from the second predetermined range 18 of at least one moving image frame 14 out of one or more moving image frames 14 generated between the first moving image frame and the second moving image frame. Is detected ”.
- the first notification by the notification unit 2060 is performed when the abnormal region 30 enters the predetermined range 16 but goes out of the predetermined range 16 without entering the second predetermined range 18. . Therefore, the first notification by the notification unit 2060 is performed under a high probability that the user does not recognize the abnormal region 30. Therefore, it is possible to prevent the abnormal region 30 from being overlooked by the user.
- FIG. 7 is a diagram illustrating the movement of the abnormal region 30 in the moving image data 12.
- the abnormal region 30 enters the predetermined range 16, passes through the second predetermined range 18, and exits from the predetermined range 16. Therefore, in this case, notification by the notification unit 2060 is not performed.
- the determination unit 2040 may use only one of the predetermined conditions described above, or may use any two or more. In the latter case, for example, the determination unit 2040 determines that “the predetermined condition is satisfied” when at least one of the plurality of predetermined conditions is satisfied, and none of the plurality of predetermined conditions is satisfied It is determined that “the predetermined condition is not satisfied”. In this case, the first notification is performed when none of the plurality of predetermined conditions is satisfied.
- the determination unit 2040 determines that “the predetermined condition is satisfied” when all of the plurality of predetermined conditions are satisfied, and any one of the plurality of predetermined conditions is not satisfied. In this case, it is determined that “the predetermined condition is not satisfied”. In this case, the first notification is performed when any one of the plurality of predetermined conditions is not satisfied.
- the determination unit 2040 determines that “the predetermined condition is satisfied” when the number of the predetermined conditions that are satisfied among the plurality of predetermined conditions is equal to or greater than a predetermined value, and is satisfied. When the number of the predetermined conditions is less than the predetermined value, it is determined that “the predetermined conditions are not satisfied”. In this case, the first notification is performed when the number of predetermined conditions that are satisfied is less than a predetermined value.
- the notification unit 2060 outputs a predetermined sound such as a beep sound or a voice message from the speaker as the first notification.
- a speaker is connected to the information processing apparatus 2000.
- This predetermined sound may be preset in the notification unit 2060 or may be stored in a storage device accessible from the notification unit 2060.
- the notification unit 2060 causes the display device 20 to display the video data 12 as a first notification.
- the highlighted display of the moving image data 12 is a display in which a frame line surrounding the moving image data 12 is displayed or a frame line of the already displayed moving image data 12 is thickened.
- the highlighted display of the moving image data 12 is a display for blinking the moving image data 12 and its frame line.
- FIG. 8 is a diagram illustrating highlighting of the moving image data 12.
- FIG. 8A shows the display device 20 before the first notification is performed.
- FIG.8 (b) is the display apparatus 20 after the 1st notification is performed.
- FIG. 8B the thickness of the frame line surrounding the moving image data 12 is thicker than in the case of FIG.
- the notification unit 2060 may cause the display device 20 to display the position where the abnormal region 30 was last detected in the predetermined range 16 of the moving image data 12 after the first moving image frame as the first notification.
- FIG. 9 is a diagram illustrating a display showing a position within the predetermined range 16 where the abnormal region 30 was last detected.
- the predetermined range 16 is the entire image area of the moving image frame 14.
- FIG. 9A shows a locus of movement of the abnormal region 30 in the moving image data 12.
- FIG. 9B illustrates the first notification performed when the abnormal region 30 moves along the trajectory illustrated in FIG.
- the position of the abnormal region 30 when the abnormal region 30 is out of the predetermined range 16 is a position 80 (see FIG. 9A). Therefore, the notification unit 2060 displays a display 90 indicating the position 80 on the display device 20.
- the notification unit 2060 may cause the display device 20 to display a display serving as a guide for putting the abnormal region 30 that has exited from the predetermined range 16 into the predetermined range 16 again as the first notification.
- FIG. 10 is a diagram illustrating the above-described guide.
- FIG. 10A shows the movement trajectory of the abnormal region 30 in the moving image data 12.
- FIG. 10B illustrates the first notification performed when the abnormal region 30 moves along the trajectory illustrated in FIG.
- the abnormal area 30 When the abnormal area 30 moves out of the predetermined range 16, the abnormal area 30 can be re-entered within the predetermined range 16 by tracking the movement of the abnormal area 30. Therefore, the user can move the imaging range of the camera 10 in the same direction as the direction in which the abnormal region 30 has moved, and put the abnormal region 30 in the predetermined range 16.
- the moving direction of the abnormal region 30 when it goes out of the predetermined range 16 is represented by a direction 100.
- FIG. 10B the notification unit 2060 displays the guide 110 (first notification) indicating the direction 100 on the display device 20. By doing so, the user can easily perform an operation for putting the abnormal region 30 in the predetermined range 16 again.
- This guide display is particularly effective in the case where the predetermined range 16 is the entire image area of the moving image frame 14. This is because the abnormal region 30 outside the predetermined range 16 is not displayed on the display device 20 in this case.
- an existing technique such as a method for calculating an optical flow can be used.
- the notification unit 2060 may cause the display device 20 to display the moving image frame 14 between the first moving image frame and the second moving image frame and including the abnormal region 30 within the predetermined range 16 as the first notification.
- FIG. 11 is a diagram illustrating a state in which the moving image frame 14 in which the abnormal region 30 is included in the predetermined range 16 is displayed as the first notification.
- FIG. 11A shows the locus of the abnormal region 30 in the moving image data 12.
- the abnormal region 30 is included in the predetermined range 16 between the time point t1 and the time point t2. Therefore, the notification unit 2060 causes the display device 20 to display one of the moving image frames 14 generated between the time point t1 and the time point t2.
- FIG. 11 is a diagram illustrating a state in which the moving image frame 14 in which the abnormal region 30 is included in the predetermined range 16 is displayed as the first notification.
- FIG. 11A shows the locus of the abnormal region 30 in the moving image data 12.
- the abnormal region 30 is included in the predetermined range 16 between the time
- 11B shows a state in which the moving image frame 14 generated between the time point t1 and the time point t2 is displayed on the display device 20.
- the information processing device 2000 displays the moving image frame 14 displayed as the first notification in the first region 22 of the display device 20.
- the information processing apparatus 2000 displays the moving image data 12 in the second area 24 of the display device 20.
- the moving image frame 14 displayed in the first region 22 is a past moving image frame 14 including the abnormal region 30 that is not likely to be recognized by the user. It can be said that there is.
- the moving image frame 14 including the abnormal region 30 having a high probability of not being recognized by the user on the display device 20 together with the moving image data 12 the user can see the first region 22 of the display device 20.
- the abnormal region 30 can be recognized later. Therefore, it is possible to prevent the user from overlooking the abnormal part.
- the notification unit 2060 displays, in the first region 22, the last (newest) moving image frame 14 including the abnormal region 30 among the moving image frames 14 generated between the first moving image frame and the second moving image frame.
- the moving image frame 14 is determined. According to this method, the user can grasp the last position where the abnormal region 30 was imaged by the camera 10.
- the notification unit 2060 identifies, from among the moving image frames 14 generated between the first moving image frame and the second moving image frame, the one in which the position of the abnormal region 30 is closest to the center position of the moving image frame 14
- the moving image frame 14 that has been set as the moving image frame 14 to be displayed in the first area 22 is used.
- the notification unit 2060 calculates the distance between the abnormal region 30 included in the moving image frame 14 and the center coordinates of the moving image frame 14 for each moving image frame 14. Then, the notification unit 2060 determines the moving image frame 14 having the smallest distance as the moving image frame 14 to be displayed in the first region 22.
- an object included in an image generated by a camera is easier to see as the object is closer to the center of the image. Therefore, by displaying the moving image frame 14 in which the position of the abnormal region 30 is close to the center position of the moving image frame 14 in the first region 22 in this manner, the abnormal region 30 can be easily seen by the user.
- the greater the contrast of an image the easier it is to identify individual objects contained in the image. Therefore, by displaying the moving image frame 14 having a large contrast of the entire moving image frame 14 in the first region 22 in this manner, the abnormal region 30 can be easily seen by the user.
- the notification unit 2060 may use the contrast of the image region representing the abnormal region 30 instead of the contrast of the entire moving image frame 14. That is, the notification unit 2060 calculates the index value of the contrast of the image region representing the abnormal region 30 for each moving image frame 14 generated between the first moving image frame and the second moving image frame. Then, the notification unit 2060 identifies the moving image frame 14 having the highest contrast of the image region representing the abnormal region 30 by comparing the calculated index values, and displays the identified moving image frame 14 in the first region 22.
- the inside of the abnormal area 30 can be easily seen by the user.
- the notification unit 2060 may store the moving image frame 14 to be displayed in the first area 22 described above in the storage area, instead of being displayed in the first area 22 or in the first area 22.
- the storage device that stores the moving image frame 14 is referred to as an image storage unit 120.
- FIG. 12 is a diagram illustrating an information processing apparatus 2000 connected to the image storage unit 120. Note that the image storage unit 120 may be provided inside the information processing apparatus 2000.
- the method for determining the moving image frame 14 to be stored in the image storage unit 120 is the same as the method for determining the moving image frame 14 to be displayed in the first area.
- the notification unit 2060 may appropriately process the moving image frame 14 stored in the image storage unit 120.
- the notification unit 2060 causes the image storage unit 120 to store the moving image frame 14 on which an image in which the position of the abnormal region 30 is known (a superimposition mark 60 described later) is superimposed. By doing so, it is possible to easily grasp where the abnormal region 30 is in the moving image frame 14.
- the notification unit 2060 causes the image storage unit 120 to store the moving image frame 14 and information for specifying the position of the abnormal region 30 included in the moving image frame 14 in association with each other.
- the moving image frame 14 may be stored in the storage device when the user operates a keyboard, a shutter button, or the like.
- an endoscope system is provided with a shutter button, and when a user presses the shutter button, a moving image frame generated by the camera at the timing of the pressing is stored in a storage device.
- the information processing apparatus 2000 may store the moving image frame 14 specified by the operation in the image storage unit 120 in accordance with the user operation.
- the moving image frame 14 stored in the image storage unit 120 according to the user's operation in this manner is referred to as a designated storage frame.
- the moving image frame 14 (the moving image frame 14 automatically stored in the image storage unit 120 by the information processing apparatus 2000) stored in the image storage unit 120 by the notification unit 2060 is referred to as an automatic storage frame.
- the method for storing the designated storage frame and the automatic storage frame in the image storage unit 120 in an identifiable manner is arbitrary.
- the information processing apparatus 2000 stores the flag indicating whether the moving image frame 14 is the designated storage frame or the automatic storage frame in the image storage unit 120 in association with the moving image frame 14.
- FIG. 13 is a diagram illustrating the format of information stored in the image storage unit 120 in a table format.
- the table shown in FIG. The table 200 has two columns of a moving image frame 202 and a type flag 204.
- the moving image frame 202 represents the moving image frame 14 itself.
- the type flag 204 indicates whether the moving image frame 14 indicated in the moving image frame 202 is a designated storage frame or an automatic storage frame.
- the method of storing the designated storage frame and the automatic storage frame in an identifiable manner is not limited to the method of providing the flag described above.
- the information processing apparatus 2000 may store the designated storage frame and the automatic storage frame in different storage devices.
- the information processing device 2000 may have a function of displaying the moving image frame 14 displayed on the display device 20 to represent the abnormal region 30 included in the moving image frame 14. By doing so, the user can easily grasp the abnormal region 30 included in the moving image frame 14.
- this display is referred to as a first display.
- the information processing apparatus 2000 displays a predetermined mark on the abnormal region 30 so as to be superimposed on the abnormal region 30 of the moving image frame 14 displayed on the first region 22.
- this mark is referred to as a superimposed mark.
- the superimposed mark is the first display.
- FIG. 14 is a diagram illustrating various superimposed marks 60 that are superimposed on the abnormal region 30.
- the information processing apparatus 2000 may perform a first display (hereinafter, an instruction mark) indicating the abnormal region 30 in the vicinity of the moving image frame 14.
- FIG. 15 is a diagram illustrating an instruction mark 70 that points to the abnormal region 30.
- FIG. 16 is a block diagram illustrating an information processing apparatus 2000 according to the second embodiment. Except for the items described below, the information processing apparatus 2000 according to the second embodiment is the same as the information processing apparatus 2000 according to the first embodiment.
- the information processing apparatus 2000 includes a second notification unit 2080.
- the second notification unit 2080 performs a predetermined notification when the abnormal area 30 that is the target of the first notification is detected from the predetermined range 16 of the moving image frame 14 generated after the second moving image frame. This predetermined notification is called a second notification.
- the fact that “the abnormal region 30 targeted for the first notification has been detected from the predetermined range 16 of the moving image frame 14 generated after the second moving image frame” indicates that there is a certain abnormality without being recognized by the user. This means that, in a situation where there is a high probability that the area 30 has moved out of the predetermined range 16, the abnormal area 30 has entered the predetermined range 16 again. It is considered that the user who has received the first notification by the notification unit 2060 operates the camera 10 and tries to put the abnormal area 30 as the target of the first notification within the predetermined range 16. Therefore, the fact that the abnormal region 30 has entered the predetermined range 16 means that the user has been able to operate the camera 10 as intended. Therefore, if any notification is made to the user so that the user can recognize this, the convenience of the information processing apparatus 2000 is enhanced for the user. Therefore, the information processing apparatus 2000 according to the present embodiment performs the second notification described above.
- the information processing apparatus 2000 uses the second moving image frame generated after the first moving image frame in which the abnormal region 30 is detected for each abnormal region 30 detected from the moving image data 12. If the abnormal region 30 is not detected from the above and the predetermined condition is not satisfied, the first notification for the abnormal region 30 is performed.
- the notification unit 2060 stores information for specifying the abnormal region 30 that is the target of the first notification in the storage device when the first notification is performed. This information is called history information.
- the second notification unit 2080 determines whether or not the abnormal area 30 is indicated in the history information. When the abnormal area 30 is indicated in the history information, the second notification unit 2080 makes a second notification for the abnormal area 30. On the other hand, when the abnormal area 30 is not indicated in the history information, the second notification unit 2080 does not perform the second notification for the abnormal area 30.
- FIG. 17 is a diagram illustrating a flow of processing executed by the information processing apparatus 2000 according to the second embodiment. A series of processing shown in FIG. 17 is executed after the abnormal region 30 is detected from the moving image frame 14 by the detection unit 2020 (S106: YES).
- the second notification unit 2080 determines whether or not the abnormal region 30 detected by the detection unit 2020 is included in the history information (S202). When the abnormal area 30 is included in the history information (S202: YES), the second notification unit 2080 makes a second notification for the abnormal area 30 (S204). On the other hand, when the abnormal area 30 is not included in the history information (S202: NO), the process of FIG. 17 ends.
- Second notification example 1 The second notification unit 2080 outputs a predetermined sound from the speaker as the second notification.
- the method of using a predetermined voice as the notification is as described in the first embodiment as “First notification example 1”.
- the first notification is also a voice, it is preferable to use different voices for the first notification and the second notification.
- Second notification example 2 makes the display apparatus 20 perform the display which emphasizes the moving image data 12 as a 2nd notification.
- the method for emphasizing the moving image data 12 is as described in the first embodiment as “Example 2 of first notification”.
- the first notification is also displayed to emphasize the moving image data 12, it is preferable to make a difference in the aspect in which the moving image data 12 is emphasized between the first notification and the second notification.
- the first notification is a display that thickens the frame of the moving image data 12
- the second notification is a display that blinks the moving image data 12.
- ⁇ second notification example 3 As a premise, as described in “Example 5 of the first notification”, as the first notification, “the moving image frame 14 between the first moving image frame and the second moving image frame and the abnormal area 30 is within the predetermined range 16. It is assumed that “included” is displayed in the first area 22 of the display device 20. That is, the moving image frame 14 in which the abnormal region 30 having a high probability that is not recognized by the user is included in the predetermined range 16 is displayed in the first region 22.
- FIG. 18 is a diagram illustrating a second notification that highlights the moving image frame 14 displayed in the first region 22.
- two moving image frames 14 (moving image frame 14-1 and moving image frame 14-2) are displayed in the first area 22.
- the abnormal areas 30 included in these moving image frames 14 are different from each other. This means that there are a plurality of abnormal areas 30 that are not recognized by the user.
- the abnormal region 30-3 is detected from the predetermined range 16 of the moving image frame 14-3 (the latest moving image frame 14 generated by the camera 10) displayed in the second region.
- the abnormal area 30 is assumed to be the same as the abnormal area 30-2 included in the moving image frame 14-2.
- the second notification unit 2080 performs highlighting to thicken the frame of the moving image frame 14-2 as the second notification.
- the user can grasp that the abnormal area 30 included in the predetermined range 16 of the imaging range of the camera 10 is the same as the abnormal area 30 included in the moving image frame 14-2. .
- the user can grasp which abnormal region 30 is currently confirmed by the moving image data 12 among the abnormal regions 30 that the user has not recognized.
- the notification unit 2060 stores information that can identify the abnormal region 30 that is the target of the first notification in the history information.
- this information is a feature amount of an image area representing the abnormal area 30 (for example, a parameter representing the color, shape, or pattern of the image area).
- a specific example of the structure of the history information will be shown.
- FIG. 19 is a diagram illustrating history information in a table format.
- the table 300 has two columns of an abnormal area identifier 302 and data 304.
- the abnormal area identifier 302 is an identifier assigned to the abnormal area 30.
- the data 304 indicates a set of “a feature amount of the abnormal region 30 and an identifier of the moving image frame 14 in which the abnormal region 30 is detected”.
- the notification unit 2060 performs the first notification, for the abnormal region 30 that is the target of the first notification, the combination of “the feature amount of the abnormal region 30 and the identifier of the moving image frame 14 in which the abnormal region 30 is detected”. Is generated in the data 304 and stored in the table 300.
- the second notification unit 2080 searches the table 300 for a record indicating a feature amount similar to the feature amount of the abnormal region 30 detected from the predetermined range 16 of the moving image frame 14. Then, when there is a record indicating a feature amount similar to the feature amount of the abnormal region 30 detected from the moving image frame 14, the second notification unit 2080 performs the second notification. On the other hand, when there is no such record, the second notification unit 2080 does not perform the second notification.
- the feature amount of the abnormal region 30 detected from the predetermined range 16 of the moving image frame 14 by the detection unit 2020 is v6. Further, it is assumed that this ⁇ v6 ⁇ is similar to the feature quantity v3 shown in the table 300 of FIG.
- the second notification unit 2080 searches the table 300 for a feature quantity similar to v6, as a result, the record in the third row of the table 300 in FIG. 19 is found. Therefore, the second notification unit 2080 performs the second notification. For example, when the moving image frame 14 having the identifier img3 is displayed in the first area 22 of the display device 20, the second notification unit 2080 performs a second notification that highlights the moving image frame 14.
- the notification unit 2060 may store information obtained from a plurality of moving image frames 14 representing the same abnormal region 30 in one record of history information. For example, as described above, when the abnormal region 30 detected from the first moving image frame is tracked by each moving image frame 14 generated thereafter, the abnormal region 30 can be detected from the plurality of moving image frames 14. Therefore, a set of “feature amount of abnormal region 30 and identifier of moving image frame 14” for each of the plurality of moving image frames 14 in which the abnormal region 30 that is the target of the first notification is detected is stored in one record of the table 300. Keep it.
- the feature amount of the abnormal region 30 may vary depending on the angle at which the abnormal region 30 is captured. Therefore, by storing a plurality of feature amounts for the abnormal region 30 that is the target of the first notification, the abnormal region 30 that should be the target of the second notification can be identified with higher accuracy.
- the hardware configuration of a computer that implements the information processing apparatus 2000 according to the second embodiment is represented by, for example, FIG.
- the storage device 1080 of the computer 1000 that implements the information processing apparatus 2000 of this embodiment further stores a program module that implements the functions of the information processing apparatus 2000 of this embodiment.
- An information processing apparatus comprising: a notification unit configured to perform a first notification when it is determined that the predetermined condition is not satisfied. 2.
- the predetermined range of each moving image frame of the moving image is the entire moving image frame. The information processing apparatus described in 1. 3.
- the predetermined condition is that, in any one or more moving image frames generated between the first moving image frame and the second moving image frame, a second predetermined range that is included in the predetermined range and is smaller than the predetermined range. 1. that the abnormal region is detected;
- the predetermined range of each moving image frame of the moving image is an area smaller than the entire moving image frame.
- the predetermined condition is that a moving speed of the abnormal region included in the predetermined range is equal to or less than a predetermined value in a moving image from the first moving image frame to the second moving image frame. Or 4.
- the predetermined condition is that the abnormal region is detected after the abnormal region is detected from the predetermined range of the first moving image frame until it is determined that the abnormal region is not detected from the predetermined range of the second moving image frame.
- a predetermined action by a user is detected. Or 4.
- the video is displayed on a display device,
- the notification means causes the display device to display a moving image frame including the abnormal region determined that the predetermined condition is not satisfied.
- the information processing apparatus according to any one of Items 6 to 6. 8).
- the information processing apparatus according to any one of the above. 9.
- the second notification is a highlighted display for a moving image frame that is displayed on a display device and includes the abnormal region that is the target of the second notification.
- a control method executed by a computer A detection step of detecting an abnormal region in the body from a video in which the body is photographed; The abnormal region is detected from within a predetermined range of the first moving image frame of the moving image, and the abnormal region is detected from within the predetermined range of the second moving image frame of the moving image generated after the first moving image frame.
- a determination step for determining whether or not a predetermined condition is satisfied if not, A notification step of performing a first notification when it is determined that the predetermined condition is not satisfied.
- the predetermined range of each moving image frame of the moving image is the entire moving image frame; The control method described in 1.
- the predetermined condition is that, in any one or more moving image frames generated between the first moving image frame and the second moving image frame, a second predetermined range that is included in the predetermined range and is smaller than the predetermined range. 10. that the abnormal region is detected; The control method described in 1. 13. 9. The predetermined range of each moving image frame of the moving image is an area smaller than the entire moving image frame. The control method described in 1. 14 The predetermined condition is that the moving speed of the abnormal region included in the predetermined range is equal to or less than a predetermined value in the moving image from the first moving image frame to the second moving image frame. Or 13. The control method described in 1. 15.
- the predetermined condition is that the abnormal region is detected after the abnormal region is detected from the predetermined range of the first moving image frame until it is determined that the abnormal region is not detected from the predetermined range of the second moving image frame. 10.
- a predetermined action by a user is detected. Or 13.
- the video is displayed on a display device,
- the notifying step displays on the display device a moving image frame including the abnormal area determined that the predetermined condition is not satisfied.
- a second notification is made when the abnormal region is detected from the moving image frame of the moving image generated after the second moving image frame in which the abnormal region targeted for the first notification is detected. 10. having a notification step; To 16.
- the second notification is an emphasis display on a moving image frame that is displayed on a display device and includes the abnormal region that is a target of the second notification.
Abstract
Description
当該制御方法は、1)体内が撮影された動画から前記体内の異常領域を検出する検出ステップと、2)前記動画の第1動画フレームの所定範囲内から前記異常領域が検出され、なおかつ前記第1動画フレームよりも後に生成された前記動画の第2動画フレームの前記所定範囲内からはその異常領域が検出されない場合に、所定条件が満たされているか否かを判定する判定ステップと、3)前記所定条件が満たされていないと判定された場合に第1通知を行う通知ステップと、を有する。
図1は、実施形態1の情報処理装置2000を概念的に例示する図である。図1(a)は、情報処理装置2000の利用環境を概念的に示している。図1(b)は、情報処理装置2000の動作を概念的に示している。なお、図1は、情報処理装置2000の理解を容易にするためにその動作の一例を示しているにすぎず、情報処理装置2000の機能を何ら限定するものではない。
図2は、情報処理装置2000の機能構成を例示するブロック図である。情報処理装置2000は、検出部2020、判定部2040、及び通知部2060を有する。検出部2020は、動画データ12から異常領域30を検出する。判定部2040は、動画データ12の或る動画フレーム14(第1動画フレームと呼ぶ)の所定範囲16から異常領域30が検出され、その動画フレーム14よりも後に生成された動画フレーム14(第2動画フレームと呼ぶ)の所定範囲16からはその異常領域30が検出されない場合に、前述した所定条件が満たされているか否かを判定する。通知部2060は、この所定条件が満たされていないと判定された場合に前述した第1通知を行う。
情報処理装置2000の各機能構成部は、各機能構成部を実現するハードウエア(例:ハードワイヤードされた電子回路など)で実現されてもよいし、ハードウエアとソフトウエアとの組み合わせ(例:電子回路とそれを制御するプログラムの組み合わせなど)で実現されてもよい。以下、情報処理装置2000の各機能構成部がハードウエアとソフトウエアとの組み合わせで実現される場合について、さらに説明する。
図4は、情報処理装置2000の利用環境の具体例を示す図である。例えば情報処理装置2000は、スコープ40及び内視鏡システム50と共に利用される。スコープ40は内視鏡システム50と接続されている。また、スコープ40にはカメラ10が設けられている。このケースでは、動画データ12は、スコープ40に設けられたカメラ10によって生成される複数の動画フレーム14で構成される。内視鏡システム50は、動画データ12を情報処理装置2000へ出力する。例えば動画データ12は、内視鏡システム50に設けられている映像出力用のインタフェース(例えば HDMI(High-Definition Multimedia Interface)(登録商標)インタフェース)から、情報処理装置2000が有する映像入力用のインタフェースへ出力される。そして情報処理装置2000は、内視鏡システム50から取得した動画データ12を処理して、第1通知を行う。
図5は、実施形態1の情報処理装置2000によって実行される処理の流れを例示するフローチャートである。S102からS114は、カメラ10から取得する各動画フレーム14について実行されるループ処理である。S102において、情報処理装置2000は、まだループ処理Aの対象としていない動画フレーム14のうち、最も生成時点が早い動画フレーム14を選択する。ここで選択される動画フレーム14を、動画フレームiと表記する。なお、既に全ての動画フレーム14についてループ処理Aが実行されている場合、例えば情報処理装置2000は、新たな動画フレーム14が生成されるまで待機する。その他にもこの場合、図5の処理は終了してもよい。
検出部2020が動画データ12を取得する方法は任意である。例えば検出部2020は、動画データ12が記憶されている記憶装置にアクセスすることで、動画データ12を取得する。動画データ12が記憶されている記憶装置は、カメラ10の内部に設けられていてもよいし、カメラ10の外部に設けられていてもよい。また例えば、検出部2020は、カメラ10から送信される動画データ12を受信することで、動画データ12を取得してもよい。さらに検出部2020は、カメラ10に接続されている他の装置(例えば前述した内視鏡システム50)から動画データ12を取得してもよい。
検出部2020は、動画データ12を構成する各動画フレーム14から異常領域30を検出する。ここで、体内が撮像されている画像を解析して異常な部位を検出する技術には、既存の技術を利用することができる。例えば、特徴量マッチングやテンプレートマッチングなどの手法が利用できる。例えば特徴量マッチングで腫瘍を検出する場合、腫瘍の外観(色、模様、又は形状など)の特徴を表す値(特徴量)を予め1つ以上定義しておく。検出部2020は、動画フレーム14の画像領域の中から、予め定めておいた腫瘍の特徴量との類似度が高い画像領域を、動画フレーム14から検出する。そして、検出部2020は、検出された画像領域を、異常領域30を表す画像領域として扱う。傷や異物を検出するケースについても同様の方法を採用できる。
判定部2040は、第1動画フレームの所定範囲16から検出された異常領域30が、その後に生成された動画フレーム14の所定範囲16から検出されないことを特定する(S106)。言い換えれば、判定部2040は、第1動画フレームよりも後に生成された動画フレーム14のうち、その所定範囲16から、第1動画フレームの所定範囲16から検出された異常領域30が検出されないものを、第2動画フレームとして特定する。
判定部2040は、或る動画フレーム14の所定範囲16から異常領域30が検出された後(S106:YES)、その動画フレーム14よりも後に生成された動画フレーム14の所定範囲16からはその異常領域30が検出されない場合に(S108:NO)、前述した所定条件が満たされているか否かを判定する(S110)。この所定条件としては、様々なものを採用できる。以下、この所定条件について例示する。なお、特に言及しない限り、所定範囲16は、動画フレーム14の画像領域に含まれる任意の範囲とすることができ、動画フレーム14の画面領域全体とすることもできる。所定範囲16が動画フレーム14の中のどの範囲を表すかは、情報処理装置2000に予め設定されていてもよいし、情報処理装置2000からアクセス可能な記憶装置に記憶させておいてもよい。
動画データ12に含まれる異常領域30がユーザによって認識された場合、ユーザは、カメラ10の移動をやめたり、カメラ10の移動速度を遅くしたりすることにより、異常領域30を詳しく観察しようとする蓋然性が高い。よって、異常領域30がユーザによって認識された場合、動画データ12における異常領域30の移動速度が比較的遅いと考えられる。言い換えれば、異常領域30がユーザによって認識されていない場合、動画データ12における異常領域30の移動速度が比較的速いと考えられる。
動画データ12に含まれる異常領域30がユーザによって認識された場合、ユーザが何らかの行為をする蓋然性が高い。そこで例えば、上記所定条件を、「第1動画フレームの所定範囲16から異常領域30が検出された後、第2動画フレームの所定範囲16から異常領域30が検出されないと判定されるまでの間に、異常領域30に対するユーザによる所定の行為が検出される」という条件とする。第1動画フレームの所定範囲16から異常領域30が検出された後、第2動画フレームの所定範囲16から異常領域30が検出されないと判定されるまでの間に、ユーザによる所定の行為が検出された場合、判定部2040は、上記所定条件が満たされていると判定する。よって、通知部2060による第1通知が行われない。一方、上記の期間にユーザによる所定の行為が検出されない場合、判定部2040は、上記所定条件が満たされていないと判定する。よって、通知部2060による第1通知が行われる。
ここでは、所定範囲16が動画フレーム14の画像領域全体であるとする。さらに、動画フレーム14の中に、所定範囲16よりも小さい第2所定範囲18が設定されているとする。図6は、第2所定範囲18が設定されている動画フレーム14を例示する図である。例えば第2所定範囲18は、その中心位置が動画フレーム14の中心位置にあり、動画フレーム14と同じ形状であり、なおかつその面積が動画フレーム14の80%であるという範囲である。
判定部2040は、上述した所定条件のいずれか1つのみを利用してもよいし、いずれか2つ以上を利用してもよい。後者の場合、例えば判定部2040は、複数の所定条件の少なくとも1つが満たされている場合に「所定条件が満たされている」と判定し、複数の所定条件のいずれもが満たされていない場合に「所定条件が満たされていない」と判定する。このケースでは、複数の所定条件のいずれもが満たされていない場合に第1通知が行われる。
上述した所定条件が満たされていない場合(S110:NO)、通知部2060による第1通知が行われる(S112)。通知部2060による第1通知には、様々なものを採用できる。以下、その具体例について説明する。
通知部2060は、第1通知として、ビープ音や音声メッセージなどの所定の音声をスピーカから出力させる。この場合、情報処理装置2000にはスピーカが接続されている。この所定の音声は、通知部2060に予め設定されていてもよいし、通知部2060からアクセス可能な記憶装置に記憶されていてもよい。
通知部2060は、第1通知として、動画データ12を強調させる表示をディスプレイ装置20に行わせる。例えば動画データ12の強調表示は、動画データ12を囲う枠線を表示させたり、既に表示されている動画データ12の枠線を太くしたりする表示である。その他にも例えば、動画データ12の強調表示は、動画データ12やその枠線を点滅させる表示である。
通知部2060は、第1通知として、ディスプレイ装置20に、第1動画フレーム以降の動画データ12の所定範囲16内において異常領域30が最後に検出された位置を示す表示を行わせてもよい。図9は、異常領域30が最後に検出された所定範囲16内の位置を示す表示を例示する図である。図9において、所定範囲16は動画フレーム14の画像領域全体である。図9(a)は、動画データ12における異常領域30の移動の軌跡を表している。図9(b)は、図9(a)に示す軌跡で異常領域30が移動した場合において行われる第1通知を例示している。
通知部2060は、第1通知として、所定範囲16から出た異常領域30を再び所定範囲16の中に入れるためのガイドとなる表示をディスプレイ装置20に表示させてもよい。図10は、上述のガイドを例示する図である。図10(a)は、動画データ12における異常領域30の移動の軌跡を表している。図10(b)は、図10(a)に示す軌跡で異常領域30が移動した場合において行われる第1通知を例示している。
通知部2060は、第1通知として、第1動画フレームと第2動画フレームの間の動画フレーム14であって異常領域30が所定範囲16の中に含まれるものをディスプレイ装置20に表示させてもよい。図11は、異常領域30が所定範囲16の中に含まれる動画フレーム14が第1通知として表示される様子を例示する図である。図11(a)は、動画データ12における異常領域30の軌跡を表している。異常領域30は、時点 t1 と時点 t2 の間において所定範囲16の中に含まれている。そこで通知部2060は、時点 t1 と時点 t2 の間に生成された動画フレーム14の内の一つを、ディスプレイ装置20に表示させる。図11(b)は、時点 t1 と時点 t2 の間に生成された動画フレーム14がディスプレイ装置20に表示されている様子を示している。図11(b)において、情報処理装置2000は、第1通知として表示される動画フレーム14を、ディスプレイ装置20の第1領域22に表示させている。また、情報処理装置2000は、動画データ12を、ディスプレイ装置20の第2領域24に表示させている。
通知部2060は、第1動画フレームと第2動画フレームとの間に生成された動画フレーム14のうち、異常領域30が含まれる最後の(最も新しい)動画フレーム14を、第1領域22に表示させる動画フレーム14として決定する。この方法によれば、ユーザは、異常領域30がカメラ10によって撮像されていた最後の位置を把握することができる。
通知部2060は、第1動画フレームと第2動画フレームとの間に生成された各動画フレーム14について、異常領域30を表す画像領域が体内の異常を表す確度を特定する。例えば特徴量マッチングやテンプレートマッチングで動画フレーム14から異常領域30を検出する場合、異常領域30を表す画像領域が体内の異常を表す確度は、その画像領域と、予め定義しておいた特徴量やテンプレートとの類似度の高さで表される。通知部2060は、この確度が最も高い動画フレーム14を、第1領域22に表示させる動画フレーム14として決定する。
通知部2060は、第1動画フレームと第2動画フレームとの間に生成された各動画フレーム14の中から、異常領域30の位置が動画フレーム14の中心位置に最も近いものを特定し、特定された動画フレーム14を、第1領域22に表示させる動画フレーム14とする。具体的には、通知部2060は、各動画フレーム14について、動画フレーム14に含まれる異常領域30と動画フレーム14の中心座標との距離を算出する。そして、通知部2060は、その距離が最も小さい動画フレーム14を、第1領域22に表示させる動画フレーム14として決定する。
通知部2060は、第1動画フレームと第2動画フレームとの間に生成された各動画フレーム14のうち、動画フレーム14全体のコントラストが最も大きいものを、第1領域22に表示させる動画フレーム14として決定する。具体的には、通知部2060は、各動画フレーム14について、動画フレーム14全体のコントラストを表す指標値を算出する。そして通知部2060は、算出した指標値を比較することにより、コントラストが最も大きい動画フレーム14を特定し、特定された動画フレーム14を、第1領域22に表示させる動画フレーム14とする。なお、コントラストを表す指標値には、例えば Michelson コントラストなどを利用することができる。
通知部2060は、動画フレーム14全体のコントラストではなく、異常領域30を表す画像領域のコントラストを利用してもよい。つまり、通知部2060は、第1動画フレームから第2動画フレームの間に生成された各動画フレーム14について、異常領域30を表す画像領域のコントラストの指標値を算出する。そして、通知部2060は、算出された指標値を比較することにより、異常領域30を表す画像領域のコントラストが最も大きい動画フレーム14を特定し、この特定した動画フレーム14を第1領域22に表示させる。
通知部2060は、前述した第1領域22に表示させる動画フレーム14を、第1領域22に表示させると共に、又は第1領域22に表示させることに代えて、記憶装置に記憶させてもよい。以下、動画フレーム14を記憶させる記憶装置を画像記憶部120と呼ぶ。図12は、画像記憶部120と接続されている情報処理装置2000を例示する図である。なお、画像記憶部120は、情報処理装置2000の内部に設けられていてもよい。
情報処理装置2000は、ディスプレイ装置20に表示される動画フレーム14について、その動画フレーム14に含まれる異常領域30を表す表示を行う機能を有していてもよい。こうすることで、ユーザが動画フレーム14に含まれる異常領域30を容易に把握することができるようになる。以下、この表示を第1表示と呼ぶ。
図16は、実施形態2の情報処理装置2000を例示するブロック図である。以下で説明する事項を除き、実施形態2の情報処理装置2000は、実施形態1の情報処理装置2000と同様である。
図17は、実施形態2の情報処理装置2000によって実行される処理の流れを例示する図である。図17に示される一連の処理は、検出部2020によって動画フレーム14から異常領域30が検出された後(S106:YES)に実行される。
第2通知としては、様々なものを採用できる。以下、その具体例を説明する。
第2通知部2080は、第2通知として、所定の音声をスピーカから出力させる。通知として所定の音声を利用する方法については、「第1通知の例1」として実施形態1で説明した通りである。なお、第1通知も音声とする場合、第1通知と第2通知には、互いに異なる音声を利用することが好適である。
第2通知部2080は、第2通知として、動画データ12を強調させる表示をディスプレイ装置20に行わせる。動画データ12を強調させる方法については、「第1通知の例2」として実施形態1で説明した通りである。なお、第1通知も動画データ12を強調させる表示とする場合、第1通知と第2通知とで、動画データ12を強調させる態様に差異をつけることが好適である。例えば、第1通知は、動画データ12の枠線を太くする表示とし、第2通知は、動画データ12を点滅させる表示とする。
前提として、「第1通知の例5」として説明したように、第1通知として「第1動画フレームと第2動画フレームの間の動画フレーム14であって異常領域30が所定範囲16の中に含まれるもの」がディスプレイ装置20の第1領域22に表示されるとする。つまり、ユーザが認識していない蓋然性が高い異常領域30が所定範囲16に含まれる動画フレーム14が、第1領域22に表示されるケースである。
通知部2060は、第1通知の対象となった異常領域30を特定できる情報を履歴情報に記憶させておく。例えばこの情報は、異常領域30を表す画像領域の特徴量(例えばその画像領域の色、形状、又は模様を表すパラメタ)である。以下、履歴情報の構成の具体例を示す。
実施形態2の情報処理装置2000を実現する計算機のハードウエア構成は、実施形態1と同様に、例えば図3によって表される。ただし、本実施形態の情報処理装置2000を実現する計算機1000のストレージデバイス1080には、本実施形態の情報処理装置2000の機能を実現するプログラムモジュールがさらに記憶される。
1. 体内が撮影された動画から前記体内の異常領域を検出する検出手段と、
前記動画の第1動画フレームの所定範囲内から前記異常領域が検出され、なおかつ前記第1動画フレームよりも後に生成された前記動画の第2動画フレームの前記所定範囲内からはその異常領域が検出されない場合に、所定条件が満たされているか否かを判定する判定手段と、
前記所定条件が満たされていないと判定された場合に第1通知を行う通知手段と、を有する情報処理装置。
2. 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体である、1.に記載の情報処理装置。
3. 前記所定条件は、前記第1動画フレームと前記第2動画フレームの間に生成されたいずれか1つ以上の動画フレームにおいて、前記所定範囲に含まれなおかつ前記所定範囲よりも小さい第2所定範囲から前記異常領域が検出されることである、2.に記載の情報処理装置。
4. 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体よりも小さい領域である、1.に記載の情報処理装置。
5. 前記所定条件は、前記第1動画フレームから前記第2動画フレームまでの動画において、前記所定範囲に含まれる前記異常領域の移動速度が所定値以下であることである、2.又は4.に記載の情報処理装置。
6. 前記所定条件は、前記第1動画フレームの所定範囲から前記異常領域が検出された後、前記第2動画フレームの所定範囲から前記異常領域が検出されないと判定されるまでの間に、前記異常領域に対するユーザによる所定の行為が検出されることである、2.又は4.に記載の情報処理装置。
7. 前記動画はディスプレイ装置に表示されており、
前記通知手段は、前記所定条件が満たされていないと判定された前記異常領域を含む動画フレームを前記ディスプレイ装置に表示させる、1.乃至6に記載の情報処理装置。
8. 前記第1通知の対象となった前記異常領域が検出された前記第2動画フレームよりも後に生成された動画フレームからその異常領域が検出された場合に、第2通知を行う第2通知手段を有する、1.乃至7.いずれか一つに記載の情報処理装置。
9. 前記第2通知は、ディスプレイ装置に表示されておりなおかつ前記第2通知の対象となる前記異常領域が含まれる動画フレームに対する強調表示である、8.に記載の情報処理装置。
体内が撮影された動画から前記体内の異常領域を検出する検出ステップと、
前記動画の第1動画フレームの所定範囲内から前記異常領域が検出され、なおかつ前記第1動画フレームよりも後に生成された前記動画の第2動画フレームの前記所定範囲内からはその異常領域が検出されない場合に、所定条件が満たされているか否かを判定する判定ステップと、
前記所定条件が満たされていないと判定された場合に第1通知を行う通知ステップと、を有する制御方法。
11. 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体である、10.に記載の制御方法。
12. 前記所定条件は、前記第1動画フレームと前記第2動画フレームの間に生成されたいずれか1つ以上の動画フレームにおいて、前記所定範囲に含まれなおかつ前記所定範囲よりも小さい第2所定範囲から前記異常領域が検出されることである、11.に記載の制御方法。
13. 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体よりも小さい領域である、10.に記載の制御方法。
14. 前記所定条件は、前記第1動画フレームから前記第2動画フレームまでの前記動画において、前記所定範囲に含まれる前記異常領域の移動速度が所定値以下であることである、11.又は13.に記載の制御方法。
15. 前記所定条件は、前記第1動画フレームの所定範囲から前記異常領域が検出された後、前記第2動画フレームの所定範囲から前記異常領域が検出されないと判定されるまでの間に、前記異常領域に対するユーザによる所定の行為が検出されることである、11.又は13.に記載の制御方法。
16. 前記動画はディスプレイ装置に表示されており、
前記通知ステップは、前記所定条件が満たされていないと判定された前記異常領域を含む動画フレームを前記ディスプレイ装置に表示させる、10.乃至15に記載の制御方法。
17. 前記第1通知の対象となった前記異常領域が検出された前記第2動画フレームよりも後に生成された前記動画の動画フレームからその異常領域が検出された場合に、第2通知を行う第2通知ステップを有する、10.乃至16.いずれか一つに記載の制御方法。
18. 前記第2通知は、ディスプレイ装置に表示されておりなおかつ前記第2通知の対象となる前記異常領域が含まれる動画フレームに対する強調表示である、17.に記載の制御方法。
Claims (19)
- 体内が撮影された動画から前記体内の異常領域を検出する検出手段と、
前記動画の第1動画フレームの所定範囲内から前記異常領域が検出され、なおかつ前記第1動画フレームよりも後に生成された前記動画の第2動画フレームの前記所定範囲内からはその異常領域が検出されない場合に、所定条件が満たされているか否かを判定する判定手段と、
前記所定条件が満たされていないと判定された場合に第1通知を行う通知手段と、を有する情報処理装置。 - 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体である、請求項1に記載の情報処理装置。
- 前記所定条件は、前記第1動画フレームと前記第2動画フレームの間に生成されたいずれか1つ以上の動画フレームにおいて、前記所定範囲に含まれなおかつ前記所定範囲よりも小さい第2所定範囲から前記異常領域が検出されることである、請求項2に記載の情報処理装置。
- 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体よりも小さい領域である、請求項1に記載の情報処理装置。
- 前記所定条件は、前記第1動画フレームから前記第2動画フレームまでの動画において、前記所定範囲に含まれる前記異常領域の移動速度が所定値以下であることである、請求項2又は4に記載の情報処理装置。
- 前記所定条件は、前記第1動画フレームの所定範囲から前記異常領域が検出された後、前記第2動画フレームの所定範囲から前記異常領域が検出されないと判定されるまでの間に、前記異常領域に対するユーザによる所定の行為が検出されることである、請求項2又は4に記載の情報処理装置。
- 前記動画はディスプレイ装置に表示されており、
前記通知手段は、前記所定条件が満たされていないと判定された前記異常領域を含む動画フレームを前記ディスプレイ装置に表示させる、請求項1乃至6に記載の情報処理装置。 - 前記第1通知の対象となった前記異常領域が検出された前記第2動画フレームよりも後に生成された動画フレームからその異常領域が検出された場合に、第2通知を行う第2通知手段を有する、請求項1乃至7いずれか一項に記載の情報処理装置。
- 前記第2通知は、ディスプレイ装置に表示されておりなおかつ前記第2通知の対象となる前記異常領域が含まれる動画フレームに対する強調表示である、請求項8に記載の情報処理装置。
- コンピュータによって実行される制御方法であって、
体内が撮影された動画から前記体内の異常領域を検出する検出ステップと、
前記動画の第1動画フレームの所定範囲内から前記異常領域が検出され、なおかつ前記第1動画フレームよりも後に生成された前記動画の第2動画フレームの前記所定範囲内からはその異常領域が検出されない場合に、所定条件が満たされているか否かを判定する判定ステップと、
前記所定条件が満たされていないと判定された場合に第1通知を行う通知ステップと、を有する制御方法。 - 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体である、請求項10に記載の制御方法。
- 前記所定条件は、前記第1動画フレームと前記第2動画フレームの間に生成されたいずれか1つ以上の動画フレームにおいて、前記所定範囲に含まれなおかつ前記所定範囲よりも小さい第2所定範囲から前記異常領域が検出されることである、請求項11に記載の制御方法。
- 前記動画の各動画フレームの前記所定範囲はその動画フレーム全体よりも小さい領域である、請求項10に記載の制御方法。
- 前記所定条件は、前記第1動画フレームから前記第2動画フレームまでの前記動画において、前記所定範囲に含まれる前記異常領域の移動速度が所定値以下であることである、請求項11又は13に記載の制御方法。
- 前記所定条件は、前記第1動画フレームの所定範囲から前記異常領域が検出された後、前記第2動画フレームの所定範囲から前記異常領域が検出されないと判定されるまでの間に、前記異常領域に対するユーザによる所定の行為が検出されることである、請求項11又は13に記載の制御方法。
- 前記動画はディスプレイ装置に表示されており、
前記通知ステップは、前記所定条件が満たされていないと判定された前記異常領域を含む動画フレームを前記ディスプレイ装置に表示させる、請求項10乃至15に記載の制御方法。 - 前記第1通知の対象となった前記異常領域が検出された前記第2動画フレームよりも後に生成された前記動画の動画フレームからその異常領域が検出された場合に、第2通知を行う第2通知ステップを有する、請求項10乃至16いずれか一項に記載の制御方法。
- 前記第2通知は、ディスプレイ装置に表示されておりなおかつ前記第2通知の対象となる前記異常領域が含まれる動画フレームに対する強調表示である、請求項17に記載の制御方法。
- 請求項10乃至18いずれか一項に記載の制御方法の各ステップをコンピュータに実行させるプログラム。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2019520218A JP6716853B2 (ja) | 2017-05-25 | 2018-05-18 | 情報処理装置、制御方法、及びプログラム |
US16/615,160 US11553829B2 (en) | 2017-05-25 | 2018-05-18 | Information processing apparatus, control method and program |
EP18806136.0A EP3632295A4 (en) | 2017-05-25 | 2018-05-18 | INFORMATION PROCESSING DEVICE, ORDERING PROCESS, AND PROGRAM |
CN201880034276.8A CN110662476B (zh) | 2017-05-25 | 2018-05-18 | 信息处理装置、控制方法和程序 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017103349 | 2017-05-25 | ||
JP2017-103349 | 2017-05-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2018216618A1 true WO2018216618A1 (ja) | 2018-11-29 |
Family
ID=64396429
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2018/019310 WO2018216618A1 (ja) | 2017-05-25 | 2018-05-18 | 情報処理装置、制御方法、及びプログラム |
Country Status (5)
Country | Link |
---|---|
US (1) | US11553829B2 (ja) |
EP (1) | EP3632295A4 (ja) |
JP (1) | JP6716853B2 (ja) |
CN (1) | CN110662476B (ja) |
WO (1) | WO2018216618A1 (ja) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020071086A1 (ja) * | 2018-10-04 | 2020-04-09 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
WO2020166528A1 (ja) * | 2019-02-13 | 2020-08-20 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びコンピュータ読み取り可能な記録媒体 |
WO2021149169A1 (ja) | 2020-01-21 | 2021-07-29 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びコンピュータ読み取り可能な記録媒体 |
WO2021176852A1 (ja) * | 2020-03-03 | 2021-09-10 | 富士フイルム株式会社 | 画像選択支援装置、画像選択支援方法、及び画像選択支援プログラム |
WO2021220822A1 (ja) * | 2020-04-27 | 2021-11-04 | 公益財団法人がん研究会 | 画像診断装置、画像診断方法、画像診断プログラムおよび学習済みモデル |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019207740A1 (ja) * | 2018-04-26 | 2019-10-31 | オリンパス株式会社 | 移動支援システム及び移動支援方法 |
CN110536177B (zh) * | 2019-09-23 | 2020-10-09 | 北京达佳互联信息技术有限公司 | 视频生成方法、装置、电子设备及存储介质 |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007159934A (ja) | 2005-12-15 | 2007-06-28 | Hitachi Medical Corp | 比較読影支援装置 |
JP2010512173A (ja) * | 2006-08-21 | 2010-04-22 | エスティーアイ・メディカル・システムズ・エルエルシー | 内視鏡からの映像を用いるコンピュータ支援解析 |
JP2010172673A (ja) * | 2009-02-02 | 2010-08-12 | Fujifilm Corp | 内視鏡システム、内視鏡用プロセッサ装置、並びに内視鏡検査支援方法 |
JP2011036371A (ja) * | 2009-08-10 | 2011-02-24 | Tohoku Otas Kk | 医療画像記録装置 |
WO2011132468A1 (ja) | 2010-04-21 | 2011-10-27 | コニカミノルタエムジー株式会社 | 医用画像表示装置及びプログラム |
JP2016048426A (ja) | 2014-08-27 | 2016-04-07 | テクマトリックス株式会社 | 読影レポートシステム |
JP2016202722A (ja) | 2015-04-27 | 2016-12-08 | コニカミノルタ株式会社 | 医用画像表示装置及びプログラム |
JP2017103349A (ja) | 2015-12-02 | 2017-06-08 | 日立金属株式会社 | 通信装置およびその制御方法 |
Family Cites Families (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1166342C (zh) | 2000-01-20 | 2004-09-15 | 深圳市辉大高科技发展有限公司 | 导纳式血液循环自动检测方法 |
JP4434698B2 (ja) | 2003-11-13 | 2010-03-17 | 株式会社東芝 | X線ct装置 |
US7480528B2 (en) | 2004-07-23 | 2009-01-20 | Cardiac Pacemakers, Inc. | Method and apparatus for monitoring heart failure patients with cardiopulmonary comorbidities |
CN101669807B (zh) * | 2004-08-23 | 2013-08-21 | 奥林巴斯株式会社 | 图像显示装置、图像显示方法和图像显示程序 |
JP2006198106A (ja) * | 2005-01-19 | 2006-08-03 | Olympus Corp | 電子内視鏡装置 |
US8005279B2 (en) | 2005-03-22 | 2011-08-23 | Osaka University | Capsule endoscope image display controller |
CN1333552C (zh) * | 2005-03-23 | 2007-08-22 | 北京首信科技有限公司 | 基于机器学习的用户行为异常的检测方法 |
JP5165872B2 (ja) | 2006-10-11 | 2013-03-21 | 住友電気工業株式会社 | フェルール及び該フェルールを用いた光導波路用コネクタの製造方法及び光導波路用コネクタ |
JP5281826B2 (ja) | 2008-06-05 | 2013-09-04 | オリンパス株式会社 | 画像処理装置、画像処理プログラムおよび画像処理方法 |
WO2010123043A1 (ja) * | 2009-04-21 | 2010-10-28 | 日本電気株式会社 | 癌の評価方法 |
JP5056798B2 (ja) | 2009-06-08 | 2012-10-24 | 日本電気株式会社 | 判定装置、指紋入力装置、判定方法および判定プログラム |
JP5385713B2 (ja) | 2009-07-16 | 2014-01-08 | 学校法人東京女子医科大学 | 臨床検査データ解析表示装置 |
JP5597021B2 (ja) | 2010-04-15 | 2014-10-01 | オリンパス株式会社 | 画像処理装置及びプログラム |
EP2618556B1 (en) | 2010-12-14 | 2015-05-27 | Olympus Medical Systems Corp. | Imaging apparatus |
JP2012170774A (ja) * | 2011-02-24 | 2012-09-10 | Fujifilm Corp | 内視鏡システム |
JP6231885B2 (ja) | 2011-03-16 | 2017-11-15 | コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. | 頸部を検査する医療機器 |
JP6045417B2 (ja) | 2012-12-20 | 2016-12-14 | オリンパス株式会社 | 画像処理装置、電子機器、内視鏡装置、プログラム及び画像処理装置の作動方法 |
JP2014236726A (ja) | 2013-05-07 | 2014-12-18 | 独立行政法人国立がん研究センター | 胃がんの再発を予測する方法 |
CN105682534B (zh) * | 2013-12-25 | 2017-10-27 | 奥林巴斯株式会社 | 内窥镜系统以及像素校正方法 |
JP6425125B2 (ja) | 2014-01-31 | 2018-11-21 | パナソニックIpマネジメント株式会社 | 歯茎マッサージ装置 |
JP6334288B2 (ja) | 2014-06-23 | 2018-05-30 | オリンパス株式会社 | 撮像装置およびカプセル内視鏡 |
CN105249549A (zh) | 2014-11-14 | 2016-01-20 | 北京至感传感器技术研究院有限公司 | 用于检测乳房生理变化的装置 |
KR102381140B1 (ko) | 2015-02-25 | 2022-03-31 | 엘지전자 주식회사 | 디지털 디바이스 및 그의 운전자 모니터링 방법 |
WO2016199273A1 (ja) | 2015-06-11 | 2016-12-15 | オリンパス株式会社 | 内視鏡装置及び内視鏡装置の作動方法 |
EP3360461A4 (en) * | 2015-11-10 | 2019-05-08 | Olympus Corporation | ENDOSCOPE DEVICE |
WO2017203560A1 (ja) * | 2016-05-23 | 2017-11-30 | オリンパス株式会社 | 内視鏡画像処理装置 |
CN110461204B (zh) * | 2017-03-28 | 2021-05-18 | 富士胶片株式会社 | 测量辅助装置、内窥镜系统及处理器 |
EP3636134A4 (en) * | 2017-05-25 | 2021-03-10 | Nec Corporation | INFORMATION PROCESSING DEVICE, ORDERING PROCESS AND PROGRAM |
-
2018
- 2018-05-18 CN CN201880034276.8A patent/CN110662476B/zh active Active
- 2018-05-18 JP JP2019520218A patent/JP6716853B2/ja active Active
- 2018-05-18 US US16/615,160 patent/US11553829B2/en active Active
- 2018-05-18 WO PCT/JP2018/019310 patent/WO2018216618A1/ja active Application Filing
- 2018-05-18 EP EP18806136.0A patent/EP3632295A4/en active Pending
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2007159934A (ja) | 2005-12-15 | 2007-06-28 | Hitachi Medical Corp | 比較読影支援装置 |
JP2010512173A (ja) * | 2006-08-21 | 2010-04-22 | エスティーアイ・メディカル・システムズ・エルエルシー | 内視鏡からの映像を用いるコンピュータ支援解析 |
JP2010172673A (ja) * | 2009-02-02 | 2010-08-12 | Fujifilm Corp | 内視鏡システム、内視鏡用プロセッサ装置、並びに内視鏡検査支援方法 |
JP2011036371A (ja) * | 2009-08-10 | 2011-02-24 | Tohoku Otas Kk | 医療画像記録装置 |
WO2011132468A1 (ja) | 2010-04-21 | 2011-10-27 | コニカミノルタエムジー株式会社 | 医用画像表示装置及びプログラム |
JP2016048426A (ja) | 2014-08-27 | 2016-04-07 | テクマトリックス株式会社 | 読影レポートシステム |
JP2016202722A (ja) | 2015-04-27 | 2016-12-08 | コニカミノルタ株式会社 | 医用画像表示装置及びプログラム |
JP2017103349A (ja) | 2015-12-02 | 2017-06-08 | 日立金属株式会社 | 通信装置およびその制御方法 |
Non-Patent Citations (1)
Title |
---|
See also references of EP3632295A4 |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020071086A1 (ja) * | 2018-10-04 | 2020-04-09 | 日本電気株式会社 | 情報処理装置、制御方法、及びプログラム |
WO2020166528A1 (ja) * | 2019-02-13 | 2020-08-20 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びコンピュータ読み取り可能な記録媒体 |
JPWO2020166528A1 (ja) * | 2019-02-13 | 2021-12-02 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びプログラム |
JP7215504B2 (ja) | 2019-02-13 | 2023-01-31 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びプログラム |
WO2021149169A1 (ja) | 2020-01-21 | 2021-07-29 | 日本電気株式会社 | 施術支援装置、施術支援方法、及びコンピュータ読み取り可能な記録媒体 |
EP4094672A4 (en) * | 2020-01-21 | 2023-01-11 | NEC Corporation | OPERATING ASSIST DEVICE, OPERATING ASSISTING METHOD AND COMPUTER READABLE RECORDING MEDIA |
WO2021176852A1 (ja) * | 2020-03-03 | 2021-09-10 | 富士フイルム株式会社 | 画像選択支援装置、画像選択支援方法、及び画像選択支援プログラム |
JP7427766B2 (ja) | 2020-03-03 | 2024-02-05 | 富士フイルム株式会社 | 画像選択支援装置、画像選択支援方法、及び画像選択支援プログラム |
WO2021220822A1 (ja) * | 2020-04-27 | 2021-11-04 | 公益財団法人がん研究会 | 画像診断装置、画像診断方法、画像診断プログラムおよび学習済みモデル |
Also Published As
Publication number | Publication date |
---|---|
JPWO2018216618A1 (ja) | 2020-04-02 |
CN110662476A (zh) | 2020-01-07 |
CN110662476B (zh) | 2022-02-11 |
EP3632295A1 (en) | 2020-04-08 |
US20200170485A1 (en) | 2020-06-04 |
JP6716853B2 (ja) | 2020-07-01 |
EP3632295A4 (en) | 2021-03-10 |
US11553829B2 (en) | 2023-01-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018216618A1 (ja) | 情報処理装置、制御方法、及びプログラム | |
JP6799301B2 (ja) | 情報処理装置、制御方法、及びプログラム | |
EP3968272A1 (en) | Medical endoscope image identification method and system, and endoscope image system | |
US20160133014A1 (en) | Marking And Tracking An Area Of Interest During Endoscopy | |
JP6348078B2 (ja) | 分岐構造判定装置、分岐構造判定装置の作動方法および分岐構造判定プログラム | |
CN111295127B (zh) | 检查支持装置、内窥镜装置及记录介质 | |
JP2022106849A5 (ja) | 画像処理装置、撮像装置、制御方法、プログラム、及び記憶媒体 | |
US20220012457A1 (en) | Image processing method, microscope, image processing system, and medium based on artificial intelligence | |
US11642005B2 (en) | Endoscope system, endoscope image processing method, and storage medium | |
WO2020067105A1 (ja) | 医用画像処理装置、医用画像処理方法、及びプログラム、診断支援装置並びに内視鏡システム | |
CN112584742B (zh) | 医疗图像处理系统 | |
US11957300B2 (en) | Information processing apparatus, control method, and program | |
JP7253152B2 (ja) | 情報処理装置、情報処理方法、及びプログラム | |
JP6647703B2 (ja) | 手術トレーニングプログラム | |
WO2018061299A1 (ja) | 情報処理装置および情報処理方法 | |
KR20200021714A (ko) | 의료 영상 분석 방법 및 장치 | |
US11653831B2 (en) | Visual performance examination device, visual performance examination method, and computer program | |
KR20170020678A (ko) | 시간열 이미지 분석을 위한 그래픽 유저 인터페이스 제공 방법 | |
CN115393295A (zh) | 一种出血点评估方法、装置、设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18806136 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2019520218 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2018806136 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2018806136 Country of ref document: EP Effective date: 20200102 |