WO2017073337A1

WO2017073337A1 - Endoscope device

Info

Publication number: WO2017073337A1
Application number: PCT/JP2016/080309
Authority: WO
Inventors: 今泉　克一; 橋本　進
Original assignee: オリンパス株式会社
Priority date: 2015-10-27
Filing date: 2016-10-13
Publication date: 2017-05-04
Also published as: JPWO2017073337A1

Abstract

This endoscope device has: a detection unit 34, into which an observation image G1 of a subject is inputted, and which detects a feature region L in the observation image G1 on the basis of a predetermined feature amount in the observation image G1; a notification unit 35a that, if a feature region L is detected by the detection unit 34, notifies an operator, via notification processing, that the feature region L was detected, in a region that is different than the feature region L; and an enhancement processing unit 35b that, at a time other than the start of the notification by the notification unit 35a, displays the observation image G1 with the feature region L enhanced via enhancement processing.

Description

Endoscope device

The present invention relates to an endoscope apparatus.

Conventionally, in an endoscopic apparatus, an operator determines the presence or absence of a lesion by looking at an observation image. In order to suppress an oversight of a lesion when an operator views an observation image, for example, as shown in Japanese Patent Application Laid-Open No. 2011-255006, an alert image is added to a region of interest detected by image processing. An endoscope apparatus that displays an observation image has been proposed.

However, in a conventional endoscopic device, an alert image may be displayed before the surgeon finds a lesion, reducing the operator's attention to the area not indicated by the alert image, There is a concern that the surgeon's willingness to find a lesion will be cut off, and the ability to detect the lesion will be hindered.

Therefore, an object of the present invention is to provide an endoscope apparatus that presents a region of interest to an operator without suppressing a reduction in attention to an observed image and preventing an improvement in lesion finding ability.

An endoscope apparatus according to an aspect of the present invention includes a detection unit that receives an observation image of a subject and detects a feature region in the observation image based on a predetermined feature amount of the observation image; and the detection unit When the feature region is detected by the notification unit, a notification unit that notifies the operator that the feature region has been detected by a notification process in a region different from the feature region, and a notification by the notification unit And an enhancement processing unit that displays the observation image with the feature region enhanced by enhancement processing at a timing different from the start.

1 is a block diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention. It is a block diagram which shows the structure of the detection assistance part of the endoscope system concerning embodiment of this invention. It is explanatory drawing explaining the example of the screen structure of the image for a display concerning the embodiment of this invention. It is a flowchart explaining the flow of the detection result output process of an endoscope system concerning embodiment of this invention. It is explanatory drawing explaining the example of the screen transition in the detection result output process of the endoscope system concerning embodiment of this invention. It is explanatory drawing explaining the example of the screen structure of the image for a display concerning the modification 1 of embodiment of this invention of an endoscope system. It is explanatory drawing explaining the example of the screen transition in the detection result output process of an endoscope system concerning the modification 1 of embodiment of this invention. It is a block diagram which shows the structure of the detection assistance part of the endoscope system concerning the modification 2 of embodiment of this invention. It is explanatory drawing explaining the example of the screen structure of the image for a display concerning the embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Constitution)
FIG. 1 is a block diagram showing a schematic configuration of an endoscope system 1 according to an embodiment of the present invention.

The endoscope system 1 includes a light source driving unit 11, an endoscope 21, an operation switch W1 that is an operation unit, a video processor 31, and a display unit 41. The light source driving unit 11 is connected to the endoscope 21 and the video processor 31. The endoscope 21 is connected to the video processor 31. The video processor 31 is connected to the operation switch W1 and the display unit 41.

The light source driving unit 11 is a circuit that drives the LED 23 provided at the distal end of the insertion unit 22 of the endoscope 21. The light source driving unit 11 is connected to the control unit 32 of the video processor 31 and the LED 23 of the endoscope 21. The light source drive unit 11 is configured to receive a control signal from the control unit 32, output a drive signal to the LED 23, and drive the LED 23 to emit light.

The endoscope 21 is configured so that the insertion unit 22 can be inserted into the subject and the inside of the subject can be imaged. The endoscope 21 includes an imaging unit that includes an LED 23 and an imaging element 24.

The LED 23 is provided in the insertion unit 22 of the endoscope 21 and is configured to irradiate the subject with illumination light under the control of the light source driving unit 11.

The imaging device 24 is provided in the insertion portion 22 of the endoscope 21 and is arranged so that the reflected light of the subject irradiated with the illumination light can be taken in through an observation window (not shown).

The imaging device 24 photoelectrically converts the reflected light of the subject taken in from the observation window, converts the analog imaging signal into a digital imaging signal by an AD converter (not shown), and outputs it to the video processor 31.

The operation switch W1 is configured by a switch such as a foot switch that can be switched between an ON state and an OFF state by an operator's operation instruction. The operation switch W1 is connected to the detection result output unit 35 (FIG. 2) of the video processor 31.

The initial state of the operation switch W1 is set to OFF. The operation switch W1 is switched from the initial OFF state to the ON state according to the operator's operation instruction. When there is an operator operation instruction in the ON state, the operation switch W1 is switched to the OFF state.

The video processor 31 is an endoscopic image processing apparatus having an image processing circuit. The video processor 31 includes a control unit 32 and a detection support unit 33.

The control unit 32 can transmit a control signal to the light source driving unit 11 to drive the LED 23.

The control unit 32 performs image adjustment, such as gain adjustment, white balance adjustment, gamma correction, contour enhancement correction, and enlargement / reduction adjustment, on the imaging signal input from the endoscope 21, and observes a subject to be described later. The image G1 can be sequentially output to the detection support unit 33.

FIG. 2 is a block diagram showing a configuration of the detection support unit 33 of the endoscope system 1 according to the embodiment of the present invention. As shown in FIG. 2, the detection support unit 33 includes a detection unit 34 and a detection result output unit 35.

The detection unit 34 is a circuit that receives the observation image G1 of the subject and detects a lesion candidate region L that is a feature region in the observation image G1 based on a predetermined feature amount of the observation image G1. The detection unit 34 includes a feature amount calculation unit 34a and a lesion candidate detection unit 34b.

The feature amount calculation unit 34a is a circuit that calculates a predetermined feature amount for the observation image G1 of the subject. The feature amount calculation unit 34a is connected to the control unit 32 and the lesion candidate detection unit 34b. The feature amount calculation unit 34a can calculate a predetermined feature amount from the observation image G1 of the subject that is sequentially input from the control unit 32, and can output it to the lesion candidate detection unit 34b.

The predetermined feature amount is calculated for each predetermined small region on the observation image G1 by calculating a change amount between each pixel in the predetermined small region and a pixel adjacent to the pixel, that is, an inclination value. Note that the predetermined feature amount is not limited to a method calculated based on an inclination value with an adjacent pixel, and may be a value obtained by quantifying the observation image G1 by another method.

The lesion candidate detection unit 34b is a circuit that detects a lesion candidate region L of the observation image G1 from information of a predetermined feature amount. The lesion candidate detection unit 34b includes a ROM 34c so that a plurality of polyp model information can be stored in advance. The lesion candidate detection unit 34 b is connected to the detection result output unit 35.

The polyp model information is composed of feature amounts of features common to many polyp images.

The lesion candidate detection unit 34b detects a lesion candidate region L based on the predetermined feature amount input from the feature amount calculation unit 34a and a plurality of polyp model information, and the lesion result information is sent to the detection result output unit 35. Output.

More specifically, the lesion candidate detection unit 34b compares the predetermined feature amount for each predetermined small region input from the feature amount calculation unit 34a with the feature amount of the polyp model information stored in the ROM 34c, and When the feature amounts match, the lesion candidate region L is detected. When the lesion candidate region L is detected, the lesion candidate detection unit 34b outputs lesion candidate information including the position information and size information of the detected lesion candidate region L to the detection result output unit 35.

The detection result output unit 35 is a circuit that performs detection result output processing. The detection result output unit 35 includes a notification unit 35a and an enhancement processing unit 35b. The detection result output unit 35 is connected to the display unit 41. The detection result output unit 35 detects the ON / OFF state of the operation switch W1, and performs enhancement processing based on the observation image G1 input from the control unit 32 and the lesion candidate information input from the lesion candidate detection unit 34b. And a notification process can be performed, and a display image G can be generated and output to the display unit 41.

More specifically, when the lesion candidate area L is detected by the lesion candidate detection unit 34b and the lesion candidate information and the observation image G1 are input, the detection result output unit 35 detects the state of the operation switch W1. When the operation switch W1 is in the OFF state, the notification process by the notification unit 35a is performed on the observation image G1, while when the operation switch W1 is in the ON state, the lesion candidate is displayed for the observation image G1. Based on the information, the enhancement processing unit 35b performs enhancement processing.

That is, the notification unit 35a does not perform notification while the observation image G1 highlighting the lesion candidate region L is displayed. More specifically, the notification unit 35a performs notification when the observation image G1 in which the lesion candidate region L is emphasized is not displayed and the lesion candidate region L is detected by the detection unit 34.

FIG. 3 is an explanatory diagram illustrating an example of the screen configuration of the display image G of the endoscope system 1 according to the embodiment of the present invention. In the display image G output from the detection result output unit 35, an observation image G1 is arranged as shown in FIG. FIG. 3 shows an inner wall of the large intestine having a lesion candidate region L as an example of the observation image G1. FIG. 3 shows an example of the marker image G2 and the notification image G3.

When the candidate lesion region L is detected by the detection unit 34, the notification unit 35a can notify the surgeon that the candidate lesion region L has been detected by a notification process in a region different from the candidate lesion region L. Configured as follows. The two-dot chain line in FIG. 3 shows a flag-pattern notification image G3 as an example, but the notification image G3 may be any image such as a triangle, a circle, or a star.

The enhancement processing unit 35b is configured to display an observation image G1 in which the lesion candidate region L is enhanced by enhancement processing at a timing different from the start of notification by the notification unit 35a. That is, the enhancement processing unit 35b performs enhancement processing of the lesion candidate region L when it is detected that the operation switch W1 is in the ON state after the notification by the notification unit 35a is started.

The enhancement process is a process for displaying the position of the lesion candidate region L. More specifically, in the enhancement process, the marker image G2 surrounding the lesion candidate region L is applied to the observation image G1 input from the control unit 32 based on the position information and the size information included in the lesion candidate information. It is a process to add. In FIG. 3, as an example, the marker image G <b> 2 is shown as a square, but may be any image such as a triangle, a circle, or a star. In FIG. 3, as an example, the marker image G2 is a frame image surrounding the lesion candidate region L. However, as long as the position of the lesion candidate region L can be indicated, the marker image G2 does not surround the lesion candidate region L. It does not matter. For example, the position of the lesion candidate area L may be indicated by making the brightness and color tone of the lesion candidate area L different from the surrounding area.

The display unit 41 is configured to display the display image G input from the detection result output unit 35 on the screen.

(Function)
Next, the detection result output process of the detection result output unit 35 will be described.

FIG. 4 is a flowchart for explaining the flow of detection result output processing of the endoscope system 1 according to the embodiment of the present invention.

When the subject is imaged by the endoscope 21, an image adjustment process is performed by the control unit 32, and then the observation image G 1 is input to the detection support unit 33. When the observation image G1 is input to the detection support unit 33, the feature amount calculation unit 34a calculates a predetermined feature amount of the observation image G1 and outputs it to the lesion candidate detection unit 34b. The lesion candidate detection unit 34b detects the lesion candidate region L by comparing the input predetermined feature amount with the feature amount of the polyp model information. The detection result of the lesion candidate region L is output to the detection result output unit 35 as lesion candidate information.

The detection result output unit 35 determines whether a lesion candidate region L is detected (S1). In S1, when the detection result output unit 35 determines that the lesion candidate region L is detected based on the determination result of the lesion candidate detection unit 34b (S1: Yes), the process proceeds to S2. On the other hand, when the detection result output unit 35 determines that the lesion candidate region L is not detected (S1: No), the process proceeds to S5.

The detection result output unit 35 determines whether the operation switch W1 is in the OFF state (S2). In S2, when the detection result output unit 35 detects the ON / OFF state of the operation switch W1 and determines that the operation switch W1 is in the OFF state, the process proceeds to S3. On the other hand, when the detection result output unit 35 determines that the operation switch W1 is in the ON state, the process proceeds to S4.

In S3, the detection result output unit 35 performs a notification process. In S3, the detection result output unit 35 performs a notification process and adds the notification image G3 to a region other than the observation image G1. After the process of S3, the process proceeds to S5.

In S4, the detection result output unit 35 performs enhancement processing. In S4, the detection result output unit 35 adds the marker image G2 to the observation image G1. After the process of S4, the process proceeds to S5.

The display image G is output to the display unit 41 (S5). In S <b> 5, the detection result output unit 35 generates a display image G and outputs it to the display unit 41.

The processing from S1 to S5 constitutes detection result output processing.

FIG. 5 is an explanatory diagram illustrating an example of screen transition in the detection result output process of the endoscope system 1 according to the embodiment of the present invention.

By repeating the detection result output processing from S1 to S5, the display image G transitions as shown in FIG.

In the initial state, the operation switch W1 is in the OFF state. When the lesion candidate area L is detected, the notification process is started, and the notification image G3 is displayed in the display image G. Subsequently, when the operation switch W1 is switched from the OFF state to the ON state according to the operator's operation instruction, the notification process is terminated, and in the display image G, the notification image G3 is not displayed and the enhancement process is started. A marker image G2 is displayed. Subsequently, when the operation switch W1 is switched from the ON state to the OFF state according to the operator's operation instruction, the emphasis process is terminated, and the marker image G2 is not displayed in the display image G, and the notification process is started. A notification image G3 is displayed. Subsequently, when the lesion candidate region L is no longer detected, the notification process ends, and the notification image G3 is not displayed in the display image G.

According to the above-described embodiment, since there is a time for the surgeon to visually find a lesion portion after the candidate lesion region L is detected until the marker image G2 is displayed, the observation image is displayed to the surgeon. It is possible to present a region of interest without suppressing a reduction in attention to G1 and without hindering improvement in the ability to detect a lesion.

(Modification 1 of embodiment)
In the above-described embodiment, the display image G includes the observation image G1 displayed as a moving image. However, the display image G may include the observation image G1 and the still image G4. Absent.

FIG. 6 is an explanatory diagram illustrating an example of the screen configuration of the display image G of the endoscope system 1 according to the first modification of the embodiment of the present invention. FIG. 7 is an explanatory diagram illustrating an example of screen transition in the detection result output process of the endoscope system 1 according to the first modification of the embodiment of the present invention.

In the first modification of the embodiment, the detection result output processing unit 35 includes a still image processing unit 35c and a memory 35d (one-dot chain line in FIG. 2).

The still image processing unit 35c is configured to display the still image G4 when the operation switch W1 is turned on.

The memory 35d is configured to temporarily store the still image G4.

When the lesion candidate area L is detected and the notification process is started, the detection result output unit 35 temporarily stores the still image G4 in the memory 35d. When the operation switch W1 is turned on, the detection result output unit 35 adds the marker image G2a to the still image G4 temporarily stored in the memory 35d, and causes the display unit 41 to display the still image G4. .

When the operation switch W1 is turned off, the detection result output unit 35 hides the still image G4 together with the marker image G2.

According to the first modification of the above-described embodiment, it is possible to more reliably indicate the position of the lesion candidate region L to the surgeon, and suppresses a reduction in attention to the observation image G1 to the surgeon. An area of interest can be presented without hindering the improvement of discovery ability.

(Modification 2 of embodiment)
In the above-described embodiment and the first modification of the embodiment, the emphasis process is performed until the operation switch W1 is switched to the OFF state. However, the emphasis process may be configured to end after a predetermined time has elapsed.

FIG. 8 is a block diagram illustrating a configuration of the detection support unit 33 of the endoscope system 1 according to the second modification of the embodiment of the present invention.

In the second modification of the embodiment, the continuation detection determination unit 36 is provided. The continuation detection determination unit 36 is a circuit that determines whether or not the lesion candidate region L is continuously detected. The continuation detection determination unit 36 is configured to include a RAM 36a so that the lesion candidate information of one frame before can be stored. The continuation detection determination unit 36 is connected to the lesion candidate detection unit 34 b and the detection result output unit 35.

For example, the continuation detection determination unit 36 is input from the lesion candidate detection unit 34b so that the lesion candidate region L can be tracked even when the position of the lesion candidate region L is shifted on the observation image G1. A first lesion candidate area on the first observation image and a second lesion candidate area on the second observation image that is input before the first observation image and stored in the RAM 36a based on the lesion candidate information. It is determined whether or not they are the same lesion candidate region L, and when the same lesion candidate region L is detected continuously or intermittently on a plurality of observation images G1 that are sequentially input, detection of the lesion candidate region L is continued. The determination result is output to the enhancement processing unit 35b of the detection result output unit 35.

The enhancement processing unit 35b determines that the detection of the lesion candidate region L has continued for a predetermined time based on the determination result of the continuation detection determination unit 36 after the enhancement process input from the continuation detection determination unit 36 is started. When this is done, the enhancement process is terminated.

The predetermined time is a time during which the surgeon can sufficiently recognize the lesion candidate region L from the marker image G2, and is set in advance to 1.5 seconds, for example.

According to the second modification of the embodiment, after the marker image G2 is displayed, the marker image G2 is not displayed without requiring an operation instruction to turn off the operation switch W1, and the visibility of the observation image G1 is improved. In addition, the attention area can be presented to the surgeon without suppressing a reduction in attention to the observation image G1 and without hindering the improvement of the lesion finding ability.

In the embodiment, the operation unit includes the operation switch W1, but the operation unit may include a scope switch W2 in addition to the operation switch W1 (FIGS. 1, 2, and FIG. 2). 8 two-dot chain line). The scope switch W2 is provided in the endoscope 21, and is configured by a switch that can be switched between ON and OFF states by an operation instruction with an operator's fingers, like the operation switch W1. The scope switch W2 is connected to the detection result output unit 35 of the video processor 31. The detection result output unit 35 detects the ON / OFF state of the scope switch W2, and performs enhancement processing based on the observation image G1 input from the control unit 32 and the lesion candidate information input from the lesion candidate detection unit 34b. And a notification process can be performed, and a display image G can be generated and output to the display unit 41. According to this configuration, the surgeon can give an operation instruction with either the operation switch W1 or the scope switch W2, and the operation instruction is simple.

In the embodiment, the notification unit 35a displays the notification image G3 in a region other than the observation image G1, but as shown in FIG. 9, the notification unit 35a may display an image G5 surrounding the observation image G1. I do not care. According to this configuration, the display of the image G5 surrounding the observation image G1 makes it easy for the surgeon to notice that the lesion candidate region L is detected when any part of the observation image G1 is focused.

In the embodiment, for the sake of explanation, only one lesion candidate region L is displayed in the observation image G1, but a plurality of lesion candidate regions L may be displayed in the observation image G1. In that case, the notification process and the enhancement process are performed for each lesion candidate region L, and when the lesion candidate region L is detected, the display of the notification image G3 is started, and the operation switch W1 is switched from the OFF state to the ON state. When the operation switch W1 is changed from the ON state to the OFF state, the marker image G2 is hidden and the notification image G3 is displayed. When the lesion candidate region L is not detected, the notification image G3 is hidden.

In the embodiment, the notification unit 35a is configured to notify the surgeon by displaying the notification image G3. However, if the notification unit 35a can notify the operator in a region different from the characteristic region, the notification image G3 is displayed. A method other than display may be used. For example, the notification unit 35a may generate sound from a speaker (not shown) and notify the surgeon. A notification lamp provided in the endoscope system 1 may be turned on.

In the embodiment, the control unit 32 performs image adjustment such as gain adjustment, white balance adjustment, gamma correction, contour enhancement correction, and enlargement / reduction adjustment on the imaging signal input from the endoscope 21, The adjusted observation image G1 is input to the detection support unit 33. However, part or all of the image adjustment is not performed before the detection support unit 33 is input, but to the image signal output from the detection support unit 33. It may be done.

In the embodiment, the detection support unit 33 is disposed inside the video processor 31, but may be disposed outside the video processor 31, for example, between the video processor 31 and the display unit 41.

Furthermore, in the embodiment, the enhancement processing unit 35b adds the marker image G2 to the lesion candidate area L. However, the marker image G2 may be displayed in different colors depending on the probability of the detected lesion candidate area L. I do not care. In this case, the lesion candidate detection unit 34b outputs lesion candidate information including the probability information of the lesion candidate region L to the enhancement processing unit 35b, and the enhancement processing unit 35b performs color coding based on the probability information of the lesion candidate region L. Emphasize processing by. According to this configuration, when observing the lesion candidate region L, the surgeon can estimate the possibility of false positive (false detection) based on the color of the marker image G2.

In the embodiment, the detection support unit 33 is configured by a circuit, but each function of the detection support unit 33 may be configured by a processing program that realizes the function by processing of the CPU.

The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

According to the present invention, it is possible to provide an endoscope apparatus that presents a region of interest to an operator without suppressing a reduction in attention to an observation image and without hindering improvement in lesion finding ability.

This application is filed on the basis of the priority claim of Japanese Patent Application No. 2015-210817 filed in Japan on October 27, 2015. The above disclosure is included in the present specification and claims. Shall be quoted.

Claims

A detection unit that receives an observation image of the subject and detects a feature region in the observation image based on a predetermined feature amount of the observation image;
When the feature area is detected by the detection section, a notification section for notifying the operator that the feature area has been detected by a notification process in a different area from the feature area;
An emphasis processing unit that displays the observation image in which the feature region is enhanced by an emphasis process at a timing different from the start of the notification by the notification unit;
Having
An endoscope apparatus characterized by that.
The endoscope apparatus according to claim 1, wherein the notification unit does not perform notification by the notification unit while the observation image in which the feature region is emphasized is displayed.
The notification unit performs notification by the notification unit when the observation image highlighting the feature region is not displayed and the feature region is detected by the detection unit. The endoscope apparatus described in 1.
The endoscope apparatus according to claim 1, wherein the enhancement process is a process of performing display indicating a position of the feature region.
The endoscope apparatus according to claim 1, wherein the enhancement processing unit performs the enhancement processing of the feature region after notification by the notification unit is started.
It has an operation part that can be switched between an ON state and an OFF state, and an initial state is an OFF state,
When the feature region is detected by the detection unit,
When the operation unit is in an OFF state, the notification process by the notification unit is performed,
When the operation unit is in an ON state, the enhancement processing by the enhancement processing unit is performed.
The endoscope apparatus according to claim 1.
The endoscope apparatus according to claim 6, wherein the operation unit includes one of a foot switch and a scope switch.
The observation image at the time when the notification process is started is stored as a still image, and when the operation unit is in an ON state, a still image processing unit that displays the still image is included.
The endoscope apparatus according to claim 1.
The endoscope apparatus according to claim 1, wherein the enhancement process is terminated after a predetermined time has elapsed after the enhancement process is started.
The detection unit can output the probability information of the detected characteristic region,
The enhancement processing unit performs the enhancement processing by color classification based on the certainty information.
The endoscope apparatus according to claim 1.
The endoscope apparatus according to claim 1, wherein the notification process is a process of displaying an image surrounding the observation image.
The endoscope apparatus according to claim 1, wherein the notification unit generates a sound when the feature region is detected by the detection unit.