JP7411515B2 - Endoscope system and its operating method - Google Patents

Description

The present invention relates to an endoscope system that uses image processing to automatically detect a predetermined lesion candidate region, typically a lesion, using AI, and an operating method thereof.

In the medical field, diagnosis using an endoscope system including a light source device, an endoscope, and a processor device is widely performed. In an endoscope system, illumination light emitted by a light source device is irradiated onto an observation object via an endoscope, and a processor device performs processing based on an image signal obtained by imaging the observation object being illuminated with the illumination light. Generate an image of the observation target. By displaying this image on the display, the doctor can make a diagnosis while viewing the image on the display.

In endoscopy, some doctors use normal light to observe the endoscope when it is inserted, and use special light to remove it; however, lesions that are observed during insertion using only normal light may be overlooked. There is a possibility that it will be stored away. Therefore, a technology has been developed that captures lesion candidate areas with normal light and special light and performs AI processing if necessary, for areas where the doctor freezes or does not capture images when inserting an endoscope, but is suspected of being a lesion. It is being done. For example, during endoscopic diagnosis, a computer can display and notify information about lesion candidate areas obtained by special light irradiation on behalf of the doctor on images obtained by normal light irradiation, which prevents doctors from overlooking lesion candidate areas. Therefore, it is expected to improve the accuracy of endoscopic diagnosis.

Specifically, in Patent Document 1, when an endoscope automatically switches between normal light and special light to emit light, information on areas of interest such as a lesion obtained by special light irradiation is used as a normal light diagnostic image. This makes it easier to detect lesions by improving the visibility of the region of interest on the observation target.

International Publication No. 2019/059059

In Patent Document 1, a user observes a region of interest such as a lesion detected in a specific light image in a white light image. However, as the endoscope tip moves, the object to be observed changes during endoscopic diagnosis, so it is not always possible to perform the diagnosis with appropriate illumination light, and there is a risk that visual information may be missed. For example, more information may be obtained from a white light image than from a specific light image. In order to capture visual characteristics such as the color of a lesion and to understand the shape and condition of the lesion in detail, it is necessary to apply appropriate illumination light according to the object to be observed.

The present invention provides a method for displaying observation images based on appropriate illumination light irradiation according to the observation target when first illumination light and second illumination light having different emission spectra are automatically switched and emitted. It is an object of the present invention to provide an endoscope system and an operating method thereof that can prevent overlooking of lesions and observe lesion candidate regions with high accuracy.

The endoscope system of the present invention includes a light source section that emits first illumination light and second illumination light having an emission spectrum different from that of the first illumination light, a first illumination period that emits the first illumination light, and a second illumination period that emits the first illumination light. In the case of automatically switching between a second illumination period in which light is emitted, a light source processor that emits the first illumination light in a first light emission pattern and emits the second illumination light in a second light emission pattern; an imaging sensor of an endoscope that outputs a first image signal obtained by capturing an image of an observation target illuminated by light; and a second image signal obtained by capturing an image of an observation target illuminated by second illumination light; and a control processor, the image control processor performs recognition processing on the first image signal and the second image signal, and when a lesion candidate area is recognized by the recognition process, calculates the accuracy of the lesion candidate area. When the lesion candidate area is recognized and the accuracy of the lesion candidate area based on the first image signal is higher than the accuracy of the lesion candidate area based on the second image signal, the first image signal based on the first image signal is recognized. A first display control is performed on the observation image to display the lesion candidate area recognized observation image on the display, which displays the recognition result of the lesion candidate area, and the lesion candidate area is recognized and the lesion candidate area is recognized based on the second image signal. If the accuracy of the candidate area is higher than the accuracy of the lesion candidate area based on the first image signal, display a composite observation image in which a second observation image based on the second image signal is combined with the first observation image. When the second display control is performed to display the second illumination light in the second illumination period, and the second light emission pattern is a second B light emission pattern in which the emission spectrum of the second illumination light is different in each second illumination period, the emission spectra are different from each other. In a second illumination light emission cycle in which a plurality of second illumination lights are emitted, if a lesion candidate region is recognized from the plurality of second image signals corresponding to each second illumination light, the image control processor: When performing the second display control, the first observation image and the second observation image with the highest probability of being a lesion candidate area from among the plurality of second observation images are combined to generate a composite observation image.

When the second light emission pattern is a second A light emission pattern in which the emission spectrum of the second illumination light is the same in each second illumination period, the image control processor, when performing the second display control, It is preferable to generate a composite observation image by combining the first observation image and the second observation image.

When the second light emission pattern is a second B light emission pattern in which the emission spectra of the second illumination lights are different in each second illumination period, the second light emission pattern emits a plurality of second illumination lights having different emission spectra. In the illumination light emission cycle, when a lesion candidate region is recognized from at least one of the plurality of second image signals corresponding to each second illumination light, the image control processor performs second display control. It is preferable to generate a composite observation image by combining the first observation image and a second observation image in which a lesion candidate region has been recognized through recognition processing among the plurality of second observation images.

The display has a main screen and a sub-screen, and the image control processor recognizes the lesion candidate area through recognition processing, and when the capture processing for capturing the first observation image or the second observation image is performed. When performing the second display control, it is preferable to display the composite observation image on the main screen and display the second observation image on the sub-screen.

The first light emission pattern has a first light emission pattern in which the number of frames in the first illumination period is the same in each first illumination period, and the first light emission pattern in which the number of frames in the first illumination period is different in each first illumination period. It is preferable that the light emitting pattern is one of the first and second light emitting patterns.

The second light emission pattern includes a 2A light emission pattern in which the number of frames in the second illumination period is the same in each of the second illumination periods, and the emission spectrum of the second illumination light is the same in each of the second illumination periods. A second B light emission pattern, in which the number of frames in the second illumination period is the same in each second illumination period, and the emission spectrum of the second illumination light is different in each second illumination period; A 2C light emission pattern in which the number of frames of the illumination period is different in each second illumination period, and the emission spectrum of the second illumination light is the same in each second illumination period, and the second illumination period. The number of frames is different in each second illumination period, and the emission spectrum of the second illumination light is different in each second illumination period. It is preferable that there be.

It is preferable that the image control processor performs the compositing process when the movement of the endoscope tip is less than a certain amount, and does not perform the compositing process when the movement of the endoscope tip exceeds a certain amount. .

It is preferable that the image control processor generates a composite observation image by combining the lesion candidate region of the second observation image with the first image signal.

The operating method of the endoscope system of the present invention includes a light source unit that emits first illumination light and second illumination light having a different emission spectrum from the first illumination light, a first illumination period that emits the first illumination light, and a first illumination period that emits the first illumination light; a light source processor that emits the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; The endoscope includes an imaging sensor that outputs a first image signal obtained by capturing an image of an observation target illuminated by illumination light, and a second image signal obtained by capturing an image of an observation target illuminated by second illumination light. In the endoscope system, recognition processing is performed on the first image signal and the second image signal, and when a lesion candidate region is recognized by the recognition processing, a step of calculating the accuracy of the lesion candidate region, and a step of calculating the accuracy of the lesion candidate region. , and if the accuracy of the lesion candidate area based on the first image signal is higher than the accuracy of the lesion candidate area based on the second image signal, then , performing first display control to display a lesion candidate area recognized observation image on a display displaying the recognition result of the lesion candidate area, and recognizing the lesion candidate area and detecting the lesion candidate area based on the second image signal. If the accuracy is higher than the accuracy of the lesion candidate area based on the first image signal, a composite observation image obtained by combining the first observation image with the second observation image based on the second image signal is displayed on the display. performing second display control; and when the second light emission pattern is a second B light emission pattern in which the light emission spectrum of the second illumination light is different in each second illumination period, a plurality of light emission patterns having different light emission spectra each other; In the second illumination light emission cycle in which the second illumination light is emitted, if a lesion candidate area is recognized from the plurality of second image signals corresponding to each second illumination light, when performing the second display control, , a composite observation image is generated by combining the first observation image and a second observation image with the highest probability of being a lesion candidate region among the plurality of second observation images. step and have.

According to the present invention, when the first illumination light and the second illumination light having different emission spectra are switched and emitted, the observation image is displayed based on the appropriate illumination light irradiation according to the observation target, so that the lesion can be easily detected. It is possible to prevent overlooking and to observe lesion candidate areas with high accuracy.

FIG. 1 is an external view of an endoscope system. FIG. 2 is a block diagram showing the functions of the endoscope system. It is a graph showing the emission spectra of violet light V, blue light B, green light G, and red light R. It is an explanatory view showing the 1st A light emission pattern and the 2nd A light emission pattern in the multi-light emission mode. FIG. 3 is an explanatory diagram showing a 1B light emission pattern in a multi-light emission mode. FIG. 7 is an explanatory diagram showing a second B light emission pattern in a multi-light emission mode. It is an explanatory view showing the 2nd C light emission pattern in multi-light emission mode. FIG. 7 is an explanatory diagram showing a 2D light emission pattern in a multi-light emission mode. It is a graph showing the spectral transmittance of each color filter of the image sensor. FIG. 3 is an explanatory diagram showing a first imaging period and a second imaging period in a multi-light emission mode. FIG. 3 is an explanatory diagram showing a screen display of an observation image in which a lesion candidate region has been recognized in a multi-light emission mode. FIG. 3 is an explanatory diagram showing a screen display of a composite observation image in a multi-light emission mode. FIG. 6 is an explanatory diagram showing screen switching display in second display control during multi-light emission mode. It is an explanatory view showing the 2nd display control in the case of the 2nd A light emission pattern in the multi-light emission mode. FIG. 7 is an explanatory diagram showing second display control for selecting a specific second observation image from a plurality of second observation images having a lesion candidate area during the second B light emission pattern in the multi-light emission mode. FIG. 7 is an explanatory diagram showing second display control for selecting a second observation image having a lesion candidate area from a plurality of second observation images during the second B light emission pattern in the multi-light emission mode. FIG. 7 is an explanatory diagram showing display control when no lesion is recognized in the multi-light emission mode. It is a flowchart which shows a series of flows in multi-light emission mode.

As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 14, a processor device 16, a display 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and electrically connected to the processor device 16 . The endoscope 12 has an insertion section 12a that is inserted into a subject, an operation section 12b provided at the proximal end of the insertion section 12a, and a curved section 12c and a distal end 12d provided at the distal end of the insertion section 12a. are doing. The bending portion 12c performs a bending operation by operating the angle knob 12e of the operating portion 12b. This bending action directs the tip 12d in a desired direction.

In addition to the angle knob 13a, the operation section 12b also includes a capture processing instruction section 13b used for manual image acquisition operations, a mode changeover switch 13c used for switching observation modes, and a zoom operation section used for changing zoom magnification. 13d is provided. The capture processing instruction unit 13b is used to manually perform a manual capture to obtain an image of an observation target. When an image is acquired by the user operating the capture processing instruction section 13b, the acquired image is stored in the captured image storage memory 70 (see FIG. 2). In addition to manual capture, the image storage control includes automatic capture in which the first observation image or the second observation image is automatically captured when the lesion candidate region DR (see FIG. 11) is recognized.

The endoscope system 10 has a first observation mode, a second observation mode, and a multi-light emission mode as observation modes. When the observation mode is the first observation mode, the first illumination light is emitted by combining multiple colors of light at the light intensity ratio Lc for the first observation mode, and the observation target being illuminated with the first illumination light is emitted. A first observed image is displayed on the display 18 based on the first image signal obtained by imaging. In addition, when the observation mode is the second observation mode, a second illumination light is emitted, which is a combination of light of multiple colors at a light intensity ratio Ls for the second observation mode, and during observation during illumination with this second illumination light. A second observed image is displayed on the display 18 based on the second image signal obtained by imaging the object.

Furthermore, when the observation mode is the multi-light emission mode, the first illumination light and the second illumination light are alternately emitted. If the lesion candidate region DR is not recognized through lesion recognition processing using AI (Artificial Intelligence) or the like, the first observation image obtained by imaging the observation target illuminated with the first illumination light is displayed on the display. 18 main screen 18a. In addition, in the multi-emission mode, if a lesion candidate area is recognized during observation, an optimal image of the site is acquired from the second image signal, and is sent to the synthesis processing unit along with the first observation image, and is synthesized and displayed on the display 18. I do. Note that the site-optimal image is an image determined by AI diagnosis or the like to have a high degree of certainty of the lesion candidate region DR. Further, the accuracy of the lesion candidate region DR is expressed by the probability that it is a lesion such as cancer.

Processor device 16 is electrically connected to display 18 and user interface 19 . The display 18 outputs and displays images of objects to be observed, information attached to the images, and the like. The user interface 19 accepts input operations such as function settings.

In FIG. 2, the light source device 14 includes a light source section 20 and a light source processor 21 that controls the light source section 20. The light source section 20 has, for example, a plurality of semiconductor light sources, turns on or off each of these, and when turned on, controls the amount of light emitted from each semiconductor light source, thereby emitting illumination light that illuminates the observation target. In the present embodiment, the light source section 20 includes a V-LED (Violet Light Emitting Diode) 20a, a B-LED (Blue Light Emitting Diode) 20b, a G-LED (Green Light Emitting Diode) 20c, and an R-LED (Red Light Emitting Diode). Emitting Diode) has 20d four-color LEDs.

As shown in FIG. 3, the V-LED 20a generates violet light V with a center wavelength of 405±10 nm and a wavelength range of 380 to 420 nm. The B-LED 20b generates blue light B with a center wavelength of 450±10 nm and a wavelength range of 420 to 500 nm. The G-LED 20c generates green light G with a wavelength range of 480 to 600 nm. The R-LED 20d generates red light R with a center wavelength of 620-630 nm and a wavelength range of 600-650 nm.

The light source processor 21 controls the V-LED 20a, B-LED 20b, G-LED 20c, and R-LED 20d. By independently controlling each of the LEDs 20a to 20d, the light source processor 21 can emit violet light V, blue light B, green light G, or red light R by independently changing the amount of light. Further, the light source processor 21 is configured to emit white light in which the light amount ratio among the violet light V, blue light B, green light G, and red light R is Vc:Bc:Gc:Rc in the first observation mode. Then, each LED 20a to 20d is controlled. Note that Vc, Bc, Gc, and Rc>0.

In addition, in the second observation mode, the light source processor 21 has a light amount ratio of Vs:Bs:Gs:Rs between violet light V, blue light B, green light G, and red light R as short wavelength narrow band light. Each of the LEDs 20a to 20d is controlled so as to emit the second illumination light. The light amount ratio Vs:Bs:Gs:Rs is different from the light amount ratio Vc:Bc:Gc:Rc used in the first observation mode, and is appropriately determined depending on the observation purpose. For example, when emphasizing superficial blood vessels, it is preferable to make Vs larger than other Bs, Gs, and Rs, and when emphasizing middle and deep blood vessels, it is preferable to make Gs larger than other Vs, Gs, and Rs. It is also preferable to make it larger.

Further, in the case where the light source processor 21 emits light by automatically switching between the first illumination light and the second illumination light in the multi-light emission mode, the light source processor 21 emits the first illumination light in the first light emission pattern and the second illumination light. Light is emitted in a second light emitting pattern. Specifically, as shown in FIG. 4, the first light emission pattern includes a first light emission pattern in which the number of frames in the first illumination period is the same in each first illumination period, and a first light emission pattern, as shown in FIG. It is preferable that the number of frames in the first illumination period is one of the 1B light emission patterns that are different in each of the first illumination periods.

As shown in FIG. 4, the second light emission pattern is such that the number of frames in the second illumination period is the same in each second illumination period, and the emission spectrum of the second illumination light is the same in each second illumination period. As shown in FIG. 6, the number of frames in the second illumination period is the same in each second illumination period, and the emission spectrum of the second illumination light is the same in each second illumination period. As shown in FIG. 7, the number of frames in the second illumination period is different in each second illumination period, and the emission spectrum of the second illumination light is different in the second illumination period. As shown in FIG. 8, the second C light emission pattern is the same in each of the second illumination periods, and the number of frames in the second illumination period is different in each of the second illumination periods; Preferably, the spectrum is one of the second D emission patterns that is different in each second illumination period. Note that the emission spectrum of the first illumination light may be the same or different in each first illumination period.

Here, the first illumination period is preferably longer than the second illumination period, and the first illumination period is preferably two or more frames. For example, in FIG. 4, when the first light emission pattern is the 1A light emission pattern and the second light emission pattern is the 2A light emission pattern (number of frames in the second illumination period: the same, emission spectrum of the second illumination light: the same) In this case, the first illumination period is two frames, and the second illumination period is one frame. Since the first illumination light is used to generate the first observation image displayed on the display 18, it is preferable that a bright image be obtained by illuminating the observation target with the first illumination light.

For example, it is preferable that the first illumination light is white light. On the other hand, since the second illumination light is used for the compositing process, it is preferable that an image suitable for the compositing process be obtained by illuminating the observation target with the second illumination light. For example, when performing synthesis processing based on shape information of multiple blood vessels with different blood vessel depths, it is possible to use violet light V, blue light B, green light G, and red light R as the second illumination light. preferable. In this case, the second light emission pattern is set to the 2A light emission pattern (the number of frames in the second illumination period: the same, the emission spectrum of the second illumination light: the same) or the 2C light emission pattern (the number of frames in the second illumination period: different, the second illumination light emission spectrum: the same). In the case where the emission spectra of the two illumination lights are the same, it is preferable to use any one of violet light V, blue light B, green light G, and red light R. On the other hand, the second light emission pattern is changed to the 2B light emission pattern (the number of frames in the second illumination period: the same, the emission spectrum of the second illumination light: different) or the 2D light emission pattern (the number of frames in the second illumination period: different, the second Emission spectrum of illumination light: different), in the second illumination period, at least two of the violet light V, blue light B, green light G, and red light R are switched in a specific order and emitted. It is preferable.

The details of the first and second light emitting patterns, which are switching patterns between the first illumination period and the second illumination period, will be described later since they are determined based on the imaging control of the imaging sensor 44 by the imaging processor 45. Further, a frame refers to a unit of period including at least a period from a specific timing to completion of signal readout in the image sensor 44.

For example, superficial blood vessels within a depth of 50 μm from the mucosal surface, middle blood vessels within a depth of 200 μm from the mucosal surface, and deep blood vessels within a depth of 600 μm from the mucosal surface. When obtaining blood vessel shape information and performing synthesis processing based on the shape information of superficial, middle, and deep blood vessels, violet light V emphasizes superficial blood vessels, green light G emphasizes middle blood vessels, It is preferable to use red light R that emphasizes deep blood vessels.

Note that in this specification, the light amount ratio includes a case where the ratio of at least one semiconductor light source is 0 (zero). Therefore, this includes cases where one or more of the semiconductor light sources do not light up. For example, when the light intensity ratio between violet light V, blue light B, green light G, and red light R is 1:0:0:0, only one of the semiconductor light sources is turned on, and the other three It is assumed that the light intensity ratio is maintained even when the light is not lit.

As shown in FIG. 2, the light emitted from each of the LEDs 20a to 20d is incident on a light guide 25 via an optical path coupling section 23 composed of a mirror, a lens, and the like. The light guide 25 is built into the endoscope 12 and a universal cord (a cord that connects the endoscope 12, the light source device 14, and the processor device 16). The light guide 25 propagates the light from the optical path coupling section 23 to the distal end portion 12d of the endoscope 12.

The distal end portion 12d of the endoscope 12 is provided with an illumination optical system 30a and an imaging optical system 30b. The illumination optical system 30a has an illumination lens 32, and the illumination light propagated by the light guide 25 is irradiated onto the observation target via the illumination lens 32. The imaging optical system 30b includes an objective lens 42 and an imaging sensor 44. Light emitted from the observation target by illumination light enters the image sensor 44 via the objective lens 42 and the zoom lens 43. As a result, an image of the observation target is formed on the image sensor 44. The zoom lens 43 is a lens for enlarging an observation target, and is moved between a telephoto end and a wide end by operating the zoom operation section 12h.

The image sensor 44 is a primary color sensor, and includes a B pixel (blue pixel) having a blue color filter, a G pixel (green pixel) having a green color filter, and an R pixel (red pixel) having a red color filter. It has three types of pixels. As shown in FIG. 9, the blue color filter BF mainly transmits light in a blue band, specifically, light in a wavelength band of 380 to 560 nm. The transmittance of the blue color filter BF reaches a peak near a wavelength of 460 to 470 nm. The green color filter transmits GF, mainly light in the green band, specifically, light in the wavelength band of 460 to 620 nm. The red color filter RF mainly transmits light in the red band, specifically, light in the wavelength band of 580 to 760 nm.

Moreover, it is preferable that the image sensor 44 is a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The imaging processor 45 controls the imaging sensor 44 . Specifically, an image signal is output from the image sensor 44 by reading out a signal from the image sensor 44 by the image sensor 45 . In the first observation mode, when the image sensor 44 is exposed to the first illumination light, the image sensor 45 reads out the signal, so that the B pixel of the image sensor 44 outputs a Bc image signal, and the G pixel outputs a Bc image signal. A Gc image signal is output, and an Rc image signal is output from the R pixel. In the second observation mode, when the image sensor 44 is exposed to the second illumination light, the image sensor 45 reads the signal, so that the Bs image signal is output from the B pixel of the image sensor 44, and the Bs image signal is output from the G pixel of the image sensor 44. A Gs image signal is output, and an Rs image signal is output from the R pixel.

In the multi-light emission mode, as shown in FIG. 10, the imaging processor 45 reads out signals from the imaging sensor 44 while exposing the imaging sensor 44 to the first illumination light during the first illumination period. Output one image signal. The period during which the first image signal is output is referred to as a first imaging period. The first image signal includes a B1 image signal output from the B pixel, a G1 image signal output from the G pixel, and an R1 image signal output from the R pixel. Further, the imaging processor 45 causes the imaging sensor 44 to output a second image signal by reading a signal while exposing the imaging sensor 44 to the second illumination light during the second illumination period. The period during which the second image signal is output is defined as a second imaging period. The second image signal includes a B2 image signal output from the B pixel, a G2 image signal output from the G pixel, and an R2 image signal output from the R pixel.

As shown in FIG. 2, a CDS/AGC (Correlated Double Sampling/Automatic Gain Control) circuit 46 performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the image sensor 44. . The image signal that has passed through the CDS/AGC circuit 46 is converted into a digital image signal by an A/D (Analog/Digital) converter 48. The digital image signal after A/D conversion is input to the processor device 16.

In the processor device 16, programs related to processing such as image processing are stored in a program memory (not shown). In the processor device 16, an image acquisition section 50, a DSP (Digital Signal Processor) 52, and a noise reduction section are operated by a central control section 68 constituted by an image control processor, by operating a program in a program memory. 54, an image processing switching section 56, an image processing section 58, and a display control section 60. Furthermore, with the realization of the functions of the image processing section 58, the functions of the first observation image generation section 62, the second observation image generation section 64, and the recognition processing section 66 are realized. Note that in the multi-light emission mode, the image control processor performs image processing based on the first image signal or the second image signal, and controls the display 18.

The image acquisition unit 50 acquires a color image input from the endoscope 12. The color image includes a blue signal (B image signal), a green signal (G image signal), and a red signal (R image signal) output from the B pixel, G pixel, and R pixel of the image sensor 44. The acquired color image is sent to the DSP 52. The DSP 52 performs various signal processing such as defect correction processing, offset processing, gain correction processing, matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing on the received color image. In the defect correction process, signals of defective pixels of the image sensor 44 are corrected. In the offset processing, dark current components are removed from the image signal that has been subjected to the defect correction processing, and an accurate zero level is set. The gain correction process adjusts the signal level of the color image by multiplying the image signal of each color after the offset process by a specific gain coefficient. The specific gain coefficients include a B gain coefficient by which the B image signal is multiplied, a G gain coefficient by which the G image signal is multiplied, and an R gain coefficient by which the R image signal is multiplied. .

The image signal of each color after the gain correction process is subjected to matrix processing to improve color reproducibility. After that, the brightness and saturation of the color image are adjusted by gamma conversion processing. The color image after matrix processing is subjected to demosaic processing (also referred to as isotropic processing or simultaneous processing), and a signal of the missing color of each pixel is generated by interpolation. Through demosaic processing, all pixels have signals of each RGB color. The DSP 52 performs YC conversion processing on the demosaiced color image, and outputs the luminance signal Y, color difference signal Cb, and color difference signal Cr to the noise reduction unit 54.

The noise reduction unit 54 performs noise reduction processing using, for example, a moving average method or a median filter method on the color image that has been subjected to demosaic processing or the like by the DSP 56 . The color image with reduced noise is input to the image processing switching unit 56.

The image processing switching unit 56 determines whether the image signal from the noise reduction unit 54 is transmitted to the first observation image generation unit 62 or the second observation image generation unit, which constitutes the image processing unit 58, depending on the set observation mode. 64 or recognition processing section 66. Specifically, when the first observation mode is set, the image signal from the noise reduction section 54 is input to the first observation image generation section 62. When set to the second observation mode, the image signal from the noise reduction section 54 is input to the second observation image generation section 64. When the multi-light emission mode is set, the image signal from the noise reduction section 54 is input to the recognition processing section 66.

The first observation image generation unit 62 performs first observation image image processing on the input Rc image signal, Gc image signal, and Bc image signal for one frame. Image processing for the first observation image includes 3×3 matrix processing, gradation conversion processing, color conversion processing such as three-dimensional LUT (Look Up Table) processing, color enhancement processing, and structure enhancement processing such as spatial frequency enhancement. included. The Rc image signal, Gc image signal, and Bc image signal that have been subjected to the first observation image image processing are input to the display control unit 60 as the first observation image.

The second observation image generation unit 64 performs second observation image image processing on the input Rs image signal, Gs image signal, and Bs image signal for one frame. Image processing for the second observation image includes 3×3 matrix processing, gradation conversion processing, color conversion processing such as three-dimensional LUT (Look Up Table) processing, color enhancement processing, and structure enhancement processing such as spatial frequency enhancement. included. The Rs image signal, Gs image signal, and Bs image signal subjected to the second observation image image processing are input to the display control unit 60 as the second observation image.

The recognition processing unit 66 performs recognition processing on the first image signal and the second image signal to determine whether or not they have a lesion candidate region DR, and when the lesion candidate region DR is recognized, it also determines and compares the accuracy. conduct. The recognition processing unit 66 recognizes the lesion candidate region DR, and if the accuracy of the lesion candidate region DR based on the second image signal is higher than the accuracy of the lesion candidate region DR based on the first image signal, the recognition processing unit 66 recognizes the lesion candidate region DR. The first observation image generation unit 62 and second observation image generation are performed for image signals (R1 image signal, G1 image signal, B1 image signal) and second image signals (R2 image signal, G2 image signal, B2 image signal). The image processing unit 64 performs the same image processing as the unit 64 to obtain a first observation image and a second observation image, and transmits them to the synthesis processing unit 67 . On the other hand, when the lesion candidate region DR is recognized and the accuracy of the lesion candidate region DR based on the first image signal is higher than the accuracy of the lesion candidate region DR based on the second image signal, the lesion candidate region DR is recognized based on the first image signal. For the first observation image, a lesion candidate region recognized observation image OP1 that displays the recognition result of the lesion candidate region DR is transmitted to the display control unit 60. Note that if neither the first image signal nor the second image signal recognizes the lesion candidate region DR, image processing is performed only on the first image signal, and the first observation image is transmitted to the display control unit 60. Furthermore, when the lesion candidate region DR is recognized, automatic capture is performed to automatically capture an image.

The combination processing unit 67 performs a combination process on the first observed image input from the recognition processing unit 66 and the second observed image. In the compositing process, the optimal part of the lesion candidate region DR (see the figure) in the second observation image is composited with the same part in the first observation image. The composite observation image OP2 obtained through the composite processing is input to the display control unit 60.

Note that the optimum region of the lesion candidate region DR in the second observation image to be subjected to the synthesis process is preferably a region with a certain level of accuracy or more, and is preferably a rectangular area.

The display control unit 60 performs control to display the input observation image on the display 18. Specifically, the display control unit 60 converts each observed image into a video signal that can be displayed in full color on the display 18. The converted video signal is input to the display 18. As a result, each observed image is displayed on the display 18.

In the multi-light emission mode, when the lesion candidate region DR is recognized and the accuracy of the lesion candidate region DR based on the first image signal is higher than the accuracy of the lesion candidate region DR based on the second image signal, the lesion candidate region DR is recognized. When the completed observation image OP1 is input to the display control unit 60, the display control unit 60 performs first display control to display the lesion candidate region recognized observation image OP1 on the display 18, as shown in FIG. . In FIG. 11, the recognition result of the lesion candidate region DR is displayed as a rectangular area BA1 surrounding the lesion candidate region DR.

On the other hand, when the lesion candidate region DR is recognized and the accuracy of the lesion candidate region DR based on the second image signal is higher than the accuracy of the lesion candidate region DR based on the first image signal, the composite observation image OP2 is displayed. When input to the control unit 60, the display control unit 60 performs second display control to display the composite observation image OP2 on the display 18, as shown in FIG. In the composite observation image OP2 of FIG. 12, the optimum site BA2 of the lesion candidate region DR in the second observation image is displayed in a composite manner with respect to the first observation image. In the composite observation image OP2, only the lesion candidate region DR is displayed in a color different from that of normal mucous membranes, and the rest is displayed in the color of normal mucous membranes. The candidate region DR can be highlighted.

Note that, as shown in FIG. 13, the display control unit 60 has a main screen 18a and a sub-screen 18b. When a capture process for capturing observation images is performed, the composite observation image OP2 may be displayed on the main screen 18a, and the second observation image may be displayed on the sub-screen. Further, if the lesion candidate region DR is not recognized, the first observation image is displayed on the main screen 18a. The display period of the same observation image is performed in accordance with the frame of the imaging period, and the observation image of the immediately previous frame is continuously displayed in the frame corresponding to the second illumination period.

Display control will be described in the case where the lesion candidate region DR is recognized in the second imaging period with reference to FIGS. 14 to 16 and in the case where the lesion candidate region DR is not recognized in the second imaging period with reference to FIG. 17. Note that the lesion candidate regions DR recognized from the second image signal with this display control are all assumed to have higher accuracy than the first image signal, and if the lesion is not recognized, even if the lesion is recognized, the accuracy is lower than the first image signal. Also included.

When the emission spectrum of the second illumination light in the multi-emission mode is one type (the first emission pattern is the 1A emission pattern, the second emission pattern is the 2A emission pattern (number of frames in the second illumination period: the same, The second display control in the case where the emission spectra of the two illumination lights are the same will be described. As shown in FIG. 14, the second image signal in which the lesion candidate region DR has been recognized by the recognition processing unit 66 is subjected to image processing, and is sent to the display control unit 60 as a second observation image. It is transmitted to the synthesis processing unit 67 together with the obtained first observation image, and a synthesis process is performed. The second observation image having the lesion candidate region DR is subjected to a composition processing by combining the optimal part of the lesion candidate region DR with the first observation image in the composition processing section 67 to obtain a composite observation image OP2, which is transmitted to the display control section 60. . Further, when the lesion candidate region DR is recognized, automatic capture is performed to automatically capture an image. The display control unit 60 displays the composite observation image OP2 on the main screen 18a of the display 18, and displays the second observation image used for composition on the sub-screen 18b. Every time the recognition processing unit 66 recognizes the lesion candidate region DR in the newly captured second observation image, the synthesis processing unit 67 acquires a synthesis observation image OP2, and the display on the display 18 is switched.

In the multi-light emission mode, as shown in FIG. In the case where the number of frames of the illumination period is the same and the emission spectrum of the second illumination light is different), the second illumination light emission cycle (in FIG. 15, the emission spectrum X, Y The second display control when the lesion candidate region DR is recognized from the plurality of second image signals corresponding to each second illumination light in the cycle in which two types of second illumination light are emitted will be described.

The lesion candidate region DR is determined by both the second image signal acquired during the imaging period CP2X based on the second illumination light of emission spectrum X and the second image signal acquired during the imaging period CP2Y based on the second illumination light of emission spectrum Y. If the second observation image is recognized, the second observation image having the lesion candidate region DR with the highest probability of being the same lesion among the plurality of second observation images is combined with the first observation image. In FIG. 15, it is assumed that the accuracy of the lesion candidate region DR recognized from the second image signal of the second imaging period CP2X is higher than the accuracy of the lesion candidate region DR recognized from the second image signal of the second imaging period CP2Y. The optimal part BA2 of the second image signal of the second imaging period CP2X is combined with the first observation image. The composite observation image OP2 obtained by the composition is displayed on the main screen 18a of the display 18. Note that the combination processing of the same second observation image in the composite observation image OP2 can be continued only within a certain period of time and only when the first observation image captures the same lesion candidate region DR. As image processing continues for the same second observation image, the display on the sub-screen 18b similarly continues to display the second observation image.

In the multi-light emission mode, as shown in FIG. In the case where the number of frames of the illumination period is the same and the emission spectrum of the second illumination light is different), the second illumination light emission cycle (in FIG. 16, the emission spectra X, Y The second display control when the lesion candidate region DR is recognized from at least one of the plurality of second image signals corresponding to each second illumination light in the cycle in which two types of second illumination light are emitted will be described. .

The lesion candidate region DR is recognized using the second image signal acquired during the imaging period CP2X based on the second illumination light of emission spectrum X, and the second image signal acquired during the imaging period CP2Y based on the second illumination light of emission spectrum Y. If the lesion candidate region DR is not recognized (in the case of non-recognition), the optimum region BA2 of the second image signal of the second imaging period CP2X is synthesized with the first observation image. The composite observation image OP2 obtained by the composition is displayed on the main screen 18a of the display 18. Note that the same second observation image is subjected to a synthesis process for a certain period of time, or the synthesis is stopped when the number of parts corresponding to the first observation image to be synthesized becomes less than a certain amount. If the second image signal including the new lesion candidate region DR cannot be obtained by this time, the display control is shifted to when no lesion is recognized.

Display control when no lesion is recognized in the multi-light emission mode will be explained using FIG. 17. If the second image signal is not recognized as a lesion candidate region DR as a result of the recognition process on the second image signal, the first observation image is sent to the display control unit 60 without image processing. The display control unit 60 displays the first observation image on the main screen 18a of the display 18. The acquired first observation image is displayed frame by frame, but in the second illumination period, the first observation image of the immediately previous frame is displayed continuously. As long as the second image signal having the lesion candidate region DR is not acquired, the first observation image is continued to be displayed on the display 18, and when the lesion candidate region DR is recognized, the display control shifts to the time of lesion recognition. Note that when no lesion is recognized, the second observation image may be continuously displayed instead of the first observation image.

In the case of a 2B or 2D light emission pattern that is imaged in a plurality of frames during the same second imaging period, the lesion candidate region DR is determined using any second image signal. As a result of the determination, the second image signal that has the lesion candidate region DR and is more accurate is set as the second image signal acquired in the second imaging period.

Note that it is preferable that the sub-screen 18b on the display 18 be developed from the time when the recognition processing unit 66 acquires the second observation image having the lesion candidate region DR or from the start of observation. In FIG. 16, when recognition and capture processing of the lesion candidate region DR is performed, a composite observation image OP2 is displayed on the main screen 18a of one display 18, and a second observation image is displayed on the sub-screen 18b. ing. Alternatively, a plurality of displays 18 may be prepared, and the first display 18 may display the main screen, and the second display 18 may display the sub-screen 18b.

In order to obtain the composite observation image OP2, the imaging range in the frame for obtaining the first observation image and the frame for obtaining the second observation image must be approximately the same, so the movement of the tip of the endoscope is If the amount is less than a certain amount, compositing processing is performed. If the movement of the endoscope tip exceeds a certain amount, the first observation image is displayed on the main screen 18a of the display 18 without performing the compositing process, even if the second observation image has a lesion candidate region DR. .

Details of image storage control in multi-light emission mode will be explained. In the multi-light emission mode, when the lesion candidate region DR is recognized, the image control processor issues a capture processing instruction for the recognized first imaging period or second imaging period. When a capture processing instruction is given during the first imaging period, the first observation image with the smallest blur is stored in the captured image storage memory 70 from among the plurality of first observation images obtained during the first imaging period. When the capture process is executed and a capture image acquisition instruction is given in the second imaging period (see FIG. 10), the second observation image obtained in the second imaging period and the above from the first imaging period immediately after that are executed. In the same manner as above, the first observation image with the smaller blur is obtained. Furthermore, a process of combining the first and second observation images is performed to obtain a combined observation image OP2. Each acquired observation image is subjected to a capture process to be stored in the captured image storage memory 70.

Specifically, when the capture process is performed in the first imaging period, the image acquisition period is set to all frames in the most recent first imaging period, including the frame at the timing when the capture process was performed. That is, the image acquisition period is all frames of the first imaging period from the end of the second imaging period before the first imaging period until the start of the next second imaging period. Then, among the first captured images obtained during the image acquisition period, the first captured image with less blur is stored in the captured image storage memory 70. When the capture process is performed during the second imaging period, the image acquisition period includes the frame immediately preceding the timing at which the capture image acquisition instruction was performed during the second imaging period, as well as when there is light emission with a different spectrum. also includes frames of the second imaging period corresponding to all remaining types of emission spectra. For example, when the second illumination light is violet light V, green light G, and red light R, if the capture processing instruction is performed during the second imaging period of violet light V, the image acquisition period is , frames of the second imaging period corresponding to the violet light V, frames of the second imaging period corresponding to the green light G, and frames of the second imaging period corresponding to the red light R are included. Therefore, by issuing a capture processing instruction, the second observation image corresponding to the violet light V, the second observation image corresponding to the green light, and the second observation image corresponding to the red light R are stored in the memory for storing captured images. 70 is saved.

Recognition processing is performed on each acquired second image signal, and the accuracy of the lesion candidate region DR is compared. Based on the comparison result, the first observation image of the frame immediately after the second image signal with high accuracy of the lesion candidate region DR is sent to the synthesis processing unit 67, and the synthesis processing is performed and the obtained synthesis observation image OP2 is also saved as a captured image. The data is saved in the memory 70 for use.

Next, a series of steps in the multi-light emission mode will be explained along the flowchart shown in FIG. 18. First, operate the mode changeover switch 12f to switch the observation mode to the multi-light emission mode. When switched to the multi-light emission mode, the first illumination period and the second illumination period are automatically switched in a specific light emission pattern. The first illumination light and the second illumination light, which are automatically switched according to the specific light emission pattern in this way, are irradiated onto the observation target.

The imaging processor 45 outputs a first image signal from the imaging sensor 44 by causing the imaging sensor 44 to take an image of the observation target illuminated with the first illumination light. Similarly, the imaging processor 45 outputs a second image signal from the imaging sensor 44 by causing the imaging sensor 44 to image the observation target illuminated with the second illumination light during the second illumination period.

The recognition processing unit 66 performs recognition processing on the first image signal and the second image signal to recognize the lesion candidate region DR and determine the accuracy. When the second image signal has a highly accurate lesion candidate region DR, image processing is performed on the first and second image signals to obtain a first observation image and a second observation image. The first and second observation images are sent to the combination processing unit 67, and the combination processing unit 67 obtains a combination observation image OP2. The composite observation image OP2 is transmitted to the display control unit 60 and displayed on the display 18 together with the original second observation image. If the second image signal does not have a lesion candidate region DR with higher accuracy than the first image signal, the first observed image is transmitted to the display control unit 60 and displayed on the display 18. When observation continues and new first and second image signals are acquired, recognition of the lesion candidate tract region and determination of accuracy are performed each time.

In this embodiment, violet light V, blue light B, green light G, and red light R are used as the second illumination light, but light with other emission spectra may be used. For example, in the first illumination light, first special light in which the amount of violet light V is larger than the amount of other blue light B, green light G, and red light R may be used. Further, in the first illumination light, second special light may be used in which the amount of green light G is larger than the amount of other violet light V, blue light B, and red light R. Further, as the second illumination light, both the first special light and the second special light are used, the second light emission pattern is set as the 2B light emission pattern or the 2D light emission pattern, and the first special light and the second special light are alternately used. It may also be made to emit light. Note that when the first special light is used, color difference expansion processing may be performed on the second image signal to expand the color difference between a normal area and a diseased area included in the observation target. The compositing process may be performed on the second image signal that has undergone the color difference expansion process.

Note that in this embodiment, the first observation image based on the first image signal is used as an observation image when the lesion candidate region DR is not recognized or when the accuracy of the lesion candidate region DR is higher than the second image signal. The second observation image based on the two-image signal is used for synthesis processing when the lesion candidate region DR is recognized from the first image signal, and is displayed on the display 18 together with the synthesis observation image OP2. The second observed image may be displayed on the display 18 when the combining process is not performed. In this case, the first observation image and the second observation image are switched and displayed on the display 18, or a new sub-screen 18b is developed. It is preferable that display or non-display of the first and second observation images on the display 18 can be set as appropriate using the user interface 19. Further, it is preferable that the user interface 19 allows settings such as switching the display of the main screen 18a and the sub-screen 18b as appropriate.

In the above embodiment, the first observation image generation unit 62 and the second observation image generation unit included in the light source processor 21, the imaging processor 45, the image acquisition unit 50, the DPS 52, the noise reduction unit 54, the image processing switching unit 56, and the image processing unit 58 The hardware structure of the processing unit that executes various processes such as the observation image generation section 64, the recognition processing section 66, the composition processing section 67, and the central control section 68 includes the following various processors ( processor). Various types of processors include CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays), which are general-purpose processors that execute software (programs) and function as various processing units.The circuit configuration is changed after manufacturing. These include a programmable logic device (PLD), which is a capable processor, and a dedicated electric circuit, which is a processor having a circuit configuration specially designed to execute various types of processing.

One processing unit may be composed of one of these various types of processors, or may be composed of a combination of two or more processors of the same type or different types (for example, multiple FPGAs or a combination of a CPU and an FPGA). may be done. Further, the plurality of processing units may be configured with one processor. As an example of configuring multiple processing units with one processor, first, as typified by computers such as clients and servers, one processor is configured with a combination of one or more CPUs and software, There is a form in which this processor functions as a plurality of processing units. Second, there are processors that use a single IC (Integrated Circuit) chip to implement the functions of an entire system including multiple processing units, as typified by System On Chip (SoC). be. In this way, various processing units are configured using one or more of the various processors described above as a hardware structure.

Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in the form of a combination of circuit elements such as semiconductor elements. Further, the hardware structure of the storage unit is a storage device such as an HDD (hard disc drive) or an SSD (solid state drive).

10 Endoscope system 12 Endoscope 12a Insertion section 12b Operation section 12c Curved section 12d Tip section 13a Angle knob 13b Capture processing instruction section 13c Mode changeover switch 13d Zoom operation section 14 Light source device 16 Processor device 18 Display 18a Main screen 18b Sub Screen 19 User interface 20 Light source section 20a V-LED
20b B-LED
20c G-LED
20d R-LED
21 Light source processor 23 Optical path coupling section 25 Light guide 30a Illumination optical system 30b Imaging optical system 32 Illumination lens 42 Objective lens 43 Zoom lens 44 Image sensor 45 Imaging processor 46 CDS/AGC circuit 48 A/D converter 50 Image acquisition section 52 DSP
54 Noise reduction section 56 Image processing switching section 58 Image processing section 60 Display control section 62 First observation image generation section 64 Second observation image generation section 66 Recognition processing section 67 Combination processing section 68 Central control section 70 Captured image storage memory BF Blue color filter GF Green color filter RF Red color filter DR Lesion candidate region BA1 Rectangular area surrounding the lesion candidate region DR in the first observation image BA2 Optimal site OP1 of the lesion candidate region DR in the second observation image Lesion candidate region Recognized observation image OP2 Composite observation image

Claims (9)

a light source unit that emits first illumination light and second illumination light having a different emission spectrum from the first illumination light;
In the case where a first illumination period in which the first illumination light is emitted and a second illumination period in which the second illumination light is emitted are automatically switched, the first illumination light is emitted in a first light emission pattern, and a light source processor that emits the second illumination light in a second light emission pattern;
an endoscope that outputs a first image signal obtained by imaging the observation target illuminated by the first illumination light; and a second image signal obtained by imaging the observation target illuminated by the second illumination light. an image sensor,
Equipped with an image control processor,
The image control processor includes:
Performing a recognition process on the first image signal and the second image signal, and when a lesion candidate area is recognized by the recognition process, calculating the accuracy of the lesion candidate area,
When the lesion candidate area is recognized and the accuracy of the lesion candidate area based on the first image signal is higher than the accuracy of the lesion candidate area based on the second image signal, the first image signal performing first display control to display a lesion candidate region recognized observation image on a display, which displays the recognition result of the lesion candidate region, on the first observation image based on the first observation image;
When the lesion candidate area is recognized and the accuracy of the lesion candidate area based on the second image signal is higher than the accuracy of the lesion candidate area based on the first image signal, the first observed image , performing a second display control to display on the display a composite observation image obtained by combining a second observation image based on the second image signal;
When the second light emission pattern is a second B light emission pattern in which the emission spectra of the second illumination light are different in each of the second illumination periods, emitting a plurality of second illumination lights having mutually different emission spectra. In the second illumination light emission cycle, when the lesion candidate region is recognized from all of the plurality of second image signals corresponding to each second illumination light, the image control processor controls the second display control. an endoscope that generates the composite observation image by combining the first observation image and a second observation image with the highest accuracy of the lesion candidate region from among the plurality of second observation images. system. When the second light emission pattern is a second A light emission pattern in which the emission spectrum of the second illumination light is the same in each of the second illumination periods, the image control processor controls the second display control The endoscope system according to claim 1, wherein the first observation image and the second observation image are combined to generate the composite observation image. When the second light emission pattern is a second B light emission pattern in which the emission spectra of the second illumination light are different in each of the second illumination periods, emitting a plurality of second illumination lights having mutually different emission spectra. In the second illumination light emission cycle, when the lesion candidate region is recognized from at least one of the plurality of second image signals corresponding to each second illumination light, the image control processor When performing control, the first observation image and a second observation image in which a lesion candidate area has been recognized by the recognition process among the plurality of second observation images are combined to generate the composite observation image. The endoscope system according to item 1. The display has a main screen and a sub screen,
The image control processor includes:
If a lesion candidate area is recognized by the recognition process and a capture process is performed to capture the first observation image or the second observation image, when performing the second display control, the composite observation The endoscope system according to any one of claims 1 to 3 , wherein the image is displayed on the main screen, and the second observation image is displayed on a sub-screen. The first light emission pattern includes a first light emission pattern in which the number of frames in the first illumination period is the same in each of the first illumination periods, and a first light emission pattern in which the number of frames in the first illumination period is the same in each of the first illumination periods. The endoscope system according to any one of claims 1 to 4 , wherein the light emitting pattern is one of the different 1B light emission patterns. In the second light emission pattern, the number of frames of the second illumination period is the same in each of the second illumination periods, and the emission spectrum of the second illumination light is the same in each of the second illumination periods. A certain 2nd A light emission pattern,
a second B light emission pattern in which the number of frames of the second illumination period is the same in each of the second illumination periods, and the emission spectrum of the second illumination light is different in each of the second illumination periods;
a second C light emission pattern in which the number of frames of the second illumination period is different in each of the second illumination periods, and the emission spectrum of the second illumination light is the same in each of the second illumination periods, and ,
A 2D light emission pattern in which the number of frames of the second illumination period is different in each of the second illumination periods, and the emission spectrum of the second illumination light is different in each of the second illumination periods. The endoscope system according to claim 1, which is any one of the patterns. The image control processor includes:
Any one of claims 1 to 6 , wherein the synthesis process is performed when the movement of the tip of the endoscope is less than a certain amount, and the synthesis process is not performed when the movement of the end of the endoscope exceeds the certain amount. The endoscopic system described in Section. The image control processor includes:
The endoscope system according to any one of claims 1 to 7 , wherein the combined observation image is generated by synthesizing the lesion candidate region part of the second observation image with the first image signal. A light source unit that emits first illumination light and second illumination light having a different emission spectrum from the first illumination light; a first illumination period that emits the first illumination light; and a second illumination unit that emits the second illumination light. a light source processor that emits the first illumination light in a first light emission pattern and the second illumination light in a second light emission pattern; An endoscope system comprising an endoscope imaging sensor that outputs a first image signal obtained by imaging an observation target, and a second image signal obtained by imaging the observation target illuminated by the second illumination light. In,
performing a recognition process on the first image signal and the second image signal, and when a lesion candidate area is recognized by the recognition process, calculating the accuracy of the lesion candidate area;
When the lesion candidate area is recognized and the accuracy of the lesion candidate area based on the first image signal is higher than the accuracy of the lesion candidate area based on the second image signal, the first image signal performing first display control to display a lesion candidate area recognized observation image on a display, which displays the recognition result of the lesion candidate area, with respect to the first observation image based on the first observation image;
When the lesion candidate area is recognized and the accuracy of the lesion candidate area based on the second image signal is higher than the accuracy of the lesion candidate area based on the first image signal, the first observed image performing second display control to display on the display a composite observation image obtained by combining second observation images based on the second image signal ;
When the second light emission pattern is a second B light emission pattern in which the emission spectra of the second illumination light are different in each of the second illumination periods, emitting a plurality of second illumination lights having mutually different emission spectra. In the second illumination light emission cycle, if the lesion candidate region is recognized from a plurality of second image signals corresponding to each second illumination light, when performing the second display control, the first Operation of an endoscope system comprising the step of synthesizing an observation image and a second observation image with the highest accuracy of the lesion candidate region from among the plurality of second observation images to generate the composite observation image. Method.

Images