JP6257475B2 - Scanning endoscope device - Google Patents

Description

  The present invention relates to a scanning endoscope apparatus, and more particularly to a scanning endoscope apparatus that acquires an image by scanning a subject.

  Conventionally, in an endoscope apparatus in the medical field, various techniques for reducing the diameter of an insertion portion to be inserted into a body cavity of a subject have been proposed in order to reduce the burden on the subject. ing. As one of the proposals, for example, there is a scanning endoscope apparatus that does not have a solid-state imaging device in a portion corresponding to the above-described insertion portion.

  As a method for adjusting the brightness in a scanning endoscope apparatus, Japanese Patent Laid-Open No. 2010-131112 discloses that circular scanning and spiral scanning are alternately performed one round at a time in order to quickly adjust an observation image in real time. Then, a method is proposed for adjusting the image of the observation image, such as brightness, based on the image signal of the spiral scan line for the previous round for pixels corresponding to the circular scan line for one round. ing. Specifically, the illumination light amount of the even line is increased or decreased at the scanning position where the luminance data outside the allowable range exceeding the upper threshold and the lower threshold is detected by the previous scanning of the odd line.

  Japanese Patent Laid-Open No. 2010-268961 discloses dimming or brightening at the timing when light is emitted toward a portion corresponding to an abnormal pixel in order to perform processing for abnormal images such as whiteout or blackout. Thus, a scanning endoscope apparatus that controls a light source has been proposed.

JP 2010-131112 A JP 2010-268961 A

  However, in the scanning endoscope apparatuses according to these proposals, for example, the amount of illumination light is adjusted to be within the allowable range at the scanning position where the luminance data outside the allowable range is detected. There is a problem that the dynamic range of the camera becomes narrow.

In addition, the obtained endoscopic image is different from a normal appearance, and there is a problem that the viewer feels uncomfortable.
For example, in the obtained endoscopic image, the part that should be dark before darkening is brightened, and the part that should be bright even when not whitening is darkened. The dynamic range of the mirror image is narrowed and the resulting image looks different from the actual appearance.

  In addition, some of the conventional image composition techniques generate images with a wide dynamic range from a plurality of images. However, since images are generated from a plurality of acquired images, Since the real-time property in the display of the endoscopic image is lost, such an image composition technique cannot be used.

  Therefore, an object of the present invention is to provide a scanning endoscope apparatus that can generate an endoscopic image with a wide dynamic range while ensuring the real-time property of the image.

According to one aspect of the present invention, a light source capable of emitting first illumination light and second illumination light having a light amount different from that of the first illumination light, the first illumination light, and the second illumination light. An optical fiber that guides the illumination light from the proximal end to the distal end, and the first illumination light and the second illumination light from the scanning start position in order to scan once within a predetermined range on the subject. A driving unit that drives the tip of the optical fiber to a scanning end position; and the first end of the optical fiber is driven from the scanning start position to the scanning end position for the one scan. A light source control unit that controls the light source so as to emit discretely and sequentially the illumination light and the second illumination light; and a first return light of the first illumination light from the subject in the one scan. Generating a first image signal composed of the first received light data generated from An image signal generation unit and a second image for generating a second image signal composed of second received light data generated from the second return light of the second illumination light from the subject in the one scan. A signal generation unit; and an image synthesis unit that generates a synthesized image using the first image signal and the second image signal , wherein the light source control unit is configured for each pixel of the first image signal. When the brightness value or the brightness of each predetermined area is less than or equal to the first threshold value, the amount of the first illumination light is increased so as to increase the brightness of the pixels or areas less than or equal to the first threshold value. When the luminance value for each pixel of the second image signal or the brightness for each predetermined area is greater than or equal to the second threshold, the brightness of the pixel or area greater than or equal to the second threshold is reduced. Controlling the light source so as to reduce the amount of the second illumination light; .
According to another aspect of the present invention, a light source capable of emitting first illumination light and second illumination light having a light amount different from that of the first illumination light, and the first illumination light An optical fiber for guiding the second illumination light from the proximal end to the distal end, and the first illumination light and the second illumination light are scanned in order to scan once for a predetermined range on the subject. A drive unit that drives the tip of the optical fiber from a start position to a scan end position, and the tip of the optical fiber is driven from the scan start position to the scan end position for the one scan. A light source control unit that controls the light source so as to sequentially and sequentially emit the first illumination light and the second illumination light; and a first of the first illumination light from the subject in the one scan. Of the return light and the second return light of the second illumination light. A first received light data selection unit for selecting first received light data and second received light data, a first image signal generating unit for generating a first image signal composed of the first received light data, and the second received light. A second image signal generation unit that generates a second image signal composed of data, and an image synthesis unit that generates a synthesized image using the first image signal and the second image signal, A light reception data selection unit stores information on the first light reception data and the second light reception data to be selected from the light reception signals of the first return light and the second return light, respectively. , The first light reception data and the second light reception data are selected.
Furthermore, according to the other aspect of this invention, the light source which can radiate | emit the 1st illumination light and the 2nd illumination light from which the said 1st illumination light differs in light quantity, The said 1st illumination light, An optical fiber for guiding the second illumination light from the proximal end to the distal end, and the first illumination light and the second illumination light are scanned in order to scan once for a predetermined range on the subject. A drive unit that drives the tip of the optical fiber from a start position to a scan end position, and the tip of the optical fiber is driven from the scan start position to the scan end position for the one scan. A light source control unit that controls the light source so as to sequentially and sequentially emit the first illumination light and the second illumination light; and a first of the first illumination light from the subject in the one scan. A first image signal composed of first received light data generated from the return light of A first image signal generation unit configured to generate a second image signal including second received light data generated from the second return light of the second illumination light from the subject in the one scan. A second image signal generation unit; and an image synthesis unit that generates a synthesized image by high dynamic range synthesis using the first image signal and the second image signal.

  ADVANTAGE OF THE INVENTION According to this invention, the scanning endoscope apparatus which can produce | generate an endoscopic image with a wide dynamic range, ensuring the real-time property of an image can be provided.

1 is a configuration diagram illustrating a configuration of an endoscope apparatus 1 according to an embodiment of the present invention. It is a figure which shows an example of the virtual XY plane set on the surface of a subject relating to the embodiment of the present invention. The temporal displacement of the illumination point coordinates of illumination light from point SA to point YMAX when illumination light is illuminated on a virtual XY plane as shown in FIG. 2 according to the embodiment of the present invention. It is a schematic diagram for demonstrating. When the illumination light is irradiated onto the virtual XY plane as shown in FIG. 2 according to the embodiment of the present invention, the temporal displacement of the illumination point coordinates of the illumination light from the point YMAX to the point SA is shown. It is a schematic diagram for demonstrating. It is a block diagram which shows the structure of the controller 25 in connection with embodiment of this invention. It is a figure for demonstrating the locus | trajectory of the spot light at the time of producing | generating the image of 1 frame regarding embodiment of this invention. It is a figure which shows the example of the data table of the light reception data of the under image combined with the HDR synthetic | combination part 49 in connection with embodiment of this invention, and an over image. It is a figure for demonstrating the image composition in the HDR synthetic | combination part 49 in connection with embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a configuration diagram showing a configuration of an endoscope apparatus 1 according to the present embodiment. As shown in FIG. 1, an endoscope apparatus 1 includes a scanning endoscope (hereinafter referred to as an endoscope) 2 inserted into a body cavity of a subject and a main body apparatus that can connect the endoscope 2. 3 is a scanning endoscope apparatus.

  The main unit 3 is connected to a monitor 4 for displaying an endoscopic image. An input device (not shown) for operating instructions and various settings is connected to the main body device 3, and the user instructs various operation such as display, photographing, and recording of endoscopic images from the input device. Can be given to the endoscope apparatus 1.

  The endoscope 2 includes an insertion portion 11 formed with an elongated shape and flexibility that can be inserted into a body cavity of a subject. The base end portion of the insertion portion 11 is configured to be detachable from the main body device 3.

  The insertion portion 11 includes an illumination fiber 12, a light receiving fiber 13, a condensing optical system 14, and an actuator 15. The condensing optical system 14 has lenses 14a and 14b.

The main device 3 includes a light source unit 21, a driver unit 22, a detection unit 23, a memory 24, and a controller 25.
The light source unit 21 is a light source that has three light sources 31a, 31b, and 31c and a multiplexer 32 and emits illumination light. As will be described later, the light source unit 21, under the control of the controller 25, is a discrete bright illumination spot light for obtaining a slightly overexposed bright image (hereinafter referred to as an overimage) and a darkly underexposed light. Discrete dark illumination spot light for obtaining an image (hereinafter referred to as an under image) is alternately output. That is, the light source unit 21 is capable of emitting illumination light that is spot light for an under image and illumination light that is spot light for obtaining an over image having a light amount different from that of the spot light for the under image. It is.

The driver unit 22 includes a signal generator 33, two digital / analog converters (hereinafter referred to as D / A devices) 34a and 34b, and two amplifiers 35a and 35b.
The detection unit 23 includes a duplexer 36, a detection unit 37, and an analog / digital conversion unit (hereinafter referred to as an A / D conversion unit) 38.

The memory 24 is a rewritable nonvolatile memory such as a flash memory.
The controller 25 includes a hardware circuit for various functions to be described later, a central processing unit (CPU), a ROM, and a RAM. The controller 25 implements various functions by reading and executing predetermined software programs stored in the ROM or the memory 24.
The various functions of the controller 25 are realized by a hardware circuit and a software program, but all may be realized by a hardware circuit.

The illumination fiber 12 and the light receiving fiber 13 are inserted into the insertion portion 11 from the proximal end portion to the distal end portion. That is, the illumination fiber 12 is an optical fiber that is inserted into the insertion portion 11 of the endoscope 2 and guides illumination light from the proximal end to the distal end.
The illumination fiber 12 is a light guide member that guides the illumination light supplied from the light source unit 21 of the main body device 3 to the condensing optical system 14.

The light receiving fiber 13 is a light guide member that receives return light from the subject and guides it to the detection unit 23 of the main body device 3. That is, the light receiving fiber 13 is an optical fiber that is inserted into the insertion portion 11 of the endoscope 2 and guides return light from the subject from the distal end to the proximal end.
The end including the light incident surface of the illumination fiber 12 is disposed at the output end of the multiplexer 32 provided inside the main body device 3. Further, the end portion 12 a including the light emission surface of the illumination fiber 12 is disposed in the vicinity of the light incident surface of the lens 14 a provided at the distal end portion of the insertion portion 11.

  The end portion 13 a including the light incident surface of the light receiving fiber 13 is disposed and fixed around the light emitting surface of the lens 14 b at the distal end surface of the distal end portion of the insertion portion 11. Further, the end including the light emitting surface of the light receiving fiber 13 is connected to the input end of the duplexer 36 provided inside the main body device 3.

  The condensing optical system 14 includes a lens 14a on which illumination light that has passed through the light exit surface of the illumination fiber 12 is incident, and a lens 14b that emits illumination light that has passed through the lens 14a to a subject. .

  An actuator 15 is attached to the middle portion of the illumination fiber 12 on the distal end side of the insertion portion 11. The actuator 15 is driven based on a drive signal supplied from the driver unit 22 of the main body device 3.

  Hereinafter, an imaginary plane perpendicular to the insertion axis (or the optical axis of the condensing optical system 14) corresponding to the longitudinal axis of the insertion portion 11 is defined as an XY plane as shown in FIG. The description will be given by taking the case of setting as an example.

FIG. 2 is a diagram illustrating an example of a virtual XY plane set on the surface of the subject.
Specifically, when the point SA on the XY plane in FIG. 2 is virtually set as a position where the insertion axis of the insertion portion 11 exists in the direction corresponding to the front side from the front side of the paper, It is shown as an intersection with the page. Further, the X-axis direction in the XY plane of FIG. 2 is set as a direction from the left side to the right side of the drawing. Further, the Y-axis direction in the XY plane of FIG. 2 is set as a direction from the lower side to the upper side of the drawing. Further, the X axis and the Y axis constituting the XY plane in FIG. 2 intersect at the point SA. Point SA is an irradiation point of illumination light when the end 12a of the illumination fiber 12 is at the center position O when the end 12a is stationary.

  Returning to FIG. 1, the actuator 15 swings the end including the light emitting surface of the illumination fiber 12 in the X-axis direction based on the first drive signal supplied from the driver unit 22 of the main body device 3. Based on the X-axis actuator 15X that operates and the second drive signal supplied from the driver unit 22 of the main body device 3, the end including the light emitting surface of the illumination fiber 12 is swung in the Y-axis direction. And an Y-axis actuator 15Y that operates in the same manner.

  Each of the X-axis actuator 15X and the Y-axis actuator 15Y includes, for example, one or more piezoelectric elements. Here, the actuators 15X and 15Y are actuators using a ferroelectric. The end 12a including the light exit surface of the illumination fiber 12 is swung in a spiral shape around the point SA in accordance with the operation of the actuators 15X and 15Y as described above. That is, the actuator 15 is a dielectric that drives the end 12a that is the tip of the illumination fiber 12 so that the illumination light emitted from the illumination fiber 12 scans the subject when a drive voltage is applied. Actuator. The actuator 15 has two actuators 15X and 15Y for moving the tip of the illumination fiber 12 in two directions orthogonal to each other.

  The X-axis actuator 15X is formed of, for example, a piezoelectric element that has been previously polarized so that the polarization direction coincides with the negative direction of the X-axis (the direction from the right to the left in FIG. 2). When a positive voltage is applied according to the first drive signal output from the driver unit 22 (when the direction of the electric field generated with the supply of the drive signal is forward with respect to the polarization direction) And) contracts along the Z-axis direction (normal direction of the paper surface), and when a negative voltage is applied (the direction of the electric field generated with the supply of the drive signal is opposite to the polarization direction) It is configured to extend along the Z-axis direction.

  The Y-axis actuator 15Y is formed of, for example, a piezoelectric element that has been previously polarized so that the polarization direction coincides with the negative direction of the Y-axis (the direction from the top to the bottom of FIG. 2). In response to the second drive signal output from the driver unit 22, it contracts along the Z-axis direction when a positive voltage is applied, and in the Z-axis direction when a negative voltage is applied. It is comprised so that it may extend along.

  The light source 31a of the light source unit 21 includes, for example, a laser light source, and emits light in the red wavelength band (hereinafter also referred to as R light) to the multiplexer 32 when light is emitted under the control of the controller 25. It is configured.

  The light source 31b includes a laser light source, for example, and is configured to emit light in a green wavelength band (hereinafter also referred to as G light) to the multiplexer 32 when light is emitted under the control of the controller 25. Yes.

  The light source 31c includes a laser light source, for example, and is configured to emit blue wavelength band light (hereinafter also referred to as B light) to the multiplexer 32 when light is emitted under the control of the controller 25. Yes.

  The multiplexer 32 multiplexes the R light emitted from the light source 31a, the G light emitted from the light source 31b, and the B light emitted from the light source 31c onto the light incident surface of the illumination fiber 12. It is configured to supply.

  Based on the control of the controller 25, the signal generator 33 of the driver unit 22 uses, for example, the following formula as a first drive control signal for swinging the end 12a including the light emitting surface of the illumination fiber 12 in the X-axis direction. A signal having a waveform as shown by (1) is generated and output to the D / A converter 34a. In the following formula (1), X (t) represents a signal level at time t, a represents an amplitude value independent of time t, and G (t) is used for modulation of a sine wave sin (2πft). It shall represent a predetermined function.

X (t) = a × G (t) × sin (2πft) (1)
Further, the signal generator 33 is, for example, the following formula (2) as a second drive control signal for swinging the end 12a including the light emitting surface of the illumination fiber 12 in the Y-axis direction based on the control of the controller 25. ) Is generated and output to the D / A converter 34b. In the following formula (2), Y (t) represents a signal level at time t, b represents an amplitude value independent of time t, and G (t) is used to modulate a sine wave sin (2πft + φ). It represents a predetermined function to be used, and φ represents a phase.

Y (t) = b × G (t) × sin (2πft + φ) (2)
The D / A converter 34a is configured to convert the digital first drive control signal output from the signal generator 33 into a first drive signal that is an analog voltage signal and output the first drive signal to the amplifier 35a. Yes.

  The D / A converter 34b is configured to convert the digital second drive control signal output from the signal generator 33 into a second drive signal that is an analog voltage signal and output the second drive signal to the amplifier 35b. Yes.

The amplifier 35a is configured to amplify the first drive signal output from the D / A converter 34a and output the amplified signal to the X-axis actuator 15X.
The amplifier 35b is configured to amplify the second drive signal output from the D / A converter 34b and output it to the Y-axis actuator 15Y.

  Here, for example, in the above formulas (1) and (2), a = b and φ = π / 2 are set, the first drive signal is supplied to the X-axis actuator 15X, and the second The drive signal is supplied to the Y-axis actuator 15Y.

  FIG. 3 is a schematic diagram for explaining temporal displacement of illumination point coordinates of illumination light from point SA to point YMAX when illumination light is illuminated on a virtual XY plane as shown in FIG. FIG. FIG. 4 is a schematic diagram for explaining temporal displacement of the irradiation point coordinates of the illumination light from the point YMAX to the point SA when the illumination light is irradiated on the virtual XY plane as shown in FIG. FIG. In FIGS. 3 and 4, the interval of the locus of the illumination light is shown enlarged for easy viewing.

  Specifically, at a certain time T1, illumination light is irradiated to a position corresponding to the point SA on the surface of the subject. Thereafter, as the amplitude values of the first and second drive signals increase from time T1 to time T2, as shown in FIG. 3, the illumination light irradiation coordinates within the predetermined range on the surface of the subject are point SA. It is displaced so as to draw a first spiral trajectory outward as a starting point, and when time T2 is reached, it is the outermost point from the point SA of the illumination light irradiation coordinates within a predetermined range on the surface of the subject. Illumination light is irradiated to the point YMAX. Between the time T1 and the time T2, the illumination light is irradiated to the point XMAX that is the outermost point from the point SA in the negative direction of the X axis. From the time T1 to the time T2, the illumination light irradiation coordinates move spirally as indicated by the scanning line SL1 in FIG.

Then, as the amplitude values of the first and second drive signals decrease from time T2 to time T3, as shown in FIG. 4, the illumination light irradiation coordinates on the surface of the subject are moved inward starting from the point YMAX. When the time T3 is reached, illumination light is applied to the point SA on the surface of the subject. From the time T2 to the time T3, the illumination light irradiation coordinates move in a spiral shape as indicated by the scanning line SL2 in FIG.
That is, the predetermined range in which each one scan is performed is substantially circular, and each one scan is performed in a spiral shape with respect to the predetermined range.

  Based on the first and second drive signals supplied from the driver unit 22, the actuator 15 has a spiral shape in which the irradiation position of the illumination light applied to the subject through the condensing optical system 14 is illustrated in FIGS. 3 and 4. The end portion including the light emitting surface of the illumination fiber 12 can be swung so as to draw a locus corresponding to the scanning pattern.

Here, a plurality of spot lights, which are illumination lights, are irradiated along the scanning lines SL1, SL2, but spot light for an under image and spot light for an over image are emitted alternately.
That is, the controller 25 and the driver unit 22 cause the illumination light, which is a spot light for an under image, and the spot light for obtaining an over image to scan once in a spiral shape over a predetermined range on the subject. A drive unit is configured to drive the end 12a, which is the tip of the illumination fiber 12, from a predetermined scan start position to a predetermined scan end position.

  The controller 25 controls the light source unit 21 in synchronization with the drive of the driver unit 22 to generate an over image and an under image that is different in exposure, that is, brightness, from the over image. The light source unit 21 outputs a plurality of discrete spot lights so that a plurality of spot lights for obtaining an image and a plurality of spot lights for obtaining an under image are included.

  Then, the controller 25 drives the spot light for the under image while the end 12a, which is the tip of the illumination fiber 12, is driven from a predetermined scan start position to a predetermined scan end position for one scan. And a light source control unit that controls the light source unit 21 so as to discretely and sequentially emit spot light for obtaining an over image.

  Note that a plurality of spot lights irradiated in a spiral form a substantially circular illumination area. However, the amount of light at the periphery of the circular illumination area is reduced compared to the amount of light at the center, and the brightness of the periphery is reduced. The value drops. Therefore, the controller 25 controls the light source unit 21 so that the amount of the spot light emitted to the peripheral part of the illumination area is increased in accordance with the position in the peripheral part to perform shading correction.

  For this reason, the memory 24 stores luminance shading information generated from the center (position of the point SA) of the substantially circular illumination area to the periphery. Based on the luminance shading information, the controller 25 controls the light source unit 21 so as to change the light quantity of the spot light for the under image and the spot light for the over image.

  At this time, instead of controlling both the spot light for the under image and the spot light for the over image, either the spot light for the under image or the spot light for the over image may be used. .

  That is, the memory 24 constitutes a storage unit that stores luminance shading information generated from the center point SA to the peripheral part of a predetermined range in one scan. Based on the luminance shading information stored in the memory 24, the controller 25 calculates at least one of the light amount of the spot light for the under image and the light amount of the spot light for the over image emitted from the light source unit 21. Control.

The light emitting surface of the light receiving fiber 13 is disposed so as to emit the return light emitted from the light emitting surface of the light receiving fiber 13 to the duplexer 36.
The duplexer 36 of the detection unit 23 includes a dichroic mirror and the like, and returns light emitted from the light emitting surface of the light receiving fiber 13 for each of R (red), G (green), and B (blue) color components. It is configured so as to be separated into the light and emitted to the detection unit 37.

The detection unit 37 detects the intensities of the R light, the G light, and the B light output from the duplexer 36, generates an analog signal corresponding to the detected intensity of each light, and generates an A / D conversion unit 38. Output to.
The A / D conversion unit 38 converts the analog signal of each spot light output from the detection unit 37 into a digital signal, that is, luminance value data, and outputs the digital signal to the controller 25.

Then, the controller 25 alternately repeats the scan from time T1 to time T2 and the scan from time T2 to time T3, and generates one frame image for each scan from time T1 to time T2. One frame image is generated for each scanning from time T2 to time T3.
The memory 24 stores programs and the like for realizing various functions of the main device 3.

  The controller 25 reads a control program for light source control and actuator drive control stored in the ROM or the memory 24, and controls the light source unit 21 and the driver unit 22 based on the read control program.

  The controller 25 generates an image for one frame on the basis of the R signal, the G signal, and the B signal output from the detection unit 23 during a period corresponding to the time T1 to the time T2. Use one image. Similarly, the controller 25 generates an image for one frame based on the R signal, the G signal, and the B signal output from the detection unit 23 during a period corresponding to the time T2 to the time T3. Two images are used.

Next, the configuration of the controller 25 will be described.
(Configuration of controller)
FIG. 5 is a block diagram showing the configuration of the controller 25. FIG. 5 shows only the components related to the present embodiment.

  The controller 25 includes a spot light determination unit 41, a look-up table (hereinafter abbreviated as LUT) 42, an under image storage unit 43, an over image storage unit 44, an under image brightness measurement unit 45, and an over image brightness. A height measurement unit 46, two raster conversion units 47 and 48, a high dynamic range synthesis unit (hereinafter referred to as HDR synthesis unit) 49, a brightness determination unit 50, and a light amount control unit 51.

  Each received light data that is a digital signal output from the A / D conversion unit 38 of the detection unit 23 is input to the spot light determination unit 41. The spot light discriminating unit 41 is a circuit that discriminates, selects and outputs whether or not the received light reception data is image data to be used for generating an endoscopic image with reference to the LUT 42.

  As will be described later, the HDR combining unit 49 combines the under image and the over image. The light receiving data used as the image data of the under image and the light receiving data used as the image data of the over image are discriminated. Data is stored in the LUT 42.

  The LUT 42 stores information (hereinafter also referred to as selection information) of spot light to be used, that is, to be selected for generating an endoscopic image among a plurality of spot lights emitted on the XY plane. It is. The selection information includes information indicating the position on the XY plane or the order of emission, and whether the spot light at each position or order is an over image spot light or an under image spot light (hereinafter also referred to as light / dark information).

  For example, when the order of emission of each spot light formed along the scanning line SL1 or SL2 is predetermined, the order information of the spot light to be used, that is, to be used for generating an endoscope image, Brightness / darkness information is stored in the LUT 42.

  As shown in FIGS. 3 and 4, each discrete spot light is irradiated to a predetermined position on the XY plane along a scanning line SL1 or SL2 having a predetermined spiral shape. Therefore, the spot light discriminating unit 41 refers to the LUT 42 and determines whether the light reception data received from the detection unit 23 is the light reception data of the spot light to be used for generating the endoscopic image, and the light reception data. Is received light data of bright spot light (ie spot light for over image) or dark spot light (ie spot light for under image).

  That is, the spot light discriminating unit 41 uses the spot light to be used from the received light signals of the spot light return light for the under image and the spot light return light for the over image received in one scan. A received light data selection unit for selecting the received light data. The spot light discriminating unit 41, which is a light reception data selection unit, selects the light reception data to be selected from the light reception signals of the spot light return light for the under image and the spot light return light for the over image. By referring to the LUT 42 storing the information, spot light reception data for the under image and spot light reception data for the over image are selected.

  When the received light reception data is the light reception data of the spot light to be used for generating the endoscopic image and the light reception data of the spot light for the under image, the spot light determination unit 41 When the received light reception data output to the storage unit 43 is the light reception data of the spot light to be used for generating the endoscope image and the light reception data of the spot light for the over image, the over image The data is output to the storage unit 44.

  The under image storage unit 43 is a memory circuit that stores light reception data of an under image. The over image storage unit 44 is a memory circuit that stores light reception data of an over image. That is, the under image storage unit 43 and the over image storage unit 44 constitute a storage unit that stores the image signal of the under image and the image signal of the over image generated by one scan.

  Image signal generation by the spot light discriminating unit 41 and the under image storage unit 43 for generating an image signal of an under image composed of light reception data generated from spot light return light for an under image from a subject in one scan Parts. Image signal generation by the spot light discriminating unit 41 and the over image storage unit 44 for generating an image signal of an over image composed of received light data generated from spot light return light for an over image from a subject in one scan Parts.

  The under image brightness measurement unit 45 is a circuit for measuring the brightness of the under image from the light reception data of the under image stored in the under image storage unit 43. Here, the under image brightness measurement unit 45 calculates the brightness of the under image from the luminance value of each light reception data. The brightness is calculated and measured and output to the brightness determination unit 50.

Specifically, the under image brightness measurement unit 45 measures the brightness of the entire under image from all pixel values of one frame image stored in the under image storage unit 43.
The over image brightness measurement unit 46 is a circuit for measuring the brightness of the over image from the light reception data of the over image stored in the over image storage unit 44. Here, the over image brightness is calculated from the luminance value of each light reception data. The brightness is calculated and measured and output to the brightness determination unit 50.

Specifically, the over image brightness measurement unit 46 measures the brightness of the entire over image from all pixel values of one frame image stored in the over image storage unit 44.
That is, for each frame shown in FIGS. 3 and 4, the under image storage unit 43 and the over image storage unit 44 store the image data of the under image and the over image constituting each frame image. The under image brightness measurement unit 45 and the over image brightness measurement unit 46 measure the light amounts of the under image and the over image constituting each frame image for each frame, and output them to the brightness determination unit 50.

  Further, the under image brightness measurement unit 45 determines whether each light reception data indicates a luminance value equal to or lower than a predetermined threshold TH1, and when the light reception data is equal to or lower than the predetermined threshold TH1, The position or order and the data du indicating that it is equal to or less than the predetermined threshold TH1 are output to the brightness determination unit 50.

  Similarly, the over-image brightness measuring unit 46 determines whether or not each light reception data indicates a luminance value equal to or higher than a predetermined threshold value TH2, and when the light reception data is equal to or higher than the predetermined threshold value TH2, the light reception data Are output to the brightness determination unit 50, and the data do indicating that it is equal to or greater than a predetermined threshold TH2.

The raster conversion unit 47 is a circuit that raster-converts the light reception data of the under image whose light amount has been measured by the under image brightness measurement unit 45 and outputs it to the HDR synthesis unit 49.
The raster conversion unit 48 is a circuit that raster-converts the light reception data of the over image whose light amount is measured by the over image brightness measurement unit 46 and outputs the data to the HDR synthesis unit 49.

  The raster conversion units 47 and 48 perform raster scan scanning on the monitor 4 with respect to the light reception data of the under image stored in the under image storage unit 43 and the light reception data of the over image stored in the over image storage unit 44, respectively. This is a circuit for performing data conversion into raster scan data in order to display an endoscopic image.

  The HDR synthesizing unit 49 is a circuit that performs high dynamic range synthesis (hereinafter abbreviated as HDR synthesis) using the two images from the raster converting units 47 and 48 to generate a synthesized image.

  The HDR synthesizing unit 49 performs a process of generating a wide dynamic range image by combining a plurality of image data with different exposures. Here, two images of an under image and an over image composed of the received light data selected by the spot light determination unit 41 are combined to generate one image.

  That is, the raster conversion units 47 and 48 constitute a data conversion unit that performs data conversion of an under-image image signal and an over-image image signal generated by one scanning into a raster scan format. The HDR synthesizing unit 49 constitutes an image synthesizing unit that generates a synthesized image by high dynamic synthesis using the image signal of the under image and the image signal of the over image that have been subjected to data conversion.

  Here, the HDR conversion is performed after raster conversion of the under image and the over image. However, the raster conversion may be performed after the HDR combination of the under image and the over image.

  The brightness determination unit 50 determines the brightness of the under image in one frame obtained by the under image brightness measurement unit 45 and the brightness of the over image in one frame obtained by the over image brightness measurement unit 46. Then, the brightness data obtained by the determination is output to the light quantity control unit 51.

  The light quantity control unit 51 is a processing unit that controls the light source unit 21 based on the brightness data from the brightness determination unit 50 so that the over image and the under image become images of a predetermined brightness. The predetermined brightness for the over image is, for example, the brightness corresponding to the exposure amount that is one step lower than the exposure amount for the under exposure of the under image, and the predetermined brightness for the over image is, for example, over The brightness of the image corresponds to an exposure amount that is one step higher than the exposure amount with proper exposure.

  As described above, the light amount control unit 51 uses the under image emitted from the light source unit 21 so that the brightness of the image signal of the under image and the brightness of the image signal of the over image become predetermined brightness, respectively. The amount of spot light and the amount of spot light for over-image are controlled.

  In addition, when the brightness determination unit 50 receives data du indicating that it is less than or equal to the predetermined threshold TH1 from the under-image brightness measurement unit 45, the information on the position or order of the received light data that is less than or equal to the predetermined threshold TH1. And output to the light quantity control unit 51.

  The light quantity control unit 51 increases the light quantity of the spot light (the spot light corresponding to the under image) corresponding to the received light reception data that is equal to or less than the received predetermined threshold TH1 by a predetermined amount U. The unit 21 is controlled.

  Similarly, when the brightness determination unit 50 receives from the over-image brightness measurement unit 46 data do indicating that it is greater than or equal to the predetermined threshold TH2, information on the position or order of the received light data that is greater than or equal to the predetermined threshold TH2. Is output to the light quantity control unit 51.

  The light amount control unit 51 reduces the light amount of the spot light in the position or order corresponding to the received light reception data that is equal to or greater than the predetermined threshold TH2 (spot light corresponding to the over image) by a predetermined amount D. The unit 21 is controlled.

  As described above, when the luminance value for each pixel of the image signal of the under image is equal to or lower than the predetermined threshold value TH1, the light amount control unit 51 converts the under image to increase the brightness of the pixels equal to or lower than the predetermined threshold value TH1. When the light intensity of the corresponding spot light is increased and the luminance value for each pixel of the image signal of the over image is equal to or higher than the predetermined threshold TH2, the over image is reduced so as to reduce the brightness of the pixel that is equal to or higher than the predetermined threshold TH2. The light source unit 21 is controlled so as to reduce the amount of the corresponding spot light.

Here, the comparison with the threshold values TH1 and TH2 is performed for each pixel. However, the under image and the over image are divided into a plurality of predetermined regions, and the brightness of each predetermined region of the image signal of the under image is determined. When the brightness is less than or equal to the predetermined threshold TH1, the amount of spot light corresponding to the under image is increased so as to increase the brightness of the area below the predetermined threshold TH1, and the brightness for each predetermined area of the image signal of the over image Is equal to or greater than a predetermined threshold value TH2, the light amount control unit 51 controls the light source unit 21 so as to decrease the light amount of the spot light corresponding to the over image so as to reduce the brightness of the region equal to or greater than the predetermined threshold value TH2. You may make it do.
(Function)
Next, the operation of the controller 25 will be described.

  When the endoscope 2 is connected to the main body device 3 and the subject is observed by the endoscope 2, the controller 25 controls the light source unit 21 and the driver unit 22, and based on the image signal from the detection unit 23. An endoscopic image is generated and output to the monitor 4 to display an image of the subject, that is, an endoscopic image.

First, the locus of the spot light will be described. FIG. 6 is a diagram for explaining the locus of the spot light when generating an image of one frame.
FIG. 6 corresponds to FIG. 3 and shows a state in which a plurality of spot lights are irradiated onto the subject by being scanned outwardly from the center point SA to the point YMAX. A substantially circular range IB indicated by a dotted line in FIG. 6 indicates an irradiation range of a plurality of spot lights irradiated in a spiral shape around the point SA toward the outside.
In FIG. 6, only a part of the locus of the spot light is shown, and only a locus near the center of the point SA and a portion of the locus near the upper left outer peripheral part are schematically shown.

  FIG. 6 shows a plurality of spot lights irradiated to four regions at the center of the eight regions 61 by enlarging a part of the spot light irradiated spirally from the point SA. In FIG. 6, a region IBr indicated by a two-dot chain line in a substantially circular range IB is enlarged, a white circle indicates a spot light SPo for an over image, and a black circle indicates a spot light SPu for an under image.

  FIG. 6 shows an example in which spot light SPo for the over image and spot light SPu for the under image are emitted alternately along the scanning line SL1. Each region 61 includes a plurality of spot lights SPo and a plurality of spot lights SPu.

  As described above, the spot light to be used for generating the endoscopic image is determined in advance in the LUT 42. Here, each region 61 corresponds to one pixel constituting the generated endoscopic image. Therefore, each area 61 includes a plurality of spot lights SPo and a plurality of spot lights SPu. In each area 61, the spot light for the under image and the spot light for the over image used for image composition are determined in advance. It has been.

  That is, for each region 61, spot light for an under image and spot light for an over image used for image synthesis are determined in advance from a plurality of spot lights SPo and a plurality of spot lights SPu.

  For example, in the lower right region 61k in FIG. 6, the spot light SPok and the spot light SPuK in the region 61k are determined as the spot light for the over image and the spot light for the under image, respectively.

  Then, the received light data of the over image and the under image is selected from the return light for the spot light determined in advance for each region 61, that is, for each pixel, and stored in the under image storage unit 43 and the over image storage unit 44, respectively. As a result, two images, an under image and an over image, are acquired from a plurality of spot lights irradiated in a spiral shape around the point SA.

  An over image and an under image are generated by one spiral scan shown in FIG. Similarly, an over image and an under image are generated by one spiral scan shown in FIG.

The generated light reception data of the under image and the over image is output to the raster conversion units 47 and 48, respectively.
The raster conversion unit 47 converts the light reception data of the spiral under image into the light reception data of the rectangular under image for raster scan scanning, and the raster conversion unit 48 converts the light reception data of the spiral over image to the raster. Converted into light reception data of a rectangular overimage for scan scanning.

  FIG. 7 is a diagram illustrating an example of a data table of light reception data of the under image and the over image synthesized by the HDR synthesizing unit 49. FIG. 7 shows a data table of received light data of the under image and the over image after the raster conversion by the raster conversion units 47 and 48.

  X and Y in FIG. 7 indicate pixel positions on the XY plane of the image after raster conversion, and light reception data of the under image after raster conversion and light reception data of the over image after raster conversion are stored in each pixel. The

  The HDR synthesizing unit 49 uses the light reception data in the data table of FIG. 7 to synthesize two images, an over image and an under image, to generate an image having a wide dynamic range with little whiteout and blackout. And output to the monitor 4.

  Further, the light amount control unit 51, for example, brightness data of the under image from the brightness determination unit 50 so that the under image has a brightness corresponding to the exposure amount that is one step lower than the exposure amount of appropriate exposure. Based on the above, the emitted light amount of the spot light for the under image is controlled. Similarly, the light amount control unit 51 uses the brightness data of the over image from the brightness determination unit 50 so that the over image has a brightness corresponding to the exposure amount that is one step higher than the exposure amount of appropriate exposure. Based on this, the emitted light quantity of the spot light for the over image is controlled.

  That is, the brightness determination unit 50 determines the brightness of the under image and the over image from the respective light amounts of the under image and the over image for each frame, and the light amount control unit 51 is determined by the brightness determination unit 50. Based on the brightness data of each of the under image and the over image, the emitted light amount of the spot light for each of the under image and the over image is controlled.

  As a result, the under image stored in the under image storage unit 43 becomes an image with a predetermined brightness (for example, an image with an exposure amount that is one step lower than the exposure amount with proper exposure), and is stored in the over image storage unit 44. The light amount control unit 51 controls the light source unit 21 so that the stored overimage is an image with a predetermined brightness (for example, an image with an exposure amount that is one step over the exposure amount with proper exposure). .

  Therefore, since the HDR synthesizing unit 49 performs high dynamic range synthesis from an under image with appropriate brightness and an over image with appropriate brightness, the obtained endoscopic image becomes an image with a wide dynamic range.

  FIG. 8 is a diagram for explaining image composition in the HDR composition unit 49. Based on the light reception data of the under image and the over image shown in FIG. 7, the HDR synthesizing unit 49 generates a synthesized image. The HDR synthesizing unit 49 synthesizes two images with different exposures, that is, an under image and an over image, thereby generating an image having a wide dynamic range with little whiteout or blackout as a synthesized image.

  The under image brightness measuring unit 45 outputs the information on the position or order of the received light data to the light amount control unit 51 for the received light data below the predetermined threshold TH1 in the under image. The light source unit 21 is controlled so as to increase only the light amount of the spot light (spot light corresponding to the under image) corresponding to the received light reception data that is equal to or less than the received predetermined threshold TH1 by a predetermined amount U. To do.

  Similarly, the over-image brightness measuring unit 46 outputs the information on the position or order of the received light data to the light amount control unit 51 for the received light data of the predetermined threshold TH2 or more in the over image, and the light amount control unit 51. The light source unit 21 is controlled so as to decrease only the light amount of the received spot light corresponding to the received light data that is equal to or greater than the received predetermined threshold TH2 by the predetermined amount D (spot light corresponding to the over image). Control.

  As a result, there are fewer underexposed pixels or areas in the under image, and fewer overexposed pixels or areas in the over image, so the entire composite image that combines the under image and the over image Even in this case, there is less white-out and black-out.

  As described above, according to the above-described embodiment, a plurality of images with different exposures in the above example are generated and combined in one scan, so that the real-time property of the image is improved. It is possible to provide a scanning endoscope apparatus that can generate an endoscopic image with a wide dynamic range while ensuring.

  In particular, instead of synthesizing images after acquiring two frame images as in the prior art, two image signals of under-exposure and over-image that are different from each other are acquired in one scan. Since image synthesis is performed, the real-time property of the endoscopic image can be ensured.

  In addition, the light intensity is adjusted by measuring the brightness of the under image and the brightness of the over image so that the under image and the over image each have appropriate brightness, that is, appropriate predetermined exposure. Becomes an image having an appropriate dynamic range.

  Furthermore, when there are received light data that is greater than or less than a predetermined threshold in a plurality of acquired images with different exposures, the amount of illumination spot light is controlled so that each image is underexposed or overexposed. Since pixels or regions are reduced, whiteout and blackout can be reduced even in the entire composite image.

  In the example described above, the HDR synthesizing unit 49 synthesizes images using two images with different exposures, but may synthesize three or more images with different exposures. In that case, the LUT 42 stores information on three or more spot lights to be used in image composition for each pixel. Then, the light amount is measured for each of the plurality of images having different exposures, the light amount control is performed, each image is raster-converted, and the HDR combining unit 49 combines the images.

  Furthermore, in the above-described example, the spot light for the under image and the spot light for the over image are alternately emitted from the light source unit 21 in each scan, but are emitted in other emission orders that are not alternating. It may be.

  The present invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Scanning endoscope apparatus, 2 Endoscope, 3 Main body apparatus, 4 Monitor, 11 Insertion part, 12 Illumination fiber, 13 Light reception fiber, 14 Condensing optical system, 14a, 14b Lens, 15, 15X, 15Y Actuator, 21 Light source unit, 22 Driver unit, 23 Detection unit, 24 Memory, 25 Controller, 31a, 31b, 31c Light source, 32 Multiplexer, 33 Signal generator, 34a, 34b D / A converter, 35a, 35b Amplifier , 36 demultiplexer, 37 detection unit, 38 A / D conversion unit, 41 spot light discrimination unit, 43 under image storage unit, 44 over image storage unit, 45 under image brightness measurement unit, 46 over image brightness measurement unit 47, 48 Raster conversion unit, 49 HDR synthesis unit, 50 Brightness determination unit, 51 Light quantity control unit, 61, 61k .

Claims (12)

A light source capable of emitting a first illumination light and a second illumination light having a light quantity different from that of the first illumination light;
An optical fiber for guiding the first illumination light and the second illumination light from the proximal end to the distal end;
A driving unit that drives the tip of the optical fiber from a scanning start position to a scanning end position in order to perform one scan of the first illumination light and the second illumination light on a predetermined range on the subject;
While the tip of the optical fiber is driven from the scan start position to the scan end position for the one scan, the first illumination light and the second illumination light are discretely and sequentially emitted. A light source controller for controlling the light source,
A first image signal generation unit configured to generate a first image signal including first received light data generated from the first return light of the first illumination light from the subject in the one-time scanning;
A second image signal generation unit that generates a second image signal composed of second received light data generated from the second return light of the second illumination light from the subject in the one-time scanning;
An image composition unit that generates a composite image using the first image signal and the second image signal;
With
When the luminance value for each pixel of the first image signal or the brightness for each predetermined area is equal to or less than a first threshold, the light source control unit determines the brightness of the pixel or area equal to or less than the first threshold. When the light intensity of the first illumination light is increased so as to increase the brightness value of each pixel of the second image signal or the brightness of each predetermined area is equal to or greater than a second threshold, Controlling the light source so as to reduce the amount of the second illumination light so as to reduce the brightness of a pixel or region that is equal to or greater than a threshold value of 2.
A scanning endoscope apparatus characterized by the above .   A light source capable of emitting a first illumination light and a second illumination light having a light quantity different from that of the first illumination light;
  An optical fiber for guiding the first illumination light and the second illumination light from the proximal end to the distal end;
  A driving unit that drives the tip of the optical fiber from a scanning start position to a scanning end position in order to perform one scan of the first illumination light and the second illumination light on a predetermined range on the subject;
  While the tip of the optical fiber is driven from the scan start position to the scan end position for the one scan, the first illumination light and the second illumination light are discretely and sequentially emitted. A light source controller for controlling the light source,
  In the one-time scanning, first received light data and first received light from first received light signals of the first illumination light and second returned light of the second illumination light from the subject. A light reception data selection unit for selecting the light reception data of 2;
  A first image signal generation unit for generating a first image signal composed of the first light reception data;
  A second image signal generation unit for generating a second image signal composed of the second light reception data;
  An image compositing unit that generates a composite image using the first image signal and the second image signal;
  The received light data selection unit stores information on the first received light data and the second received light data to be selected from the received light signals of the first return light and the second return light, respectively. By referring to the table, the first received light data and the second received light data are selected.
  A scanning endoscope apparatus characterized by the above.   A light source capable of emitting a first illumination light and a second illumination light having a light quantity different from that of the first illumination light;
  An optical fiber for guiding the first illumination light and the second illumination light from the proximal end to the distal end;
  A driving unit that drives the tip of the optical fiber from a scanning start position to a scanning end position in order to perform one scan of the first illumination light and the second illumination light on a predetermined range on the subject;
  While the tip of the optical fiber is driven from the scan start position to the scan end position for the one scan, the first illumination light and the second illumination light are discretely and sequentially emitted. A light source controller for controlling the light source,
  A first image signal generation unit configured to generate a first image signal including first received light data generated from the first return light of the first illumination light from the subject in the one-time scanning;
  A second image signal generation unit that generates a second image signal composed of second received light data generated from the second return light of the second illumination light from the subject in the one-time scanning;
  An image synthesis unit that generates a synthesized image by high dynamic range synthesis using the first image signal and the second image signal;
  A scanning endoscope apparatus comprising: The light source control unit emits the first light emitted from the light source so that the brightness of the first image signal and the brightness of the second image signal are the first brightness and the second brightness, respectively. The scanning endoscope apparatus according to any one of claims 1 to 3, wherein the light quantity of the first illumination light and the light quantity of the second illumination light are controlled. The first illumination light has a smaller amount of light than the second illumination light,
When the luminance value for each pixel of the first image signal or the brightness for each predetermined area is equal to or less than a first threshold, the light source control unit determines the brightness of the pixel or area equal to or less than the first threshold. When the light intensity of the first illumination light is increased so as to increase the brightness value of each pixel of the second image signal or the brightness of each predetermined area is equal to or greater than a second threshold, so as to reduce the second amount of illumination light so as to reduce the brightness of the second threshold or more pixels or areas, in any one of claims 2-4, wherein the controller controls the light source The scanning endoscope apparatus described. A light reception data selection unit for selecting the first light reception data and the second light reception data from the respective light reception signals of the first return light and the second return light received in the one scan; The scanning endoscope apparatus according to claim 1, wherein the scanning endoscope apparatus is provided. The received light data selection unit stores information on the first received light data and the second received light data to be selected from the received light signals of the first return light and the second return light, respectively. The scanning endoscope apparatus according to claim 6 , wherein the first light reception data and the second light reception data are selected by referring to a table. 8. The scanning according to claim 6 , wherein each pixel of the synthesized image is synthesized using the first received light data and the second received light data selected by the received light data selection unit. Mold endoscope device. A data converter that performs data conversion of the first image signal and the second image signal generated by the one-time scanning into a raster scan format;
9. The scanning endoscope according to claim 1 , wherein the image synthesis unit synthesizes the first image signal and the second image signal that have been converted into data. 10. apparatus. A storage unit for storing the first image signal and the second image signal generated by the one-time scanning;
The scanning type internal view according to claim 9 , wherein the data conversion unit performs the data conversion on the first image signal and the second image signal stored in the storage unit. Mirror device. A storage unit that stores luminance shading information generated from the center to the periphery of the predetermined range in the one scan;
The light source control unit controls at least one of a light amount of the first illumination light and a light amount of the second illumination light emitted from the light source based on the luminance shading information stored in the storage unit. The scanning endoscope apparatus according to any one of claims 1 to 10, wherein: The scanning endoscope apparatus according to any one of claims 1, 2, and 4 to 11 , wherein the image synthesis unit generates the synthesized image by high dynamic range synthesis.

Images