WO2017072838A1

WO2017072838A1 - Optical-scanning-type observation device and method for controlling optical-scanning-type observation device

Info

Publication number: WO2017072838A1
Application number: PCT/JP2015/080147
Authority: WO
Inventors: 啓一朗中島
Original assignee: オリンパス株式会社
Priority date: 2015-10-26
Filing date: 2015-10-26
Publication date: 2017-05-04
Also published as: JPWO2017072838A1

Abstract

This optical-scanning-type observation device (100) comprises: an optical scanning unit (2) that scans light; an optical detection unit (3) that photoelectrically converts signal light from a subject (A) and that is capable of multiplying electrons having been created; an image forming unit (4) that forms image data on the basis of the radiation position of the light and the magnitude of an electrical signal having been output from the optical detection unit (3); and a control unit (6) that controls the multiplication factor of the optical detection unit (3) on the basis of the magnitude of the electrical signal. The control unit (6) includes a white balance adjustment unit (61) that adjusts the white balance of the image data on the basis of the multiplication factor of the optical detection unit (3).

Description

Optical scanning observation apparatus and method for controlling optical scanning observation apparatus

The present invention relates to an optical scanning observation apparatus and a method for controlling the optical scanning observation apparatus.

2. Description of the Related Art Conventionally, there is known a scanning endoscope apparatus that scans a laser beam on a subject along a spiral trajectory and detects signal light from the subject (see, for example, Patent Document 1).
Since the intensity of the detected signal light changes according to the reflectance of the subject and the distance from the subject to the tip of the endoscope, the brightness of the image also changes. As a method for maintaining the brightness of an image constant, an automatic gain control that adjusts the gain of a photodetector according to the brightness of the image is generally used.

Japanese Patent No. 5025877

However, when an avalanche photodiode (APD) that multiplies photocurrent by applying a reverse bias voltage is used as a photodetector, the white balance of the image changes when the APD multiplication factor changes. There's a problem.
That is, when a color image is acquired, red (R), green (G), and blue (B) signal lights are detected by the APD. The multiplication factor of APD has wavelength dependence, and the relationship between the reverse bias voltage and the multiplication factor varies depending on the wavelength of incident light. Therefore, when the reverse bias voltage changes, a difference in the change amount of the multiplication factor occurs between the R, G, and B signal lights, and the white balance of the image is not maintained before and after the change of the reverse bias voltage.

For example, the ratio of the magnitudes of the R, G, and B output signals from the APD when a white object is shot with the APD multiplication factor set to 50 is R: G: B = 120: 100: 78. Suppose. Next, it is assumed that the same white subject is photographed with the APD multiplication factor changed to 3. At this time, the ratio of the magnitudes of the output signals of R, G, and B is not 120: 100: 78, but another ratio, for example, 125: 100: 75.

The present invention has been made in view of the above-described circumstances, and provides an optical scanning observation apparatus and an optical scanning observation apparatus control method capable of adjusting the brightness of an image while suppressing a change in white balance. The purpose is to provide.

In order to achieve the above object, the present invention provides the following means.
According to a first aspect of the present invention, an optical scanning unit that irradiates a subject while scanning light, and a signal light generated in the subject by the light irradiation are received, and the received signal light is photoelectrically converted into an electron. A photodetection unit that outputs the generated electrons as an electrical signal, the photodetection unit capable of multiplying the electrons generated by the photoelectric conversion and changing the multiplication factor of the electrons; An image forming unit that forms image data based on the electrical signal output from the light detection unit, and setting of a multiplication factor of the light detection unit based on the brightness of the image data formed by the image forming unit And a control unit that controls the light detection unit so as to match the multiplication factor with the determined setting value, the control unit based on the multiplication factor of the light detection unit. Adjust the white balance An optical scanning observation apparatus provided with the white balance adjustment unit.

According to the first aspect of the present invention, when signal light is generated by irradiating the subject with light by the light scanning unit, the signal light is detected by the light detection unit, and an electric signal based on the intensity of the signal light is generated. Generated. Then, the image data of the subject is formed by the image forming unit associating the electric signal with the light irradiation position. The control unit adjusts the brightness of the image data formed next by the image forming unit by changing the multiplication factor of the electric signal by the light detection unit based on the brightness of the image data. Thereby, it is possible to obtain image data whose brightness is appropriately adjusted.

In this case, the white balance of the image data formed by the image forming unit is controlled by the white balance adjusting unit based on the multiplication factor of the light detecting unit. Thereby, the change of the white balance of the image data accompanying the change of the multiplication factor of the light detection unit at the time of adjusting the brightness of the image data can be suppressed.

In the first aspect, the white balance adjustment unit sets white balance based on the image data acquired when the multiplication factor of the light detection unit is equal to or greater than a predetermined threshold, and the image formation The unit may form the image data having a white balance set by the white balance adjustment unit.
The wavelength dependence of the multiplication factor of the light detection unit is small in a range where the multiplication factor is relatively large. By setting the white balance based on the image data acquired when the multiplication factor is within this range, the change in the white balance of the image data accompanying the change of the multiplication factor within a relatively large range is effective. Can be suppressed.

In the first aspect, the white balance adjustment unit may permit the control unit to change the multiplication factor of the light detection unit only within a range of a predetermined threshold value or more.
The wavelength dependence of the multiplication factor of the light detection unit is small in a range where the multiplication factor is relatively large. By permitting the change of the multiplication factor of the light detection unit only within such a range, it is possible to suppress the change in the white balance of the image data accompanying the change of the multiplication factor of the light detection unit.

In the first aspect, the light detection unit generates a red signal, a green signal, and a blue signal based on intensities of red, green, and blue components included in the signal light, respectively, as the electrical signal, and the red signal A storage unit that stores adjustment coefficients for the green signal and the blue signal, and the image forming unit is based on the red signal, the green signal, and the blue signal adjusted using the adjustment coefficient stored in the storage unit. The image data may be formed.
In this way, image data with adjusted white balance can be formed by the image forming unit.

In the first aspect, the storage unit stores a table in which a set value of a multiplication factor of the light detection unit and the adjustment coefficient are associated with each other, and the white balance adjustment unit includes the control unit in the table. The adjustment coefficient associated with the set value of the multiplication factor determined by the selection is selected, and the image forming unit forms the image data using the adjustment coefficient selected by the white balance adjustment unit. Good.
In this way, image data having a predetermined white balance can be formed regardless of the multiplication factor of the light detection unit.

A second aspect of the present invention is a light detection unit that photoelectrically converts signal light from a subject to generate electrons, and outputs the generated electrons as an electrical signal, wherein the electrons generated by the photoelectric conversion are An optical scanning type comprising a light detection unit capable of multiplying and capable of changing the multiplication factor of the electrons, and forming image data of the subject based on the magnitude of the electric signal output from the light detection unit A method for controlling an observation apparatus, the method for controlling an optical scanning observation apparatus that adjusts a white balance of the image data based on a multiplication factor of the light detection unit.

According to the present invention, it is possible to adjust the brightness of an image while suppressing a change in white balance.

1 is an overall configuration diagram of an optical scanning endoscope according to a first embodiment of the present invention. It is a flowchart which shows operation | movement of the optical scanning endoscope which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the white balance setting routine in the flowchart of FIG. It is a flowchart which shows the brightness adjustment routine in the flowchart of FIG. It is a graph which shows the relationship between the setting value of the multiplication factor of APD, and the actual multiplication factor of R signal, G signal, and B signal. It is a flowchart which shows the brightness adjustment routine in operation | movement of the optical scanning observation apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the white balance setting routine in operation | movement of the optical scanning observation apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the table memorize | stored in the memory | storage part in the optical scanning observation apparatus which concerns on the 3rd Embodiment of this invention. It is a flowchart which shows operation | movement of the optical scanning type observation apparatus which concerns on the 3rd Embodiment of this invention.

(First embodiment)
The optical scanning observation apparatus 100 according to the first embodiment of the present invention will be described below with reference to FIGS.
The optical scanning observation apparatus 100 according to the present embodiment is an endoscope apparatus, and is connected to an elongated insertion part 20 that can be inserted into a body and a proximal end of the insertion part 20 as shown in FIG. The control device main body 30, and a user interface (UI) 40 and a display 50 connected to the control device main body 30 are provided.

The optical scanning observation apparatus 100 also includes a light source unit 1 that outputs laser light, an optical scanning unit 2 that spirally scans the laser light on the subject A, and light detection that detects reflected light of the laser light from the subject A. A unit 3; an image forming unit 4 that forms image data of the subject A based on an irradiation position of reflected light (signal light) and laser light; a white balance (WB) setting unit 5 that sets a white balance of the image data; , A light source unit 1, a light scanning unit 2, a light detection unit 3, an image forming unit 4, and a control unit 6 that controls the WB setting unit 5, and a storage unit 7.

The light source unit 1 includes three

laser light sources

8R, 8G, and 8B that respectively generate red (R), green (G), and blue (B) laser light, and the three

laser light sources

8R, 8G, and 8B. And a coupler 9 that coaxially combines the output R, G, and B laser beams.
The three

laser light sources

8R, 8G, and 8B are controlled by the control unit 6 so as to repeatedly output R, G, and B pulsed laser beams in order. As a result, R, G, and B laser beams are sequentially output from the coupler 9, and R, G, and B reflected light are sequentially generated from the subject A.

The optical scanning unit 2 includes an illumination optical fiber 10 disposed along the longitudinal direction in the insertion unit 20, and an actuator 11 that vibrates the tip of the optical fiber 10 in a direction intersecting the longitudinal direction of the optical fiber 10. And an actuator driver 12 for driving the actuator 11.
The proximal end of the optical fiber 10 is connected to the coupler 9. The laser light incident on the proximal end of the optical fiber 10 from the coupler 9 is guided from the proximal end to the distal end inside the optical fiber 10 and emitted from the distal end of the optical fiber 10 toward the front end of the insertion portion 20. It has become so.

The actuator 11 is, for example, a piezoelectric actuator that includes a piezoelectric element, and is attached to the tip of the optical fiber 10.
The actuator driver 12 generates a drive signal for spirally vibrating the tip of the optical fiber 10 along a spiral trajectory in a substantially plane that intersects the longitudinal direction of the optical fiber 10 according to a control signal received from the control unit 6. Then, the drive signal is supplied to the actuator 11. Thereby, the tip of the optical fiber 10 spirally vibrates, and the laser light emitted from the tip of the optical fiber 10 is spirally scanned along a spiral scanning locus.

The light detection unit 3 is an avalanche photodiode (APD; hereinafter also referred to as “APD3”), and detects reflected light received by the light receiving optical fiber 13.
The optical fiber 13 is disposed in the insertion portion 20 along the longitudinal direction. The distal end of the optical fiber 13 is disposed on the distal end surface of the insertion portion 20, and the proximal end of the optical fiber 13 is connected to the APD 3. The reflected light that has entered the tip of the optical fiber 13 from the subject A is guided from the tip to the base end of the optical fiber 13 and enters the APD 3. Although only one optical fiber 13 is shown in FIG. 1, a plurality of optical fibers 13 may be provided.

The APD 3 photoelectrically converts the reflected light incident on the APD 3 to generate an amount of electric charge corresponding to the incident light amount of the reflected light, and outputs an electric signal having a magnitude corresponding to the generated electric charge amount. Here, the APD 3 has a function of multiplying the electric signal by multiplying the charge generated by the photoelectric conversion by applying a reverse bias voltage. As the reverse bias voltage increases, the multiplication factor of the electric signal also increases. APD3 can change a multiplication factor in steps, for example between 1 time and 100 times. The multiplication factor of the APD 3 is controlled by the control unit 6 as will be described later.

An amplifier 15 that amplifies the electrical signal output from the APD 3 and an analog-digital converter (ADC) 16 are provided at the subsequent stage of the APDP 3.
The electric signal output from the APD 3 is amplified by the amplifier 15 and then input to the ADC 16.
The ADC 16 samples the electrical signal from the amplifier 15 and performs AD conversion to obtain a digital value corresponding to the magnitude of the electrical signal. The obtained digital value is transmitted to the image forming unit 4.

Here, since the reflected light of R, G, and B is received by the optical fiber 13 in order, the APD 3 generates and outputs the R signal, the G signal, and the B signal in order. The R signal, G signal, and B signal are electrical signals based on the reflected light of R, G, and B, respectively. Therefore, the R signal value, the G signal value, and the B signal value are sequentially obtained as digital values by the ADC 16.

The image forming unit 4 uses a set of R, G, and B signal values arranged in the time axis direction received from the ADC 16 as pixel values of one pixel, and the irradiation position of the laser beam received from the control unit 6 By associating with (described later), raw image data is formed each time the laser beam completes scanning the entire predetermined scanning locus.

Next, the image forming unit 4 performs WB adjustment processing on the raw image data. Specifically, the image forming unit 4 acquires adjustment coefficients Cr, Cg, and Cb (described later) from the storage unit 7, and adjusts the R coefficient value, the G signal value, and the B signal value of each pixel of the raw image data. Output image data is generated by multiplying Cr, Cg, and Cb, respectively. The image forming unit 4 may perform arbitrary image processing on the raw image data in addition to the WB adjustment processing.
The output image data is transmitted to the display 50 and displayed on the display 50.

The WB setting unit 5 receives raw image data from the image forming unit 4 when receiving a WB setting execution signal (described later) from the control unit 6, and sets white balance based on the received raw image data. WB setting operation is executed. Specifically, the WB setting unit 5 calculates a ratio Vr: Vg: Vb of the R signal value Vr, the G signal value Vg, and the B signal value Vb of the raw image data. The signal values Vr, Vg, and Vb are, for example, average values of the R signal value, the G signal value, and the B signal value of all the pixels of the raw image data or the central pixel. Since the change in brightness is small in the central portion of the raw image data, more stable signal values Vr, Vg, and Vb can be obtained by using the signal value of the pixel in the central portion.
Next, the WB setting unit 5 calculates adjustment coefficients Cr, Cg, and Cb that satisfy Vr × Cr: Vg × Cg: Vb × Cb = 1: 1: 1. The WB setting unit 5 stores the calculated values of the adjustment coefficients Cr, Cg, and Cb in the storage unit 7 via the control unit 6.

The control unit 6 controls the

laser light sources

8R, 8G, and 8B so that the

laser light sources

8R, 8G, and 8B sequentially output the laser light at regular time intervals. In addition, the control unit 6 controls the ADC 16 so that the ADC 16 samples an electrical signal in synchronization with the output of the laser light from the

laser light sources

8R, 8G, and 8B. Further, the control unit 6 calculates the irradiation position of the laser light from the control signal, and transmits information on the calculated irradiation position to the image forming unit 4.

Further, the control unit 6 receives the raw image data from the image forming unit 4 and measures the brightness of the raw image data every time the raw image data is formed by the image forming unit 4. The brightness of the raw image data is, for example, the average value of the pixel values of all the pixels or the central pixel. The maximum value may be calculated instead of the average value. Further, an average value or a maximum value may be calculated using only the G signal value among the R, G, and B signal values. The G signal value is closer to the brightness felt by human eyes. Therefore, the brightness of the raw image data can be more appropriately evaluated by using only the G pixel value.

Next, the control unit 6 determines the set value of the multiplication factor of the APD 3 based on the measured brightness. Specifically, when the brightness of the raw image data is within a predetermined appropriate range, the control unit 6 maintains the current setting value of the multiplication factor, and the brightness of the raw image data exceeds the predetermined range. Is set to a value smaller than the current setting value, and when the brightness of the raw image data is smaller than a predetermined range, the setting value of the APD 3 multiplication factor is changed. Change to a value larger than the current setting. The control unit 6 controls the APD 3 so that the multiplication factor is set to the determined setting value.

The control unit 6 is connected to the UI 40. The UI 40 has a white balance (WB) setting button (not shown), and transmits a WB setting command signal to the control unit 6 when the WB setting button is pressed. When the control unit 6 receives a WB setting command signal from the UI 40, the control unit 6 transmits the WB setting execution signal to the WB setting unit 5 to cause the WB setting unit 5 to execute the above-described WB setting operation.

Further, the control unit 6 includes a white balance (WB) adjustment unit 61 that permits or rejects the WB setting operation by the WB setting unit 5 based on the multiplication factor of the current APD 3 when a WB setting command signal is received from the UI 40. ing. The WB adjustment unit 61 permits the WB setting operation by the WB setting unit 5 only when the multiplication factor of the current APD 3 is equal to or greater than a predetermined threshold T (for example, 10 times), and sends a WB setting execution signal from the control unit 6. Is transmitted to the WB setting unit 5. When the multiplication factor of the current APD 3 is less than the predetermined threshold T, the WB adjustment unit 61 rejects the WB setting operation by the WB setting unit 5 and sends a WB setting execution signal from the control unit 6 to the WB setting unit 5. The control unit 6 outputs a warning signal (for example, sound, light, or display) to the user without transmitting to the user.

The storage unit 7 stores arbitrary adjustment coefficients Cr, Cg, and Cb (for example, Cr = Cg = Cb = 1) in an initial state before the first WB setting operation is executed. When the storage unit 7 receives new adjustment coefficients Cr, Cg, Cb from the WB setting unit 5 via the control unit 6, the storage unit 7 converts the previous adjustment coefficients Cr, Cg, Cb into new adjustment coefficients Cr, Cg, Cb. The adjustment coefficients Cr, Cg, and Cb are updated by the replacement.

In the present embodiment, the above-described functions of the image forming unit 4, the WB setting unit 5, and the control unit 6 are realized by, for example, a general-purpose or dedicated computer. The computer includes a central processing unit (CPU), a main storage device such as a RAM, and an auxiliary storage device such as a hard disk and various memories. The auxiliary storage device includes the above-described

units

4, 5, and 6. A program for causing the CPU to execute processing is stored. This program is loaded from the auxiliary storage device to the main storage device and executed, so that the CPU realizes the processing of the

respective units

4, 5, and 6.

Next, the operation of the optical scanning observation apparatus 100 configured as described above will be described with reference to FIGS.
When supply of a drive signal from the actuator driver 12 to the actuator 11 and output of laser light from the

laser light sources

8R, 8G, and 8B are started, the R, G, and B laser lights of the optical fiber 10 that spirally vibrates. Injected in order from the tip. Thus, R, G, and B laser beams are sequentially irradiated along the spiral scanning locus on the surface of the subject A that faces the distal end surface of the insertion unit 20.

The reflected light of the laser beam reflected on the surface of the subject A is received by the optical fiber 13, photoelectrically converted by the APD 3, and further digitally converted by the ADC 16. As a result, R, G, and B signal values indicating the intensity of the reflected light of R, G, and B are obtained in order. The obtained R, G, and B signal values are associated with the laser light irradiation position in the image forming unit 4 to generate raw image data of the subject A (step S1). Next, WB adjustment processing is performed on the raw image data in the image forming unit 4 (step S4), and output image data in which the WB is adjusted is displayed on the display 50.

The amount of reflected light received by the optical fiber 13 depends on the reflectance of the surface of the subject A and the distance and angle from the subject A to the tip of the insertion portion 20. Accordingly, the brightness of the raw image data changes as the insertion unit 20 moves in the body. The control unit 6 adjusts the multiplication factor of the APD 3 in obtaining the next raw image data based on the brightness of the raw image data so that the brightness of the raw image data is within a predetermined appropriate range (step S5). .

Specifically, as shown in FIG. 3, the control unit 6 calculates the brightness of the raw image data every time new raw image data is generated (step S51). If the brightness of the raw image data is within a predetermined appropriate range (YES in step S52), the control unit 6 maintains the current multiplication factor setting value of the APD 3. Thereby, the next raw image data having the same brightness is acquired. On the other hand, when the brightness of the raw image data is darker than the predetermined appropriate range (NO in step S52 and NO in step S53), the control unit 6 increases the set value of the multiplication factor of the APD 3 (step S54). Thereby, the multiplication factor of the electric signal by the APD 3 is increased, and the raw image data whose brightness is increased is acquired next. On the other hand, when the brightness of the raw image data is brighter than the predetermined appropriate range (NO in step S52 and YES in step S53), the control unit 6 reduces the set value of the multiplication factor of the APD 3 (step S55). As a result, the multiplication factor of the electrical signal by the APD 3 is reduced, and raw image data with reduced brightness is acquired next.

Here, in order to set an appropriate white balance, the user sets a white subject so as to face the tip of the insertion unit 20, and the output image data of the white subject is displayed on the display 50. Press the WB setting button. For example, the white balance setting may be executed only once before the optical scanning observation apparatus 100 is shipped, or may be executed every time before the observation is performed. When the WB setting button is pressed, a WB setting command signal is transmitted from the UI 40 to the control unit 6 (YES in step S2).

In the control unit 6, as shown in FIG. 4, the WB adjustment unit 61 responds to the WB setting command signal (step S3), and whether or not the current setting value of the multiplication factor of the APD 3 is greater than or equal to a predetermined threshold T. (Steps S31 and S32). When the set value of the multiplication factor is equal to or greater than the predetermined threshold T (YES in step S32), the WB adjustment unit 61 transmits a WB setting execution signal from the control unit 6 to the WB setting unit 5, thereby causing the WB setting unit 5 to Then, the adjustment coefficients Cr, Cg, and Cb are calculated based on the raw image data (step S33).

On the other hand, when the set value of the multiplication factor is less than the predetermined threshold T (NO in step S32), the WB adjustment unit 61 outputs a warning signal from the control unit 6 (step S34). Based on the warning signal, the user can recognize that the multiplication factor of the current APD 3 is out of the range appropriate for the white balance setting. That the multiplication factor is less than the threshold value T means that the amount of incident light of the reflected light on the optical fiber 13 is too large and the raw image data is too bright. Therefore, for example, the user moves the distal end of the insertion unit 20 away from the white subject so that the set value of the multiplication factor becomes equal to or greater than the predetermined threshold T. Thereby, when the set value of the multiplication factor becomes equal to or greater than the predetermined threshold T (YES in step S32), the WB adjustment unit 6 transmits a WB setting execution signal from the control unit 6 to the WB setting unit 5, thereby setting the WB setting. The unit 5 is caused to calculate adjustment coefficients Cr, Cg, and Cb based on the raw image data (step S33).

The new adjustment coefficients Cr, Cg, and Cb calculated in step S33 are replaced with the adjustment coefficients previously stored in the storage unit 7, and a new adjustment is used for the white balance adjustment of the raw image data in the subsequent step S4. Coefficients Cr, Cg, Cb are used.

Here, the relationship between the multiplication factor set in the APD 3 and the actual multiplication factor of the R, G, and B signals by the APD 3 will be described.
As shown in FIG. 5, the larger the multiplication factor set in the APD 3, the larger the actual multiplication factors of the R signal, G signal, and B signal. However, the multiplication factor of the electric signal by the APD 3 has wavelength dependence, and even if the multiplication factor set in the APD 3 is the same, the actual multiplication factor of the R signal, the G signal, and the B signal is different. Further, when the multiplication factor of the APD 3 is changed, the change amount of the multiplication factor of the R signal, the change amount of the multiplication factor of the G signal, and the change amount of the multiplication factor of the B signal are different. Therefore, the white balance of the raw image data acquired at different multiplication factors is different from each other.

The difference in the change amount of the multiplication factor between the R, G, and B signals is particularly large in the range where the multiplication factor is small (specifically, less than 10 times), and the range where the multiplication factor is large (specifically, 10 times or more) is small. FIG. 5 shows an example of the relative output levels of the R signal and the B signal when the output level of the G signal from the APD 3 is 100%.

According to the present embodiment, only when the multiplication factor of the APD 3 is set to a threshold value T or more where the difference in the change amount of the multiplication factor among the R, G, and B signals is small, the WB adjustment unit 61 The WB setting operation by the setting unit 5 is permitted. The difference in white balance between raw image data acquired at different multiplication factors equal to or greater than the threshold T is small. Therefore, using adjustment coefficients Cr, Cg, and Cb determined based on the raw image data acquired at a multiplication factor equal to or higher than the threshold T, the WB of the raw image data acquired at another multiplication factor higher than the threshold T is used. When adjusted, output image data having substantially the same white balance can be generated. Thereby, it is possible to suppress the change in the white balance of the output image data when the multiplication factor of the APD 3 changes for brightness adjustment.

(Second Embodiment)
Next, an optical scanning observation apparatus according to a second embodiment of the present invention will be described with reference to FIGS.
The optical scanning observation apparatus according to the present embodiment is different from the first embodiment in the method for determining the set value of the multiplication factor of the APD 3 by the control unit 6. Therefore, in this embodiment, the determination method of the setting value of the multiplication factor by the control part 6 is mainly demonstrated, and the code | symbol same about the structure which is common in 1st Embodiment is attached | subjected and description is abbreviate | omitted.

In the present embodiment, the storage unit 7 stores a lower limit threshold value that defines the setting range of the multiplication factor of the APD 3. For example, when the APD 3 can change the multiplication factor within the range of 1 to 100 times, the lower limit threshold is set to 10 times.

The control unit 6 acquires the lower limit threshold value from the storage unit 7, and determines the setting value of the multiplication factor within a range equal to or higher than the lower limit threshold value. That is, when the brightness of the raw image data is larger than the predetermined range, the WB adjustment unit 61 compares the multiplication factor set value with the lower limit threshold value, and the multiplication factor set value is equal to or greater than the lower limit threshold value. Only, the control unit 6 is allowed to change the APD 3, and the control unit 6 changes the setting value of the multiplication factor of the APD 3 to a value smaller than the current setting value. On the other hand, when the set value of the multiplication factor is already the lower limit threshold, the WB adjustment unit 61 prohibits the change of the APD 3. Therefore, even if the brightness of the raw image data is smaller than the predetermined range, the control unit 6 does not change the setting value of the multiplication factor to a smaller value and continues to set the lower limit threshold value.

Next, the operation of the optical scanning observation apparatus configured as described above will be described.
The operation of the optical scanning observation apparatus according to the present embodiment differs from the first embodiment in the WB setting routine S3 and the brightness adjustment routine S5, and other steps S1, S2, and S3 are the same as those in the first embodiment. is there. Therefore, the WB setting routine S4 and the brightness adjustment routine S5 will be described.

In the brightness adjustment routine S5, as shown in FIG. 6, when the brightness of the raw image data is brighter than a predetermined appropriate range (NO in step S52 and YES in step S53), the WB adjustment unit 61 It is determined whether or not the set value of the multiplication factor of APD 3 is larger than the lower limit threshold (step S56). When the current set value of the multiplication factor is larger than the lower limit threshold value (YES in step S56), the WB adjustment unit 61 sets the set value of the multiplication factor of the APD 3 to the control unit 6 as in the first embodiment. Reduce (step S55). As a result, the multiplication factor of the electrical signal by the APD 3 is reduced, and raw image data with reduced brightness is acquired next. On the other hand, when the current set value of the multiplication factor is equal to the lower limit threshold value (NO in step S56), the WB adjustment unit 61 does not allow the control unit 6 to change the multiplication factor of the APD 3, and therefore the control unit 6 , APD3's current multiplication factor setting value is maintained. Thereby, even if the raw image data is too bright, the multiplication factor is maintained at the lower limit threshold.
Other steps S51, S52, S53, and S54 are the same as those in the first embodiment.

In the WB setting routine S3, as shown in FIG. 7, the WB adjusting unit 61 permits the WB setting operation by the WB setting unit 5 regardless of the current setting value of the multiplication factor of the APD 3, and the WB setting operation from the control unit 6 By transmitting the execution signal to the WB setting unit 5, the WB setting unit 5 is caused to calculate the adjustment coefficients Cr, Cg, and Cb based on the raw image data (step S33).

As described above, especially in a range where the multiplication factor is small (specifically, a range less than 10 times), a change in white balance of the raw image data accompanying a change in multiplication factor, that is, an image of the subject A in the raw image data. The change in color becomes noticeable. According to the present embodiment, the WB adjustment unit 61 prohibits the use of a multiplication factor that is less than a lower limit threshold that greatly changes the white balance of the raw image data, and a range in which the change of the white balance of the raw image data is less than the lower limit threshold. The change of the multiplication factor is allowed only within. Thus, when the multiplication factor of the APD 3 is changed for brightness adjustment, the change in the white balance of the raw image data is suppressed, and the output image data with a small change in the color of the image of the subject A is provided to the user. Can do.

In the present embodiment, instead of the white balance setting routine S4 shown in FIG. 7, the same white balance setting routine S4 as that in the first embodiment shown in FIG. 4 may be adopted.

(Third embodiment)
Next, an optical scanning observation apparatus according to a third embodiment of the present invention will be described with reference to FIGS.
The optical scanning observation apparatus according to the present embodiment differs from the first embodiment in the white balance adjustment method. Therefore, in the present embodiment, the white balance adjustment method will be mainly described, and the same reference numerals will be given to the same components as those in the first embodiment, and description thereof will be omitted.

In the present embodiment, as shown in FIG. 8, the storage unit 7 stores a table in which the set value of the multiplication factor of the APD 3 and the adjustment coefficients Cr, Cg, and Cb are associated with each other. The adjustment coefficients Cr, Cg, and Cb are obtained based on the obtained raw image data by acquiring the raw image data at each multiplication factor by changing the multiplication factor of the APD 3 while observing the same white subject, for example. Value.

The WB adjustment unit 61 selects adjustment coefficients Cr, Cg, and Cb corresponding to the current setting value of the APD 3 from the table, and selects the selected adjustment coefficients Cr, Cg, and Cb from the control unit 6 to the image forming unit 4. To send to.
The image forming unit 4 performs WB adjustment processing on the raw image data using the adjustment coefficients Cr, Cg, and Cb received from the control unit 6 to generate output image data.

Next, the operation of the optical scanning observation apparatus configured as described above will be described.
The operation of the optical scanning observation apparatus according to the present embodiment differs from the first embodiment in the white balance adjustment method.
When the raw image data of the subject A is generated (step S1), the WB adjustment unit 61 obtains adjustment coefficients Cr, Cg, and Cb corresponding to the current set value of the multiplication factor of the APD 3, as shown in FIG. Obtained from the table in the storage unit 7 (steps S6, S7). In the next step S4, WB adjustment processing is performed on the raw image data in the image forming unit 4 using the acquired adjustment coefficients Cr, Cg, and Cb.
The brightness adjustment routine S5 is the same as the brightness adjustment routine S5 of the first embodiment shown in FIG.

As described above, according to the present embodiment, the adjustment coefficients Cr, Cg, and Cb for adjusting the white balance of the raw image data to the same white balance are stored in the table in advance for each set value of the multiplication factor of the APD 3. ing. Accordingly, output image data having the same white balance can be obtained no matter what value the multiplication factor of APD 3 is changed. In particular, even if the multiplication factor of the APD 3 is in a small range, it is possible to obtain high quality output image data without white balance change and pixel value saturation.

DESCRIPTION OF SYMBOLS 1 Light source part 2 Optical scanning part 3 Light detection part, avalanche photodiode 4 Image formation part 5 White balance setting part 6 Control part 61 White balance adjustment part 7 Memory |

storage part

8R, 8G, 8B Laser light source 9

Coupler

10, 13 Optical fiber DESCRIPTION OF SYMBOLS 11 Actuator 12 Actuator driver 15 Amplifier 16 Analog-digital converter 20 Insertion part 30 Control apparatus main body 40 User interface 50 Display 100 Optical scanning type observation apparatus A Subject

Claims

An optical scanning unit that irradiates a subject while scanning light;
A photodetection unit that receives signal light generated in the subject by the light irradiation, photoelectrically converts the received signal light to generate electrons, and outputs the generated electrons as an electrical signal, A photodetection unit capable of multiplying the electrons generated by the conversion and changing the multiplication factor of the electrons;
An image forming unit that forms image data of the subject based on a magnitude of the electrical signal output from the light detection unit and an irradiation position of the light by the light scanning unit;
Determining a setting value of the multiplication factor of the light detection unit based on the magnitude of the electrical signal, and a control unit for controlling the light detection unit to match the multiplication factor to the determined setting value,
An optical scanning observation apparatus, wherein the control unit includes a white balance adjustment unit that adjusts a white balance of the image data based on a multiplication factor of the light detection unit.
The white balance adjustment unit sets a white balance based on the image data acquired when the multiplication factor of the light detection unit is equal to or greater than a predetermined threshold;
The optical scanning observation apparatus according to claim 1, wherein the image forming unit forms the image data having white balance set by the white balance adjusting unit.
The optical scanning observation apparatus according to claim 1 or 2, wherein the white balance adjustment unit permits the control unit to change the multiplication factor of the light detection unit only within a range equal to or greater than a predetermined threshold.
The light detection unit generates a red signal, a green signal, and a blue signal based on the intensities of red, green, and blue components included in the signal light, respectively, as the electrical signal;
A storage unit that stores adjustment coefficients for the red signal, the green signal, and the blue signal;
The optical scanning observation according to claim 1, wherein the image forming unit forms the image data based on the red signal, the green signal, and the blue signal adjusted using the adjustment coefficient stored in the storage unit. apparatus.
The storage unit stores a table in which a set value of a multiplication factor of the light detection unit is associated with the adjustment coefficient;
The white balance adjustment unit selects an adjustment coefficient associated with the set value of the multiplication factor determined by the control unit in the table;
The optical scanning observation apparatus according to claim 4, wherein the image forming unit forms the image data using an adjustment coefficient selected by the white balance adjustment unit.
A photodetection unit that photoelectrically converts signal light from a subject to generate electrons, and outputs the generated electrons as an electrical signal, wherein the electrons generated by the photoelectric conversion can be multiplied and A control method for an optical scanning observation apparatus comprising a light detection unit capable of changing a multiplication factor, and forming image data of the subject based on the magnitude of the electrical signal output from the light detection unit,
A control method of an optical scanning observation apparatus that adjusts a white balance of the image data based on a multiplication factor of the light detection unit.