WO2018127958A1

WO2018127958A1 - Optical-scanning-type image-forming device and optical-scanning-type endoscope system

Info

Publication number: WO2018127958A1
Application number: PCT/JP2017/000168
Authority: WO
Inventors: 西村　淳一
Original assignee: オリンパス株式会社
Priority date: 2017-01-05
Filing date: 2017-01-05
Publication date: 2018-07-12

Abstract

Provided is an optical-scanning-type image-forming device that comprises: a light source unit (5) that outputs illumination light; a driving-waveform generating unit (7) that generates a driving waveform; an optical scanning unit (6) that scans the illumination light in accordance with the driving waveform; a light detecting unit (9) that acquires gradation values based on observation light from a subject; a phase setting unit (8) that sets the phase of the driving waveform so as to apply a phase difference to the driving waveform between a prescribed number of frames in one set of frame periods composed of a prescribed number of frame periods; and an image processing unit (10) that acquires one set of images composed of a prescribed number of images in one set of frame periods. The image processing unit (10) interpolates a gradation value of a defective pixel included in an image, using a gradation value in images constituting a set that was obtained prior to the set including that image.

Description

Optical scanning image forming apparatus and optical scanning endoscope system

The present invention relates to an optical scanning image forming apparatus and an optical scanning endoscope system.

Conventionally, a scanning endoscope that scans light emitted from an optical fiber by vibrating the optical fiber along a spiral scanning locus and receives reflected light from a position on the scanning locus to form an image. It is known (for example, refer to Patent Document 1). When the interval of the scanning trajectory is wide, a defective pixel that does not have a gradation value and appears as a black dot occurs in the generated image. In Patent Document 1, the occurrence of a defective pixel is prevented by adjusting the interval of the scanning locus so as to coincide with the target value.

JP 2014-149354 A

However, as in Patent Document 1, in order to reduce the interval between scanning trajectories to a target value, it is necessary to increase the number of vibrations of the optical fiber per image frame. Therefore, there is a problem that the scanning time per frame becomes long and the frame rate is lowered. Although it is possible to maintain a high frame rate by increasing the drive frequency of the optical fiber, in that case, a huge amount of processing must be performed at a high speed and the load on the actuator increases.

The present invention has been made in view of the above-described circumstances, and provides an optical scanning image forming apparatus and an optical scanning endoscope system capable of preventing the occurrence of defective pixels while maintaining a high frame rate. The purpose is to do.

In order to achieve the above object, the present invention provides the following means.
According to a first aspect of the present invention, a light source unit that outputs illumination light, a drive waveform generation unit that generates a drive waveform, and a drive waveform generated by the drive waveform generation unit that emits illumination light output from the light source unit A light scanning unit that scans on the subject according to the above, a light detection unit that detects the observation light generated in the subject by irradiation of the illumination light and obtains a gradation value based on the observation light, and 2 in a time axis direction. The phase of the drive waveform generated by the drive waveform generation unit so as to give a phase difference to the drive waveform between the predetermined number of frames in a set of frame periods in which images of the predetermined number of frames are acquired. Generating one frame image based on the phase setting unit for setting the image, the gradation value obtained by the light detection unit during one frame period, and the irradiation position of the illumination light. During the period An image processing unit that obtains a set of images composed of images of the predetermined number of frames, and the image processing unit determines whether or not the image includes a defective pixel having no gradation value, and the image Is an optical scanning type image forming apparatus that interpolates the gradation value of a defective pixel using a gradation value in an image constituting a group obtained in the past than a group including the image when the pixel includes a defective pixel .

According to the first aspect of the present invention, the illumination light output from the light source unit is scanned on the subject by the light scanning unit, the observation light generated at the illumination light irradiation position is detected by the light detection unit, and the observation light is detected. One frame image is generated by the image processing unit during one frame period from the gradation value based on the above and the irradiation position of the illumination light.

In this case, since the drive waveforms in a predetermined number of consecutive frame periods have different phases, the scanning traces of illumination light are different from each other in images of a predetermined number of consecutive frames. Thus, an image equivalent to the case where the density of the scanning trajectory is increased is obtained from the images of a predetermined number of frames having different scanning trajectories. That is, the density of scanning trajectories can be substantially increased without reducing the frame rate. Thereby, it is possible to prevent the occurrence of defective pixels while maintaining a high frame rate.

In the first aspect, the phase setting unit may set a number obtained by dividing 360 ° by the predetermined number as the phase difference.
In this way, the phase difference can be easily set so that the scanning trajectories are different from each other in images of a predetermined number of consecutive frames. Further, an image equivalent to or close to that obtained when the density of the scanning trajectory is increased by a predetermined number of times can be obtained from images of a predetermined number of consecutive frames.

The first aspect may include an output unit that outputs the image generated by the image processing unit for each frame.
In this way, by sequentially displaying the images output from the output unit on the display, images with slightly different scanning trajectory positions are sequentially displayed as in the interlace method. Since an afterimage of the image displayed immediately before remains for an observer who observes the image, the apparent resolution can be improved as compared with the resolution of each image.

In the first aspect, the image processing unit generates a composite image by combining the images of the predetermined number of frames constituting the same set, and the composite image generated by the image processing unit is used as the composite image. You may provide the output part which outputs.
By doing in this way, it is possible to obtain a composite image having a higher resolution than individual images. In addition, compared with the case where the resolution is improved by increasing the length of one frame period, the distance at which the illumination light is scanned is increased and the irradiation positions at which the illumination light is sampled are dispersed, thereby improving the resolution. Can do.

In the first aspect, the drive waveform generation unit may generate the drive waveform that smoothly continues between two frame periods adjacent in the time axis direction.
By doing so, it is possible to stabilize the operation of the optical scanning unit during the transition from the frame period to the next frame period while maintaining the frame rate. A discontinuous change in the drive waveform at the boundary between adjacent frame periods is not preferable because the operation of the optical scanning unit becomes unstable. In order to ensure a stable operation of the optical scanning unit, when a blank period for continuation of the drive waveform is provided between the frame periods, the frame rate is lowered.

In the first aspect, the drive waveform generation unit may generate the drive waveform that is different only in the phase between the predetermined number of frames.
By doing so, stable driving of the optical scanning unit can be realized during a plurality of frame periods, and a stable image can be generated.

In the first aspect, the drive waveform generation unit generates the amplitude-modulated drive waveform, and the shape of the envelope of the amplitude-modulated drive waveform is the same among the predetermined number of frames. Thus, the drive waveform may be amplitude-modulated.
By doing so, a stable scanning locus can be obtained.

In the first aspect, the drive waveform generation unit may generate the drive waveform that is amplitude-modulated with a period equal to the frame period.
By doing so, a stable scanning locus can be obtained.
A second aspect of the present invention is an optical scanning endoscope system including any of the optical scanning image forming apparatuses described above.

According to the present invention, it is possible to prevent the occurrence of defective pixels while maintaining a high frame rate.

1 is an external view showing an overall configuration of an optical scanning endoscope system according to an embodiment of the present invention. It is a block diagram which shows the whole structure of the optical scanning type image forming apparatus mounted in the optical scanning type endoscope system of FIG. FIG. 3 is a diagram illustrating a set of drive waveforms generated by a drive waveform generation unit of the optical scanning image forming apparatus of FIG. 2. FIG. 3 is a diagram illustrating a scanning trajectory of illumination light and an apparent scanning trajectory by an optical scanning unit of the optical scanning image forming apparatus in FIG. 2. FIG. 3 is a diagram illustrating tone value interpolation processing by a black point processing unit of the optical scanning image forming apparatus of FIG. 2. 3 is a flowchart for explaining the operation of the optical scanning image forming apparatus in FIG. 2. FIG. 3 is a block diagram illustrating an overall configuration of a modification of the optical scanning image forming apparatus of FIG.

Hereinafter, an optical scanning image forming apparatus according to an embodiment of the present invention and an optical scanning endoscope system 1 including the same will be described with reference to the drawings.
As shown in FIG. 1, an optical scanning endoscope system 1 according to the present embodiment includes an endoscope 2 having an elongated scope 2 a inserted into the body, and a control device connected to the endoscope 2. A main body 3 and a display 4 connected to the control device main body 3 are provided. The optical scanning endoscope system 1 scans illumination light emitted from the distal end of the scope 2a on the subject A along the spiral scanning trajectories B1 and B2, and acquires an image of the subject A.

The optical scanning image forming apparatus is mounted on the optical scanning endoscope system 1, and as shown in FIG. 2, a light source unit 5 that emits illumination light and illumination light supplied from the light source unit 5. An optical scanning unit 6 that irradiates the subject A while scanning, a drive waveform generation unit 7 that generates a drive waveform for driving the optical scanning unit 6, and a phase of a drive signal generated by the drive waveform generation unit 7 A phase setting unit 8 for setting the image, a light detection unit 9 for detecting the observation light from the subject A to obtain a gradation value, and an image processing unit for drawing an image by associating the gradation value with the irradiation position of the illumination light 10 and an output unit 11 that outputs an image generated by the image processing unit 10 to the display 4.
The light source unit 5, the drive waveform generation unit 7, the phase setting unit 8, and the image processing unit 10 are provided in the control device main body 3, and the light scanning unit 6 and the light detection unit 9 extend over the endoscope 2 and the control device main body 3. Is provided.

The light source unit 5 includes a laser light source (not shown) that outputs laser light as illumination light.
The optical scanning unit 6 scans the illumination light supplied from the light source unit 5 in a direction intersecting the optical axis of the illumination light in accordance with the drive signal supplied from the drive waveform generation unit 7 and emits it toward the subject A. 12 and a condensing lens 13 for condensing the illumination light emitted from the scanner 12 on the subject A. Reference numeral 14 denotes a digital / analog (D / A) converter that generates a drive signal by analog-converting a drive waveform that is a digital signal received from the drive waveform generation unit 7.

The scanner 12 is, for example, an optical fiber scanner including an optical fiber and an actuator that vibrates the tip of the optical fiber along a spiral vibration locus according to a drive signal. The actuator is ejected from the tip of the optical fiber by vibrating the tip of the optical fiber in two radial directions (X direction and Y direction) orthogonal to each other in accordance with a drive signal for the X direction and a drive signal for the Y direction. The illumination light is scanned two-dimensionally.

The drive waveform generation unit 7 generates a drive waveform (see solid line) having a predetermined drive frequency and transmits the drive waveform to the optical scanning unit 6 as shown in FIG. The amplitude of the drive waveform is modulated so as to change with time in the same period as the frame period along a sinusoidal envelope (see broken line). The drive waveform generation unit 7 transmits to the scanner 12 a drive waveform for the X direction and a drive waveform for the Y direction whose phases are different from each other by π / 2. As a result, the illumination light is scanned along the spiral scanning trajectories B1 and B2. During one frame period in which the phase of the envelope changes by 360 °, the illumination light reciprocates once along the scanning trajectories B1 and B2.

Here, the drive waveform generation unit 7 generates a drive waveform for each of the X-direction drive waveform and the Y-direction drive waveform in a set of frame periods including a plurality of frame periods continuous in the time axis direction. These phases are made different in units of one frame period according to the phase difference set by the phase setting unit 8.
Parameters necessary for generating the drive waveform (drive frequency, number of frame periods per set, envelope shape and period) are set in the drive waveform generation unit 7 by an observer, for example.

In the phase setting unit 8, the number (predetermined number) of frame periods constituting one set is set. The phase setting unit 8 divides 360 ° which is the period of the drive waveform by the number of frame periods per set, and sets the obtained quotient as the phase difference. The number of frame periods per set can be set to an arbitrary number of 2 or more. In the present embodiment, an example will be described in which a set of frame periods is formed from two frame periods. Therefore, the phase difference is set to 180 °.

The phase difference is set by using 360 ° by a predetermined number other than using a quotient obtained by dividing 360 ° by a predetermined number (that is, a number obtained by dividing 360 ° by a predetermined number evenly). You may use the number obtained by dividing | segmenting.
For example, the phase difference may be set to 179 °. In this case, a blank period for adjusting the phase of the drive waveform is provided so that the drive waveform smoothly continues between the first frame period and the second frame period.

FIG. 3 shows an example of a set of drive waveforms generated by the drive waveform generator 7. As shown in FIG. 3, between the drive waveform in the first frame period and the drive waveform in the second frame period among the first and second frame periods adjacent in the time axis direction, 180 ° The phase difference is given.

FIG. 4 shows scanning trajectories B1 and B2 based on the drive waveform of FIG. As shown in FIG. 4, the scanning trajectory B1 in the first frame period and the scanning trajectory B2 in the second frame period are slightly shifted in the radial direction, and the scanning trajectory B2 is between the scanning trajectories B1. As shown, the scanning trajectory B1 and the scanning trajectory B2 are complementary to each other. The scanning locus B3 is obtained by superposing the scanning locus B1 and the scanning locus B2. Thus, by combining the scanning trajectory B1 and the scanning trajectory B2, the apparent scanning density can be increased to approximately twice the individual scanning trajectories B1 and B2.

Here, the drive waveform generated by the drive waveform generation unit 7 differs only in phase between the first frame period and the second frame period, and in other parameters (drive frequency, envelope shape and cycle). It is preferable that they are the same. By doing so, the operation of the scanner 12 can be further stabilized between the first frame period and the second frame period, and the image quality can be stabilized.

The light detection unit 9 receives the observation light (for example, reflected light of the illumination light) generated in the subject A by the illumination light irradiation, and the light detection detects the observation light received by the light reception unit 15. And an analog / digital (A / D) converter 17 for converting an output signal from the photodetector 16 into a digital value.

The light receiving unit 15 is, for example, an optical fiber that extends from the distal end of the scope 2 a to the photodetector 16, and guides the received observation light to the photodetector 16.
The photodetector 16 detects the observation light received from the light receiving unit 15 and outputs an electric signal corresponding to the intensity of the detected observation light. The electrical signal is converted into a digital value by the A / D converter 17 to obtain a gradation value that is a value of each pixel of the image.

The image processing unit 10 includes an image construction unit 18 that constructs an image by two-dimensionally arranging the gradation values received from the A / D converter 17, and a storage unit 19 that temporarily stores the constructed image. And a black point processing unit 20 for interpolating the gradation value of the defective pixel in the constructed image.

The image construction unit 18 calculates coordinates corresponding to the illumination light irradiation positions on the scanning traces B <b> 1 and B <b> 2 from the drive waveform received from the drive waveform generation unit 7. Then, the image construction unit 18 forms a two-dimensional image of the subject A by assigning the gradation value received from the A / D converter 17 to the coordinates corresponding to the gradation value. Here, during the set of frame periods, a set of images including the first image in the first frame period and the second image in the second frame period is generated. The image construction unit 18 transmits the formed image to the storage unit 19 and the black spot processing unit 20.
The storage unit 19 stores at least a pair of two-frame images acquired immediately before from the images received from the image construction unit 18.

The black spot processing unit 20 determines whether or not there is a missing pixel having no gradation value in one frame image received from the image construction unit 18. The defective pixel occurs because the scanning locus B1 or B2 never passes through the area on the subject A corresponding to the pixel. If there is no missing pixel, the black spot processing unit 20 transmits the image to the output unit 11 as it is. On the other hand, when there is a missing pixel, the black spot processing unit 20 performs an interpolation process for interpolating the tone value of the missing pixel using the tone value in the image stored in the storage unit 19, and then the image Is transmitted to the output unit 11. A defective pixel having no gradation value appears as a black dot in the image displayed on the display 4. Such black spots can be prevented by interpolation processing.

In the interpolation processing, the black point processing unit 20 is different from the image of the interpolation processing target among the two-frame images stored in the storage unit 19 and acquired in the past, preferably the group acquired immediately before. Select an image. For example, if the image to be interpolated is the first image, the immediately preceding set of second images is selected. Since the scanning trajectories B1 and B2 in the first image and the second image are different from each other, the first image and the second image may have gradation values at the coordinates of different positions. Next, the black spot processing unit 20 selects a gradation value having the same coordinates as the missing pixel P from the selected image, and assigns the selected gradation value to the missing pixel P, as shown in FIG. . If the selected image does not have a gradation value at the same coordinates as the defective pixel P, the gradation value at the coordinates closest to the coordinates of the defective pixel P is selected.
The output unit 11 receives an image frame by frame from the black spot processing unit 20 and outputs the received image frame by frame to the display 4 at a predetermined frame rate.

The drive waveform generation unit 7, the phase setting unit 8, and the image construction unit 18 are realized by a computer, for example. Specifically, the computer includes a processor such as a CPU (Central Processing Unit) and a storage device that stores a program. Processing by the phase setting unit 8 and the image construction unit 18 is executed.

Next, the operation of the optical scanning endoscope system 1 configured as described above will be described.
In order to observe the subject A using the optical scanning endoscope system 1 according to the present embodiment, as shown in FIG. 6, first, parameters for drive waveforms are set (step S1). Thereafter, generation of a drive waveform by the drive waveform generation unit 7 is started (step S2), and output of illumination light from the light source unit 5 is started, and illumination light is scanned on the subject A along the scanning trajectories B1 and B2. (Step S3). The observation light generated at the illumination light irradiation position is received by the light receiving unit 15 and detected by the photodetector 16, and a gradation value corresponding to the intensity of the observation light is converted from the A / D converter 17 into the image processing unit 10. To the image construction unit 18 (step S4).

The image construction unit 18 accumulates gradation values for one frame (step S5), and constructs an image of one frame from the accumulated gradation values (steps S6 and S7). Next, if there is a missing pixel in the constructed image, the tone value of the missing pixel is interpolated using the tone value in the image of the two frames of the set generated immediately before (step S8). ). Thereafter, the image is output to the display 4 via the output unit 11 (step S10). The generated image is stored for interpolation of the next missing pixel (step S9).

Steps S4 to S10 are repeated until a predetermined number of frames are generated (step S11). At this time, every time one frame period ends, the phase of the drive waveform is changed, so that the scanning locus of the illumination light is shifted so as to be positioned in the gap of the immediately preceding scanning locus. Thereby, the images in which the positions of the scanning trajectories B1 and B2 are shifted are sequentially displayed on the display 4.

As described above, according to the present embodiment, as in the interlace method, the two frames of images in which the scanning trajectories B1 and B2 are complementary and the black spots are eliminated are displayed on the display 4 in order. It is felt to the observer who observes that an image in which the density in the radial direction of the scanning trajectories B1 and B2 is doubled is displayed. Thus, there is an advantage that a substantially high-resolution image can be provided without reducing the frame rate.

In the present embodiment, the image generated by the image construction unit 18 is output frame by frame from the output unit 11 to the display 4. The synthesized image may be output from the output unit 11 to the display 4. In this case, as illustrated in FIG. 7, the image processing unit 10 further includes a synthesis unit 21 that synthesizes two frames of images constituting the same set. In the present modification, the frame rate of the image output from the output unit 11 to the display 4 is reduced by half compared to the above-described embodiment.

Even in this manner, it is possible to provide a high-resolution composite image synthesized from two-frame images in which the scanning trajectories B1 and B2 are complementary. Compared to the case where the length of one frame period is doubled and the number of revolutions (the number of vibrations of the drive waveform during one frame period) is doubled at the same drive frequency to increase the density of the scanning trajectory. As a result, the scanning distance of the illumination light becomes longer, so that the interval between the sampling positions of the observation light is increased. As described above, since the sampling positions are dispersed, the resolution can be further improved.

In this modified example, as shown in FIG. 7, instead of the synthesis unit 21 generating a synthesized image from the generated two-frame images, the image construction unit 18 sets the gradation values for two frames. The synthesis unit 21 may accumulate and directly generate a synthesized image from the gradation values for two frames received from the image construction unit 18.

In the present embodiment, it is preferable that the drive waveform generator 7 generates the drive waveform so that the drive waveform continues smoothly while vibrating at the boundary between two frame periods adjacent in the time axis direction.
In this way, the operation of the scanner 12 can be stabilized even when shifting from the frame period to the next frame period. When the drive waveform changes discontinuously at the boundary of the frame period, the operation of the scanner 12 may become unstable.

The operation of the scanner 12 can also be stabilized by providing a blank period for adjusting the drive waveform so that the drive waveform is smoothly continuous with the drive waveform of the next frame period between two adjacent frame periods. However, when such a blank period is provided, the frame rate decreases. According to this modification, stable operation of the scanner 12 can be ensured while maintaining the frame rate.

DESCRIPTION OF SYMBOLS 1 Optical scanning endoscope system 2 Endoscope 2a Scope 3 Control apparatus main body 4 Display 5 Light source part 6 Optical scanning part 7 Drive waveform generation part 8 Phase setting part 9 Photodetection part 10 Image processing part 11 Output part 12 Scanner 13 Condensing lens 14 D / A converter 15 Light receiving unit 16 Photo detector 17 A / D converter 18 Image construction unit 19 Storage unit 20 Black spot processing unit 21 Composition unit

Claims

A light source unit that outputs illumination light;
A drive waveform generator for generating a drive waveform;
An optical scanning unit that scans the illumination light output from the light source unit on the subject according to the drive waveform generated by the drive waveform generation unit;
A light detection unit that detects observation light generated in the subject by irradiation of the illumination light and obtains a gradation value based on the observation light;
The drive waveform generation unit is configured to give a phase difference to the drive waveform between the predetermined number of frames in a set of frame periods in which images of a predetermined number of two or more continuous frames are acquired in the time axis direction. A phase setting unit for setting the phase of the drive waveform to be generated;
The predetermined number of frames is generated during the set of frame periods by generating an image of one frame based on the gradation value obtained by the light detection unit during the period of one frame and the irradiation position of the illumination light. An image processing unit that obtains a set of images consisting of
The image processing unit determines whether or not the image includes a defective pixel having no gradation value. When the image includes a defective pixel, a set obtained in the past than the set including the image is determined. An optical scanning image forming apparatus that interpolates a gradation value of a defective pixel using a gradation value in an image to be configured.
2. The optical scanning image forming apparatus according to claim 1, wherein the phase setting unit sets the number obtained by dividing 360 ° by the predetermined number as the phase difference.
3. The optical scanning image forming apparatus according to claim 1, further comprising an output unit that outputs an image generated by the image processing unit for each frame.
The image processing unit generates a composite image by combining the images of the predetermined number of frames constituting the same set;
The optical scanning image forming apparatus according to claim 1, further comprising: an output unit that outputs the composite image generated by the image processing unit.
5. The optical scanning image forming apparatus according to claim 1, wherein the drive waveform generation unit generates the drive waveform smoothly continuously between two frame periods adjacent in the time axis direction.
6. The optical scanning image forming apparatus according to claim 1, wherein the drive waveform generation unit generates the drive waveform that is different only in the phase between the predetermined number of frames.
The drive waveform generation unit generates the amplitude-modulated drive waveform, and amplitude-modulates the drive waveform so that the envelope shape of the amplitude-modulated drive waveform is the same between the predetermined number of frames. The optical scanning type image forming apparatus according to claim 1.
8. The optical scanning image forming apparatus according to claim 1, wherein the drive waveform generation unit generates the drive waveform that is amplitude-modulated with a period equal to the frame period.
An optical scanning endoscope system comprising the optical scanning image forming apparatus according to any one of claims 1 to 8.