JP2004219546A - Automatic focusing device and automatic focusing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic focusing device capable of realizing accurate automatic focusing even on the fine character of a subject. <P>SOLUTION: The automatic focusing device is equipped with an automatic focusing part 15 which extracts a luminance signal from an image signal, calculates the intensity and the direction of the edge of each image on the basis of the luminance signal, selects a pixel where the pixels whose edge directions are the same and whose edge intensity is large do not exist in the pixels adjacent in the direction of the edge of the image out of the images whose intensity and direction have been calculated, and calculates an evaluation value by integrating the intensity of the edge of the selected pixel. The automatic focusing part 15 detects a position where the evaluation value becomes maximum by repeating processing to calculate the evaluation value every time a stepping motor driving circuit 21 step-drives a movable lens system 3b, and moves the lens system 3b to the detected position where the evaluation value becomes maximum. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an automatic focusing device and an automatic focusing method particularly suitable for an electronic endoscope.
[0002]
[Prior art and its problems]
In recent years, an electronic endoscope provided with an automatic focusing device has been developed. The focus detection of such a conventional automatic focus adjustment device utilizes a video signal captured by a CCD or CMOS type image sensor. For example, the focus adjustment is performed when the evaluation value regarding the high frequency component becomes the maximum as the focus position (focus position). The evaluation value was the sum of the luminance differences between adjacent pixels, and the position where the evaluation value was the maximum was the focus position.
However, in the detection of such an evaluation value, when a target to be observed is minute, it is difficult to detect a luminance difference of the target portion, and accurate focus detection cannot be performed.
[0003]
Patent Literature 1, Patent Literature 2, Patent Literature 3, Patent Literature 4, and the like are known as electronic endoscopes provided with an automatic focusing device.
[0004]
[Patent Document 1]
JP 2001-154085 A [Patent Document 2]
Japanese Patent Application Laid-Open No. 2000-5127 [Patent Document 3]
Japanese Patent Application Laid-Open No. 2000-197604 [Patent Document 4]
JP 2000-342533 A
[Object of the invention]
SUMMARY OF THE INVENTION An object of the present invention is to provide an automatic focus adjustment device and a focus adjustment method capable of performing accurate automatic focus adjustment even for minute features of a subject.
[0006]
Summary of the Invention
To achieve this object, the present invention provides an imaging optical system, an imaging unit that converts an optical image formed by the imaging optical system into an electric image signal, and a focus adjustment unit that adjusts a focus state of the optical image. Control means for step-driving the focus adjustment means between a farthest focus position and a closest focus position, extracting a luminance signal from the image signal, and calculating an image based on the luminance signal. Calculate the edge intensity and the edge direction, and from the image in which the intensity and the direction are calculated, a pixel adjacent to the edge direction has a pixel in which the edge direction is the same direction and the edge intensity is large. Focus detection means for selecting a non-existent pixel and calculating an evaluation value by integrating the intensity of the edge of the selected pixel, wherein the focus detection means calculates the evaluation value by the control means. The impatience Each time the driving unit is step-driven, the position at which the evaluation value becomes maximum is repeatedly detected, and the control unit drives the focus adjustment unit to a position at which the detected evaluation value becomes maximum. .
According to this configuration, since the evaluation value is obtained by excluding non-maximal pixels from the image obtained as the edge, it is possible to detect an accurate in-focus state.
[0007]
The edge intensity is calculated from the amounts of change Δx and Δy in the x direction and the y direction obtained by differentiating the pixel signal from the equation ((Δx) ² + (Δy) ² ) ^1/2
Can be obtained by
The direction of the edge of the image is calculated from the change amounts Δx and Δy in the x direction and the y direction obtained by differentiating the pixel signal from the expression Tan ⁻¹ (Δy / Δx).
Can be obtained by
The direction of the edge of the image is classified into a predetermined direction for each pixel, and the non-maximal pixels are eliminated by comparing the direction and intensity of the edge of the pixel in the classified direction of each pixel.
[0008]
The focus detection method of the present invention includes an imaging optical system, and a step of extracting a luminance signal from an image signal output from an imaging unit that converts an optical image formed by the imaging optical system into an electric image signal. Calculating the direction of the edge of the image and the intensity of the edge based on the luminance signal; and, from the image in which the edge and the intensity have been calculated, the pixels adjacent to the direction of the edge have the same edge direction in the same direction. And selecting a pixel in which a pixel having a large edge intensity does not exist, calculating an evaluation value by integrating the edge intensity of the selected pixel, and driving each of the steps by step driving the focus adjustment unit. And a step of driving the focus adjustment means to a position where the largest evaluation value is obtained among the evaluation values obtained in the repeating step. Having.
According to this method, since the evaluation value is obtained by excluding non-maximal pixels from the image obtained as the edge, it is possible to detect an accurate in-focus state.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of an electronic endoscope apparatus equipped with the automatic focusing apparatus of the present invention.
An example of the imaging optical system 3 will be described with reference to FIGS. As shown in FIGS. 2 and 3, a cylindrical fixed lens frame 3d is installed in the distal end portion of the insertion tube 1a of the endoscope 1, and the fixed lens system 3a of the imaging optical system 3 is fixed to the fixed lens frame 3d. I have. A cylindrical guide frame 3e is provided inside the fixed lens frame 3d, and straight guide grooves 3f and 3g are provided in the guide frame 3e in parallel with the optical axis (the left-right direction in FIG. 2).
[0010]
A movable lens frame 3h is movably fitted in the optical axis direction inside the guide frame 3e, and is provided between a key 3i of the movable lens frame 3h and a linear guide groove 3f of the guide frame 3e (located at an upper portion in FIG. 2). The movable lens frame 3h is linearly moved along the optical axis without being rotated around the optical axis by the fitting. A ring 3j is fitted on the outside of the guide frame 3e, and the ring 3j and the movable lens frame 3h are fastened by screws 3k penetrating through the linear guide grooves 3g (located at the lower part in FIG. 2) of the guide frame 3e. I have. The cylindrical member 31 having a spring seat 3n at the rear end thereof and fitted between the imaging element frame 3p and the guide frame 3e allows the screw 3k to move in the optical axis direction, not shown. A slit is provided.
[0011]
The stepping motor 4 of FIG. 1 is linked to the flange 3m of the ring 3j via a connecting member 4a, and the movable lens frame 3h is linearly moved in the optical axis direction by the stepping motor 4. A movable lens system 3b is mounted on the movable lens frame 3h so as to face the fixed lens system 3a, and the relative distance between the fixed lens system 3a and the movable lens system 3b is varied by linearly moving the movable lens frame 3h. In this embodiment, the focus is adjusted by changing the relative distance between the fixed lens system 3a and the movable lens system 3b.
A spring 3o is provided between the ring 3j and the spring seat 3n to push the movable lens frame 3h in a direction in which the movable lens system 3b approaches the fixed lens system 3a. The spring 3o keeps the movable lens frame 3h fixed at all times. When the force on the movable lens frame 3h by the stepping motor 4 is released, the movable lens frame 3h is pushed back to the fixed lens system 3a to move the movable lens system 3b to a predetermined initial position. (FIG. 3A). This initial position is the furthest in-focus position and also the pan focus position.
[0012]
An image sensor frame 3p is fitted inside the guide frame 3e. An image sensor 3c is mounted on the image sensor frame 3p so as to face the movable lens system 3b. A spring 3r is provided between the image sensor frame 3p and the movable lens frame 3h. The spring 3r urges the movable lens system 3b in a direction away from the image sensor 3c, and the movable lens system 3b The contact of the light receiving surface of 3c is prevented. 19 is an O-ring shaped sealing material.
[0013]
The imaging element frame 3p includes a circuit such as an imaging element driving circuit 3s that drives and controls the imaging element 3c to convert an optical image formed by the imaging optical system 3 into an electric electric signal and output the electric signal, and an imaging element 3c. A signal cable 3t for transmitting an electric signal from the device is mounted. The signal cable 3t transmits an electric video signal from the image sensor 3c to a first-stage video signal processing circuit 6 described later, and transmits a signal processed by the first-stage video signal processing circuit 6 to the image sensor driving circuit 3s and the like. It is interactive. Bidirectional signal transmission is controlled under time control by a timing controller 10 described later.
[0014]
A light transmission path 5 is installed in the insertion tube 1 a adjacent to the imaging optical system 3. The light transmission path 5 acts as a light guide path for guiding the illumination light emitted from the light source unit 9 to the insertion tube 1a, and illuminates the subject through a light transmission lens 5a located at the distal end face of the insertion tube 1a. Since the insertion tube 1a is inserted into a body cavity, the insertion tube 1a needs to have a small diameter, and the light transmitting path 5 is desirably configured as an optical fiber bundle in which optical fibers are bundled.
[0015]
On the other hand, as shown in FIG. 1, the processor 2 includes a first-stage video signal processing circuit 6, an image memory 7, a second-stage video signal processing circuit 8, a light source unit 9, a timing controller 10, a system controller 11, A panel switch 12 and a power supply unit 13 are mounted.
[0016]
The first-stage video signal processing circuit 6 processes the electric signal from the image sensor 3c transmitted through the signal cable 3t, and stores the video signal in the image memory 7 sequentially. Further, the subsequent-stage video signal processing circuit 8 sequentially reads out the video signals stored in the image memory 7, performs signal processing, and outputs the video signals to the monitor 14. The monitor 14 displays the video signal output from the subsequent video signal processing circuit 8 on the display surface as a visible image.
[0017]
The system controller 11 controls the overall operation of the processor 2 based on command data input from a keyboard 11a and a program stored in a memory (not shown). The timing controller 10 controls the processor based on a command from the system controller 11. 2 controls the operation timing of each block. The processor 2 receives power supply from the power supply unit 13.
[0018]
The light source unit 9 detects a brightness of a captured image based on an electric signal from the image sensor 3c transmitted through the signal cable 3t, obtains a dimming control signal, and opens and closes a diaphragm based on the dimming control signal. The light from the light source is passed through an aperture whose degree of opening and closing is controlled, and the light whose amount is adjusted is output to the light transmission path 5.
[0019]
The automatic focus adjustment unit 15 includes a focus detection circuit unit, and controls the drive of the stepping motor 4 via the stepping motor drive circuit 21 based on the focus state detected by the focus detection circuit unit. The stepping motor drive circuit 21 drives the stepping motor 4 to move the movable lens system 3b within the variable focus adjustment range of the imaging optical system 3.
[0020]
The automatic focus adjustment unit 15 adjusts the focus of the imaging optical system 3 by obtaining an evaluation value from an image signal output from the imaging element 3c. The luminance signal is extracted from the pixel signal (color signal) output from the image sensor 3c according to the degree of focusing, and the edge direction and intensity of each pixel are detected, and the non-maximal point is ignored (excluded) and integrated. By repeating the process of using the integrated value as the evaluation value while moving the movable lens system 3b, a position where the evaluation value becomes maximum is searched.
[0021]
As shown in FIGS. 3A and 3C, when the movable lens system 3b is at the furthest focus position or the closest focus position, and the focus position is located behind or in front of the image sensor 3c, FIG. As shown in (a) and (c), the high frequency component of the image signal output from the image sensor 3c decreases, and the evaluation value also decreases. When the focal point of the imaging optical system 3 coincides with the light receiving surface of the imaging device 3c, as shown in FIG. 3B, the high frequency component of the image signal output from the imaging device 3c increases, and the evaluation value increases.
[0022]
The automatic focus adjustment unit 15 uses the fact that the evaluation value of the image signal output from the image sensor 3c increases or decreases in accordance with the degree of focus (the amount of defocus) of the imaging optical system 3 to make the movable lens system 3b farthest. The evaluation value is obtained while step-driving from the in-focus position to the closest in-focus position, and the movable lens system 3b is moved to that position when the in-focus state is determined when the evaluation value is the maximum.
[0023]
Next, the focus detection operation of the electronic endoscope will be described with reference to the flowcharts shown in FIGS. 4 to 7 and the pixel arrangement example shown in FIG. These flowcharts are processes controlled by the system controller 11.
FIG. 4 is a main flowchart relating to the focus detection processing. This process is executed when the focus adjustment start button is turned on. In this specification and the drawings, steps are abbreviated as "S".
[0024]
In this process, first, 0 is substituted (cleared) into the maximum evaluation value Cmax which is a variable (S101). Next, the movable lens system 3b is moved to the farthest focusing position (S103). Then, while moving the movable lens system 3b from the furthest focus position to the closest focus position, the process enters a loop process of S105 to S115 in which a focus position at which the evaluation value becomes maximum is searched.
[0025]
When entering this loop, first, it is checked whether or not the movable lens system 3b is at the closest focusing position (S105). When the movable lens system 3b is located at the closest focus position (when the search processing has been completed), the process exits this loop processing (S105; Y, S117).
When the movable lens system 3b is not at the closest focus position, an image is acquired, that is, an image signal is input from the image sensor 3c, and an image signal (digital data of each pixel) in one screen written in the image memory 7 is obtained. It is read (S105; N, S107). Then, the direction of the edge and the intensity of the edge are detected from the image within the predetermined screen range, and the evaluation value is calculated by integrating the difference (of the intensity) ignoring the non-maximal point (S109).
[0026]
If the image signal is a color image in the RGB format, the R signal can be used as it is as the luminance, the difference in the intensity of each of the R, G, and B signals can be integrated, and the sum thereof can be used as the evaluation value. Alternatively, the RGB signal may be gray-scaled to obtain luminance.
Further, it is preferable that the area within the predetermined screen is within the focus detection area. In this case, the evaluation value is calculated within the image (trimming) cut out from the area.
[0027]
The calculated evaluation value is compared with the maximum evaluation value Cmax. If the calculated evaluation value is larger, the calculated evaluation value is substituted for the maximum evaluation position Cmax, and the process proceeds to S115 (S111; Y, S113, S115). If the calculated evaluation value is not larger, the process skips S113 and proceeds to S115 (S111; N, S115). In S115, the movable lens system 3b is moved one step closer to the nearest side, and the process returns to S105 (S115, S105).
[0028]
The above processing is repeated until the movable lens system 3b reaches the closest focusing position (S105; N, S107). When the processing at the closest focus position is completed, the movable lens system 3b is moved to the position where the maximum evaluation value Cmax is obtained, and this processing is completed (S105; Y, S117, END).
[0029]
As described above, since the edge of the image is detected, and the evaluation value is integrated while excluding the non-maximum value from the detected edge, accurate focus adjustment can be performed even on a minute subject. Note that a method for obtaining an edge is known, and is disclosed, for example, in "IEICE Technical Report PRMU98-9".
[0030]
Next, the evaluation value calculation process executed in the process of S109 will be described in more detail with reference to the flowchart shown in FIG.
In this flowchart, first, an edge strength image is generated based on the image signal, and further, an edge direction image is generated (S201). Then, a non-maximum suppressed edge strength image is created from the edge strength image and the edge direction image (S203).
[0031]
Then, after initializing the evaluation value C (substituting 0) (S205), the evaluation value C integrating process is performed for each pixel. That is, the edge strength obtained for each pixel is added as the evaluation value C (S207; N, S209, (S211), S207). The process is repeated until the edge intensities of all the pixels in the predetermined range are added (S207; N, S209, S211), and the process returns when the edge intensities of all the pixels are added to obtain the evaluation value C (S207; Y, RETURN).
As described above, in the evaluation value calculation processing, the sum of the edge intensities of the pixels is obtained as the evaluation value C for the edges of the image in the predetermined range. For the same subject, the larger the sum of the edge intensities, the smaller the defocus amount and the state closer to the focused state, and the smaller the sum of the edge intensities, the larger the defocus amount.
[0032]
The details of the edge strength image and edge direction image generation processing in S201 will be described with reference to the flowchart shown in FIG.
When entering this process, first, it is checked whether or not the processes after S303 have been completed for all pixels (S301). The loop processing of S303 to S317 is repeated until the processing is completed for all the pixels (S303; N, S305), and when the processing is completed for all the pixels, the processing returns (S301; Y, RETURN).
[0033]
In S303, the edge strength image Iamp, which is the magnitude of the gradient (the strength of the edge of the image), is calculated by the formula ((Δx) ² + (Δy) ² ) ^1/2.
Ask by
Δx is the amount of change in the x-axis direction of the image signal,
Δy is the amount of change in the y-axis direction of the image signal.
In the present embodiment, with each pixel within a predetermined range as an origin, a change in luminance in the x and y directions is obtained by a differentiation method, and this is used as an edge intensity image Iamp.
[0034]
Next, the direction Dir of the gradient (the direction of the edge) of the image is calculated by the expression Tan ⁻¹ (Δy / Δx).
(S305). Note that the gradient direction Dir is a real number satisfying 0 = <Dir <180.
[0035]
Then, the direction of the gradient Dir of each pixel, that is, the edge direction image Idir is classified into four directions (S307, S309, S311, S313, S315).
In the present embodiment, Idir is set to 0 in the case of Dir => 157.5 and Dir <22.5 (S309),
If 22.5 = <Dir <67.5, the edge direction image Idir is set to 45 (S311),
If 67.5 = <Dir <112.5, the edge direction image Idir is set to 90 (S313),
If 112.5 = <Dir <157.5, the edge direction image Idir is set to 135 (S315).
Set each. After setting, the next pixel is selected and the process returns to S301 (S317, S301).
[0036]
Next, the details of the non-maximum suppression edge strength image generation processing in S203 will be described with reference to the flowchart shown in FIG.
In this process, first, it is checked whether or not the process after S403 has been completed for all pixels (S401). Then, the loop processing of S403 to S411 is repeated until the processing is completed for all pixels (S401; N, S403), and when the processing is completed for all pixels, the processing returns (S401; Y, RETURN).
[0037]
In step S403, the edge direction of the center pixel is compared with the edge direction of the neighboring pixel of interest. The target neighboring pixel is a pixel in the direction set in any of S309 to S315. FIG. 8 shows the positional relationship between the center pixel and the neighboring pixel of interest.
If the edge directions of the central pixel and the neighboring pixel of interest are not the same, the next pixel is selected and the process returns to S401 (S405; N, S413, S401). If they are not the same, the luminance change direction changes between the central pixel and the neighboring pixel of interest, so it can be assumed that they are different edges. When the edge direction of the center pixel is the same as the edge direction of the neighboring pixel of interest, it is a state where the luminance change continues in the edge direction.
[0038]
When the edge directions are the same, the edge intensity of the center pixel and the edge intensity of the neighboring pixel of interest are compared (S405; Y, S407). If the edge intensity of the central pixel is higher, the next pixel is selected and the process returns to S401 (S409; N, S413, S401). If the edge intensity of the central pixel is lower, 0 is inserted into the edge intensity image Iamp. The process returns to S401 (S409; Y, S411, S413, S401). When the edge intensity of the central pixel is higher, it can be estimated that the central pixel is the edge peak, and when the edge intensity of the central pixel is lower, the central pixel is not the edge peak.
[0039]
As described above, in the present embodiment, the edge intensity image Iamp is set to 0 when the edge intensity of the center pixel is smaller.
[0040]
The processing of S401 to S413 described above is performed for all pixels, and the process returns (S401; Y, RETURN). By the non-maximum suppression edge strength image generation processing, the non-maximum value is removed from the edge strength image, that is, the edge image, of each pixel, the edge width is narrowed, and highly accurate edge detection can be performed.
The evaluation value is higher as the sum of the edge intensity images Iamp obtained in this way is larger and closer to the in-focus state, so the maximum evaluation value is the in-focus position.
[0041]
As described above, in the present embodiment, the evaluation value is obtained by using the pixels in the central predetermined range in the image signal obtained from the image sensor 3c. However, the range is arbitrary, and the user can select the range. You may comprise.
[0042]
Further, in the present embodiment, the focus is adjusted by moving the movable lens system 3b in the optical axis direction. However, the focus may be adjusted by moving the imaging element 3c in the optical axis direction.
[0043]
Although the above-described embodiment has exemplified the electronic endoscope device used in the medical field, it can be similarly applied to a device having a similar function used in the industrial field, and It is not limited to one.
[0044]
【The invention's effect】
As is apparent from the above description, the apparatus and method of the present invention detect the direction and intensity of the edges of the image while driving the focusing means, and eliminate the non-maximal points from the changes in the direction and intensity of the edges of the image. The integrated value is used as the evaluation value to determine the maximum evaluation value, and the focus adjustment means is driven when the maximum evaluation value is obtained, enabling accurate automatic focus adjustment even for minute feature points. Become.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of an electronic endoscope apparatus equipped with an automatic focusing apparatus of the present invention.
FIG. 2 is a longitudinal sectional view of the insertion tube, showing a relationship between an imaging element and an imaging optical system at a distal end portion of the insertion tube of the electronic endoscope device.
FIG. 3 is a diagram showing a relationship between the imaging device and an imaging optical system.
FIG. 4 is a diagram showing a main flowchart relating to focus detection processing.
FIG. 5 is a flowchart showing details of an evaluation value calculation process in the main flowchart.
FIG. 6 is a flowchart showing details of an edge intensity image and edge direction image generation process during the evaluation value calculation process.
FIG. 7 is a flowchart showing details of a non-maximum suppression edge strength image generation process during the evaluation value calculation process.
FIG. 8 is a diagram illustrating a state of a pixel near a target pixel.
[Explanation of symbols]
Reference Signs List 1 endoscope 2 processor 3 imaging optical system 3a fixed lens system 3b movable lens system (focusing lens)
4 Stepping motor 7 Image memory 11 System controller (computing means)
15 Automatic focus adjustment unit 21 Stepping motor drive circuit

Claims (7)

An imaging optical system, an imaging unit that converts an optical image formed by the imaging optical system into an electric image signal, and a focus adjustment unit that adjusts a focus state of the optical image.
Control means for step-driving the focus adjustment means between a farthest focus position and a closest focus position,
A luminance signal is extracted from the image signal, the intensity of each pixel edge and the direction of the edge are calculated based on the luminance signal, and a pixel adjacent to the direction of the edge is calculated from the image in which the intensity and the direction are calculated. A focus detection unit that selects a pixel in which the direction of the edge is the same direction and there is no pixel having a large edge intensity, and calculates an evaluation value by integrating the edge intensity of the selected pixel;
The focus detecting unit detects the position where the evaluation value is maximum by repeating the process of calculating the evaluation value each time the control unit drives the focus driving unit in a stepwise manner. An automatic focus adjustment device, wherein the focus adjustment means is driven to a position where the evaluated value becomes maximum. The focus detecting means calculates the intensity of the edge of the image from the amounts of change Δx and Δy in the x direction and the y direction obtained by differentiating the pixel signal.
((Δx) ² + (Δy) ² ) ^1/2
2. The automatic focus adjustment device according to claim 1, wherein The focus detection means calculates the direction of the edge of each pixel from the change amounts Δx and Δy in the x direction and the y direction obtained by differentiating the image signal from the expression Tan ⁻¹ (Δy / Δx).
2. The automatic focus adjustment device according to claim 1, wherein The focus detection means classifies the direction of the edge of the image into a predetermined direction for each pixel, and compares the direction and the intensity of the edge of the pixel in the classified direction of each pixel to eliminate non-maximal pixels. The automatic focusing device according to claim 3, wherein 2. The automatic focusing apparatus according to claim 1, wherein the imaging optical system includes a movable lens system that adjusts a focus by moving in an optical axis direction, and the focus adjustment unit adjusts a focus by moving the movable lens system in an optical axis direction. Adjustment device. 2. The automatic focusing apparatus according to claim 1, wherein the imaging unit moves in the optical axis direction with respect to the imaging optical system, and the focus adjustment unit moves the imaging optical system in the optical axis direction to adjust the focus. An imaging optical system, and extracting a luminance signal from an image signal output from an imaging unit that converts an optical image formed by the imaging optical system into an electric image signal;
Calculating the direction of the edge of the image and the intensity of the edge based on the luminance signal;
From the image of the calculated edge and intensity, in the pixels adjacent to the direction of the edge, the direction of the edge is the same direction, the step of selecting a pixel in which there is no large pixel of the edge intensity,
Calculating an evaluation value by integrating the intensity of the edge of the selected pixel,
Repeating each of the steps while step-driving the focus adjustment unit; and driving the focus adjustment unit to a position where the largest evaluation value is obtained among the evaluation values obtained in the repetition step. And an automatic focusing method.

Images