JPS61210310A - Auto-focusing device - Google Patents

Abstract

PURPOSE:To focus an optical system with a high reliability by detecting a position, where two-stage difference signals of the variance of a luminance standard deviation are most symmetrical, as the focus position. CONSTITUTION:A photoelectric converting means like an ITV camera is used for the optical system to obtain the luminance signal of the image of an object, and the luminance standard deviation is obtained from this signal. The luminance standard deviation of an image obtained in each position is obtained while moving the position of the stage of a microscope or the lens of a camera used as the optical system at a certain pitch in steps. The position where two- stage difference signals of the variance of the luminance standard deviation are most symmetrical is detected as the focus position. Since many samples can be photographed automatically, the inspection for quality control or the like is performed very efficiently.

Description

[Detailed description of the invention] [Industrial application field] The present invention mainly relates to melting, refining, and rolling of metals.

This invention relates to an automatic focusing device I for efficiently carrying out quality control, inspection observation, etc. using a single image in the fields of processing, heat treatment, and inspection of ultra-LSIs, semiconductors, microorganisms, etc.

[Conventional technology]

Regarding automatic focusing devices, several methods have been proposed in the past, and they can be roughly divided into the following methods.

(1) A method of mechanically measuring the distance between the image target surface and the lens (eg, Japanese Patent Application Laid-Open No. 107825/1982).

(2) A method of applying focusing light to one image target surface and detecting the in-focus position (e.g., Japanese Patent Application Laid-open No. 1965-1965) (3) Detecting the in-focus position from information on the incident light itself (Example: Japanese Unexamined Patent Publication No. 51-5662).

However, the method (1) above has the problem that when used for focusing a microscope, an oil immersion microscope cannot be used. Furthermore, it is extremely difficult to add a new autofocus function to the optical systems of existing microscopes, ITV cameras, and the like.

In addition, method (2) above uses the principle of triangulation,
The light projecting means, the light receiving means, and the photographing lens are arranged separately, and this method cannot be used for oil immersion microscopes, which has the same problem as method (1) above. Since the invention uses open-loop control for the optical system, errors in component processing cannot be absorbed, and there is no countermeasure against left-right asymmetric brightness standard deviation.

Therefore, the method (3) is more advantageous than the methods (1) and (2) above.

On the other hand, as the method (3), there is a method of determining the focus position from the maximum value of the luminance standard deviation of the A1 image (eg, Japanese Patent Laid-Open No. 56628/1983).

(A method of determining the focus position from the maximum value of the harmonic power of the B1 image (e.g., Japanese Patent Application Laid-Open No. 1983-40633) is known. The prerequisite is that the contrast is maximum when the system is in focus, and this evaluation is based on the fact that the photoelectric conversion means converts light into an electrical signal, and the variation in the signal level or the power of the harmonic component is evaluated. This is done by

Generally, as shown in FIG. 2, at the focus position, the variation (luminance standard deviation) or the power of the harmonic component takes a maximum value, and the focus position is determined from this maximum value.

[Problem that the invention seeks to solve]

In the prior art, when an ITV camera or the like is used as a photoelectric conversion means to examine changes in the luminance standard deviation and perform focusing, the following problem occurs.

(If the amount of light that exceeds the dynamic range of the photoelectric conversion means is incident on the light receiving element, the output signal will be saturated. Therefore, the output signal level of the part shown by the line AA' in Figure 3 (at) Figure 3 (as shown in bl, B8 is lower than the signal replum that should be obtained corresponding to the brightness)
becomes.

If the image in Figure 3 (al) is blurred, the output signal on the B-8' line of Figure 4 (al) will be as shown in Figure 4 (al). Similarly, saturation occurs at the L! level. However, the range in which the h level signal is output on the When considering that Figures (a) and 4 (al) are composed of a large number of pixels, the frequency distribution of the output signal becomes as shown in Figure 3 (cl) and Figure 4 (cl), respectively, and the luminance As shown in FIG. 5, the standard deviation value becomes larger when the image is blurred.

(As shown in Fig. 6 (al), the image at one in-focus position l is outside the photoelectric conversion plane, and as shown in Fig. 6 (bl), the image at the time of blur is the spread width of the blurred point spread function. As a result, the object image A, which was outside the photoelectric conversion surface when in focus, turns into a blur and enters the photoelectric conversion surface when it is blurred.Therefore, in the example of FIG. 6, the luminance standard deviation is as shown in FIG. σ increases as the distance from the in-focus position increases, and the brightness standard deviation does not reach its maximum value at the in-focus position. (c) When using a microscope as an optical system, the blurring of the image is due to the fact that the object moves away from the in-focus position. Symmetry is not necessarily exhibited between when it is shifted toward the front and when it is shifted far away, and there are cases where the luminance standard deviation σ changes as shown in Figure 8.
In the figure, the inflection point of σ coincides with the in-focus position), and if the phenomenon of (al * (bl) occurs simultaneously, the luminance standard deviation becomes more complicated as shown in Figure 9. showing change,
In such cases.

The brightness standard deviation does not necessarily show the maximum value at the in-focus position.

(Screen shake due to vibration during movement of the microscope stage and noise superimposed on the video signal are superimposed as noise on changes in the calculated luminance standard deviation, so the positions where the luminance standard deviation takes the maximum value are tends to fluctuate due to noise.

The problem of earth mud (al, (bl, (cl, di)) occurs not only when determining the focus position from the luminance standard deviation, but also when determining the focus position from harmonic power.

This is an essential problem when detecting the focal position using an image.

Therefore, conventional image focusing methods may not be of practical use when high-quality focusing is required.

[Means for solving problems]

This invention was made in view of the above-mentioned problem,
Even in cases where there is saturation of the photoelectric conversion device, the influence of object images outside the field of view, asymmetry of blur with respect to the focal position, etc., the brightness standard deviation of the image obtained by changing the distance between the photoelectric conversion device and the object Focusing on the property that the second-order difference signal of a change in brightness is always symmetrical with respect to the focus position, the position where the second-order difference signal of a change in luminance standard deviation has the highest degree of symmetry is detected as the focus position. This allows for highly reliable focusing.

This invention provides ITV for optical systems such as microscopes.
Obtain the luminance signal of the image of the subject using a charge conversion means such as a camera, and calculate the luminance standard deviation from this signal. In the case of a system, the brightness standard deviation σ of the image obtained at each position is determined while moving the position of the lens in steps at a constant pitch. The brightness standard deviation is generally the amount that shows the maximum value at the in-focus position, and this can be proven as follows.

Brightness distribution of the image at the in-focus position lζ: f(x-y) Brightness distribution of the image at the out-of-focus position: g(x-y) Blurred point spread function at the out-of-focus position: h(x-y) (where ,
h(x-y) changes depending on the amount of deviation from the in-focus position) and the brightness distribution I of the blurred image can be expressed as a function of h and f as in equation [1].

At this time, the image is out of focus! The difference between σi, the square of the standard deviation of the brightness distribution of H'(x-y), and σ2, the square of the standard deviation of the brightness distribution of the image at one focusing position, is calculated by I under the condition of equation (2).
This is expressed by equation (4) and becomes a non-negative value under the condition of equation (5).

[Condition 1] Here, [Equation 2] indicates the condition that the total amount of incident light to the photoelectric conversion surface when out of focus is the same as the total amount of incident light when in focus, and Equation (2) indicates that the blur is a spatial low-pass These conditions generally indicate that the filter is cloudy and that other optical systems such as microscopes
Therefore, as shown in formula (4), the brightness standard deviation σ of the image is maximum at the time of focus, but in reality, as mentioned earlier, The relationship does not hold in the following cases.

(1) Saturation of the photoelectric conversion device: In deriving tζ of equation (4), calculate fcx-y) and 1x-y) as values that can take the value 0-(1) in proportion to the value of the amount of incident light. However, since the dynamic range of an actual photoelectric conversion device is finite, all values greater than or equal to the saturation shikila ζ are evaluated as an obscuration. Therefore, when saturation occurs, the relationship in equation 4 does not necessarily hold.

(2) Influence of object images outside the field of view: [For the derivation of the equation, the fields of view x and y of the photoelectric conversion device are calculated as encompassing the range <ow, but A photoelectric conversion device has a finite field of view. Therefore, when calculating the brightness standard deviation when out of focus, the component of the image that is blurred outside the field of view is ignored, and the component that is blurred within the field of view of the image that is outside the field of view is taken into account, so (4) There are cases where the relationship in the formula does not hold true.

(3) Asymmetry of bokeh with respect to the in-focus position: The total amount of light incident on the photoelectric conversion surface when out of focus does not exactly match the total amount of light incident on the photoelectric conversion surface when in focus, and with respect to the in-focus position. There is a difference between when the image is blurred in the foreground and when it is blurred in the distance. Therefore, (2
) does not hold strictly true, and the relationship in formula (4) may not hold.

With these problems as a background, the inventors repeated various experiments, and as a result of comparing and contrasting them in FIG. 2
We found that the floor difference value is symmetrical with respect to the in-focus position. Then, in the second difference value of the change in luminance standard deviation,
In order to find the most symmetrical position as the in-focus position, a new evaluation function U (zl) shown below was introduced.

-iQ'a Microscope stage or optical system lens scanning range This evaluation function U (zl always takes the maximum value at the position where σ is most line symmetrical with respect to zEC.

The focusing position can be found as the position where January 21 is the maximum ([Jz) means the cross-correlation function between σ'(2) and σ#(to))6 Note that U(zl is It may be a vine function that evaluates the symmetry of the second-order difference value of the luminance standard deviation.

■It is not limited to formulas.

[Effect]

By focusing a microscope, ITV camera, etc. using an apron, even an unskilled person can do the focusing work correctly and quickly.

Since it becomes possible to automatically photograph a large number of samples during manufacturing and inspection processes, inspections for quality control, etc. can be carried out extremely efficiently.

〔Example〕

A specific example based on the above-described method will be described below.

FIG. 1 is a block diagram when this device is implemented in a microscope. In the figure, (1) is a microscope, and (
2) is a sample. Sample (2) is on the microscope stage (3)
The W4 microscopic stage (
3) by moving it up and down in the optical axis direction (two-axis direction) of the microscope. In addition, the sample (2) is exposed to the light projecting means (4),
(5) Although epi-illumination is used in this embodiment, the light projection means of the microscope is not limited to the one in this embodiment. (4) is a light source, and (5) is a half mirror or prism used to change the optical path of the light source (4).

The microscope stage (3) is equipped with a Z stage motor controller (6) and an XYY stage tap roller (7).
Therefore, by driving the Z stage movement motor (8) and the ). The 2-stage motor controller (6) and the UXY stage motor controller (7) convert the set moving distance signal into a signal that drives the moving motors (8) and (9).
This is a circuit for driving the motor by a set value.

al is a photoelectric conversion means installed on the imaging plane of the microscope (1), which is formed from a large number of photoelectric conversion elements and transfers it to the sample (2).
) is output as an analog time-series signal, and an ITV camera or the like corresponds to this. (11) is a synchronization signal separation circuit,
This is a means for separating a synchronization signal from the luminance time series signal obtained from the photoelectric conversion means α and supplying the synchronization signal to the focusing control circuit (I).

@ gives a signal to the Z stage motor controller (6) to move the position of the microscope stage (3) upward or downward at a constant pitch according to the synchronization signal obtained from the synchronization signal separation circuit αυ, and also controls the focus judgment circuit. After the in-focus position is detected by the focus determination circuit (13), the position (3) of the focus position signal detected by the focus determination circuit (13) is set to the in-focus position. Focusing control that has the function of setting and focusing, and if necessary, arbitrarily controlling the position of the microscope stage (3) in the X and Y directions to focus at any position on the sample (2). It is a circuit.

Note that the synchronization signal is not limited to the method of obtaining the synchronization signal of the charging conversion means Ql by separating it by the synchronization signal separation circuit (11), but a synchronization signal generation circuit is provided separately, and this signal is used to obtain the synchronization signal of the photoelectric conversion means α1. There is also a method of driving the focus control circuit @, and in this case, the above-mentioned synchronization signal generation circuit υ is unnecessary.

a3 is a focus determination circuit, I, α9. αQ, αη, (L8
．． (11, (a) It is constructed by the circuit shown in FIG. 1, and each stage position Z obtained by the photoelectric conversion means
The focus position is determined from the signal at i, and the focus position signal is sent to the focus control circuit (L), the specific circuit configuration of which is shown below.

Numeral 4 is an analog/digital conversion circuit, which converts the photoelectrically converted analog output amount into a digital amount in order to accurately and quickly calculate a large amount of luminance signals obtained from the photoelectric conversion means. The luminance standard deviation calculation circuit is a circuit for calculating the luminance standard deviation σ (Zl) at each stage position at the time of photoelectric conversion, and the luminance standard deviation σ is calculated using a well-known formula. : Number 1m to distinguish the photoelectric conversion element part
: Digital value of the photoelectric conversion output of the FIL-th photoelectric conversion element M: Obtained from the total number of photoelectric conversion elements.

αe is the focus measurement range (stage movement range) −N b 14;=Zi 4 NΔZ(ix−N,・
..., 0. ..., N)... with the proviso aZ:
After determining the brightness standard deviation σ(Zi) at each position of the scanning pitch in the Z direction.

This is a noise removal circuit for improving the accuracy when calculating the second-order difference value σ'' (Zi) of the luminance standard deviation.

Noise factors include stage vibration during stage movement, and in order to reduce the influence of fluctuations in σ due to these factors, the noise removal circuit αe applies a moving average to σ(Zi). Sumi η performs second-order difference on σ(Zl) obtained by the noise removal circuit αQ.

This is a luminance standard deviation second-order difference calculation circuit that calculates a second-order difference value σ' (Zl) of the luminance standard deviation. (Lυ is σ”(Zl
) is a circuit for determining the left and right line symmetry, and the left and right degree of symmetry is calculated based on the formula 0. However, Zi takes a discrete value, and ζn= n '' 7!IZ (n=-N, ・-, 0,-
・-, N) """""" (If a is the evaluation function U(3) of the symmetry degree of the diagram, then (7).

From formula [8], it can be expressed as, for example. ■As shown in equation 11rIA), each stage position ζn (n=-Nj-'-go, -, N)s
σ′(ζn)(n”-Ns””s Os””a
N), σ in Figure 11(b) is shifted by Zl.
'(ζn+Zi) and 11rIA(
It is the product of σ" (-ζn-Zl), which is obtained by shifting the left-right folding function by Δ in a) with ζn5w0, and 2!
l for 1 r 1--N,-,O,,,,N t
Teffi conversion g Seta time (7) value U (Zl) C 11th
As shown in the figure (dl), the cross-correlation between σ'(ζn) and σ#shi] is calculated. However, the correlation calculation method in the diagram is not limited to the formula (■), and the fast Fourier transform ( FF'r) can also be calculated.

al in Figure 1 is U(Zi) as shown in Figure 11.
The calculated data of
This is a window calculation circuit that multiplies .

This W(Zl) may be a function that monotonically decreases as the number of calculation data decreases, for example, a cosine function W(Zi
)4s(π・i/2N)"...(10)(1=-
N,...,0. ..., N) is used and set to 1. As a result of being multiplied by the weighting function W (Zi), the left-right symmetry evaluation function U
(Zl) is “ff(zt)=rI(zt)−w(zi)・−・・−
・-(11). (a) Finds the stage position where the [left-right symmetry evaluation function TfC2l shown in 11 trials] takes the maximum value, that is, the focus position Zf where the microscope (1) focuses. The focus position detection circuit moves the microscope stage (3) to the focus position by feeding the position signal of Zf' to the focus control circuit α2.

Note that this invention is applicable not only to the case where the second-order difference of the luminance standard deviation σ of the image is used as one focusing parameter, but also to the case where, for example,
It is also generally applicable to cases where the power of harmonic components of an image is used.

〔Effect of the invention〕

According to this invention, remarkable effects as shown below can be obtained.

(1) Compared to the method of determining the focus position from the maximum value of the luminance standard deviation of the image, the signal of the photoelectric conversion means is saturated.

It is possible to focus without being affected by disturbance factors such as the influence of images that deviate from the photoelectric conversion surface or the asymmetry of blur, and since the evaluation is performed using a statistic called a cross-correlation function, it is possible to focus without being affected by disturbances such as the influence of images that deviate from the photoelectric conversion surface, and the asymmetry of blur. Not affected by noise.

(2) The photoelectric conversion means, optical distance moving means (microscope stage moving mechanism, etc.) and signal processing means must be clearly separated, and an optical system dedicated to automatic focusing must be incorporated into the existing optical system. It is easy to configure.

[Brief explanation of the drawing]

FIG. 1X1 is a block diagram showing an embodiment of the present invention. Figure 2 is a diagram showing the relationship between focal length and luminance standard deviation of microscopes, ITV cameras, etc., and Figures 3 (al, (b), (
α1 is a diagram of the standard deviation when the photoelectric conversion element is saturated, Figure 6 (al, (bl) is a schematic diagram showing images in focus and blur, Figure 7 is an object outside the photoelectric conversion surface Diagrams showing the influence of images, Figure 8 is a diagram of σ with asymmetric blur, Figure 9 is a unique diagram of σ with asymmetric blur, and Figure 10 shows changes in luminance standard deviation σ and its 2 Figure 11 is a comparison diagram of the diagram of changes in the floor difference σ', and Figure 11 is a diagram of the evaluation function ■ formula.In the diagram, (1) is the microscope, (2) is the sample, (3) is the microscope stage, and (4) ) is a light source, (5) is a half mirror
1 (6) is a two-stage motor controller, (71 is
Y stage motor controller, (8) is the motor for moving the 2 stages, (9) is the motor for moving the XY stage, (
II is a photoelectric conversion means, I is a synchronization signal separation circuit, (l is a focus control circuit, opposite is a focus judgment circuit, and a4 is an analog/
Digital conversion circuit, (IG is luminance standard deviation calculation circuit, (
Le is a noise removal circuit, 2 is a brightness standard deviation second-order difference calculation circuit, (2) is a symmetry judgment circuit, restoration is a window calculation circuit, and 翺 is a focus position detection circuit. Agent Patent Attorney Sanro Kimura 2rl! J Rate Figure 3 Company η Ig flat plate 1 Figure 4 Figure 5 Figure 8 rlA Figure 9 4 crystal, +1rxz''' peak +0111

Claims (3)

[Claims] (1) a photoelectric conversion means, an optical system for forming an object image on the photoelectric conversion means, and an optical system driving means for changing the distance between the photoelectric conversion means and the optical system in steps;
A brightness standard deviation calculation means for calculating the standard deviation of the photoelectrically converted signal at each optical system distance, and a second-order difference value of the change in the brightness standard deviation at each optical distance calculated by the brightness standard deviation calculation means. A luminance standard deviation second-order difference means to seek, a line-symmetry determination means for determining the line symmetry of the output signal of the luminance standard deviation second-order difference means, and an optical system driving means that moves the optical system to the position determined to be most line-symmetric. An automatic focusing device comprising: a focusing means for setting a distance. (2) As an evaluation function for determining the line symmetry of the output signal of the luminance standard deviation second-order difference means, U(z)=∫＾(z_0−|z|)_(−z_0+|z
｜)σ″(z+ζ)σ″(z+ζ)dζwhere σ″: Second-order difference signal of luminance standard deviation -z~z_0: Optical distance Using the scanning range, optical The automatic focusing device according to claim 1, characterized in that a distance is set. (3) Line symmetry of the second-order difference signal of the change in the output signal at each optical distance after applying a spatial high-pass filter to the signal obtained by the photoelectric conversion means and obtaining the output signal. 2. The automatic focusing device according to claim 1, wherein the optical distance is set at a position that is most line symmetrical.