JP3109865B2 - Focus detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a focus detection device used for optical equipment such as a camera, and more particularly to a signal processing method for focus detection.

[0002]

2. Description of the Related Art Conventionally, as one method of a focus detecting device of a single-lens reflex camera, a pupil of a photographing lens is divided into two regions, and two light beams passing through the divided pupil regions form two subject images. 2. Description of the Related Art A device that detects a focus state of a photographing lens by observing a relative position displacement is well known. The principle will be briefly described with reference to FIG.

In FIG. 10, a field lens 2 is arranged with the same optical axis as a photographing lens 1 to be subjected to focus detection, and two secondary imaging lenses 3a and 3b are arranged in parallel behind the field lens 2. The photoelectric conversion sensor arrays 4a and 4b for light reception are arranged on the respective sides. Reference numerals 5a and 5b are diaphragms provided near the secondary imaging lens. The field lens 2 substantially forms an image of the exit pupil of the taking lens 1 on the pupil planes of the two secondary imaging lenses 3a and 3b. As a result, the light flux incident on each of the secondary imaging lenses 3a and 3b is shifted from the non-overlapping areas of the same area corresponding to the secondary imaging lenses 3a and 3b on the exit surface of the photographing lens 1. It will be injected.

An aerial image formed in the vicinity of the field lens 2 is converted into a sensor array 4 by secondary imaging lenses 3a and 3b.
When the re-images are formed on the surfaces a and 4b, the two re-images change their positions based on the difference in the position in the optical axis direction where the aerial image was formed.

FIG. 11 shows how this phenomenon occurs. FIG. 11 (a) mainly shows the in-focus state of FIG. 11 (a).
The so-called in-focus state as shown in (b) and (c) is the front focus,
The two images formed on the surfaces of the sensor rows 4a and 4b move in the opposite directions on the surfaces depending on whether the rear focus is on. By subjecting this image intensity distribution to photoelectric conversion and subjecting the obtained photoelectric conversion signal to electrical processing to detect the relative displacement between the two images, the focus of the photographing lens can be detected.

A number of methods for detecting the amount of shift between the two images have been proposed, and the correlation is obtained while relatively shifting the two photoelectrically converted image signals, and the image shift is performed with the shift position having the highest correlation. It is common to use a quantity.

For example, Japanese Patent Application Laid-Open No. Sho 56-75607 (hereinafter referred to as "Conventional Example 1") and Japanese Patent Application Laid-Open No. Sho 57-4
No. 5510 (hereinafter referred to as “conventional example 2”). JP-A-60-247211 (hereinafter referred to as "conventional example 3"). Or JP-A-58-14230 by the present applicant.
No. 6, JP-A-59-107313 (hereinafter referred to as "conventional example 4") and the like are disclosed.

The techniques of the correlation operation disclosed in the above-mentioned conventional example are characterized by procedures such as (1) an operator of the correlation operation, (2) a method of obtaining an extreme value of the correlation amount, and (3) a shift method of the correlation operation. is there.

First, the operator will be described.

In "Conventional example 1,""Conventional example 2," and "Conventional example 3," the operator of the correlation function is "the absolute value of the difference between the corresponding two signals" or "the absolute value of the difference between the corresponding two signals." Exponentiation "
Is used to calculate the correlation amount.

[0011] In "Conventional Example 4,""a large value (or a small value) of the corresponding two signals" is used as an operator of the correlation function.

Here, the meaning of the operator will be briefly described.

Each of the two subject image signal sequences is represented by IA
(I) and IB (i), the operator “absolute value of difference between corresponding two signals” (hereinafter referred to as “absolute value of difference”) calculates the correlation amount COR (K) at shift K by The calculation is performed according to equation (1).

[0014]

[Outside 1]

On the other hand, the "large value of the corresponding two signals" or the "small value of the corresponding two signals" (hereinafter "large value",
The operator called “small value” calculates the correlation amount COR (K) according to the equations (2) and (3), respectively.

[0016]

[Outside 2]

Where MaxＭx, y｝, Min ｛x,
y｝ is an operator for extracting the larger or smaller one of the real numbers x and y, respectively.

When the correlation amount is calculated according to the above equation,
The shift amount having the largest correlation can be detected from the obtained correlation amount for each shift. The shift amount and the image shift amount of the two subject images correspond to each other, and from this, the focus state (defocus amount) of the photographing lens can be detected.

As described above, if the image shift amount is simply the shift amount at which the correlation becomes maximum, the detectable image shift amount can be obtained only by one shift of the subject signal sequence, that is, by an integer unit of one pixel. When this is converted into a defocus amount, the resolution becomes relatively coarse, which is an insufficient level for practical use.

Therefore, each of the above-mentioned prior arts discloses a method of detecting an image shift amount at which the correlation becomes maximum with a resolution of a small unit smaller than an integer unit of one pixel. There are types.

One is a method of directly interpolating a decimal extremum in decimal units from a correlation amount in integer shift units (hereinafter referred to as a "direct interpolation method"). Another approach is to interpolate the first derivative, that is, the shift in decimal units where the differential value of the correlation amount (actually, the correlation amount is a difference value because the correlation amount is a discrete amount in shift units) is zero. (Hereinafter referred to as “differential interpolation method”). This is based on the principle that when the first derivative becomes zero, its original function takes an extreme value.

In the "direct interpolation method", it is expected that the correlation amount will be minimized based on the correlation amounts at a total of three points, that is, the shift in which the correlation amount calculated for each shift is the minimum and the shifts before and after the shift. The shift amount is obtained by interpolation using a scale smaller than the shift unit, and this is used as the image shift amount.
The aforementioned “Conventional example 3” corresponds to this, and a specific method is disclosed in detail in “Conventional example 3”.

On the other hand, in the "differential interpolation method", a difference amount of a correlation amount is calculated in advance for each shift, a shift in which the sign of the difference amount changes is detected, and a difference between the shift and two adjacent shift points is detected. From the quantity, the shift amount expected to be the difference amount to be zero, that is, the correlation amount to be an extreme value, is also obtained by interpolation using a scale smaller than the shift unit.

For example, when the operator is “absolute value of difference”, the correlation difference amount at shift K is calculated according to the following equation (4).

DIF (K) = Σ | IA (i) −IB (i + K + 1) | −Σ | IA (i + 1) −IB (i + K) | (4) The above “Conventional example 1”, “Conventional example 2”, The method of detecting an extremum in “Conventional example 4” corresponds to this, and the specific method is “Conventional example 2”.
In detail.

Next, a method of shifting the correlation operation will be described.

The setting method of Japanese Patent Application Laid-Open No. 56-50315 (hereinafter referred to as "conventional example 5") by the present applicant will be described with reference to FIG.

10a and 10b show intensity distributions of two subject images on the surface of the two sensor rows 4a and 4b in FIG. 10. In this example, a simple geometric edge image is 0.5. This shows the case where the pixel is shifted. Each sensor has eight pixels, and the number of signals in the image signal sequence is also shown as eight pixels. Arrows 12a and 12b indicate calculation regions for the correlation calculation, and the shift k of the correlation calculation is, for example, 4
In the case of, as shown in FIG.
For the signal of 0b, the correlation of the signals of the four pixels on the right is calculated.
When the shift k is 2, the operation area is symmetrically shifted by one pixel each, and the correlation of the image signals of the four pixels is similarly calculated. Similarly, while the number of operation pixels remains constant, the operation area is changed as shown in the figure according to the shift, and the sign of the shift k corresponds to the so-called front focus and rear focus.

A method of shifting the operation areas of the two image signals symmetrically and mutually according to the correlation shift will be hereinafter referred to as "mutual shift".

In the "mutual shift" method, two image signals are simultaneously shifted, so that the correlation shift must be in units of two shifts. Therefore, the number of signals in the image signal sequence is eight.
If the number of calculated pixels is four, the shift k that can be calculated is −4, −2, 0, 2, 4.

FIG. 15 is a plot of the correlation amount calculated by “mutual shift” using “absolute value of difference” as an operator. C (-4), C (-2), C (0), C
(2) and C (4) are shifts k = −4, −2,
The correlation amounts at 0, 2, and 4 are shown. From this plot, to detect the correlation extreme value in decimal units, the correlation amount C (-
From the three points of 2), C (0) and C (2), the minimum value of the correlation is interpolated using the “direct interpolation method” as shown by the broken line in the figure, and at the same time, the shift amount kp giving the minimum value Should be obtained. The method of obtaining the correlation extreme value in decimal units from the three correlation amounts is disclosed in detail in “Conventional example 3”, and thus the description thereof will be omitted.

FIG. 15 is a graph obtained by plotting the correlation amount using an operator of “absolute value of difference” to explain “mutual shift”. In “Conventional Example 5”, “mutual shift” is used as the shift method, but how to obtain the correlation extreme value is not particularly specified.

Next, the shift method in the "conventional example 3" will be described with reference to FIG.

As in FIG. 12, 10a and 10b in FIG.
The arrows 12a and 12b indicate the calculation area for the correlation calculation.

The feature of the shift method in "Conventional example 3" is that
This is a method of characterizing two subject images as a “reference part” and a “reference part”, and basically performing a correlation operation by shifting the image signal of the reference part while keeping the reference part fixed. .

As shown in the figure, when the shift is between k = -2 and 2, the operation area 12a of the image signal 10a of the reference section is fixed, and the operation area 12b of the image signal 10a of the reference section is fixed.
Only move. In this shift method, only one of the image signals is moved.
Unlike the case, the correlation operation for each shift can be performed.

By the way, at the shift k = -2 or 2, the operation area 12b of the image signal 10b of the reference section abuts the end of the entire signal area, so that further movement cannot be performed. Therefore, in order to calculate the shift k = −3 or more, the image signal 10 of the reference portion, which should be originally fixed, is calculated.
The calculation region 12a of a is moved by two pixels, and a shift of two images k = -3 is realized relatively. The same applies to shift k = 3 and thereafter.

The shift method in which the operation area of one image signal is fixed and only the other operation area is moved will be hereinafter referred to as "one-image shift".

FIG. 16 shows a plot of the correlation value calculated by the operator "absolute value of difference" and the shift method "single image shift" in "Conventional Example 3".

In the figure, C (-4) to C (4) represent the correlation amounts at shifts k = -4 to 4, respectively.

In order to detect the correlation extreme value in decimal units from this plot, the correlation amounts C (−1), C (0), C
From the three points (1) or C (0), C (1), and C (2), the minimum value of the correlation is interpolated using the “direct interpolation method” as shown by the broken line in the figure, and at the same time, the minimum value is obtained. What is necessary is just to find the shift amount kp giving the value.

In both the "conventional example 5" and the "conventional example 3", the shift amount in decimal units that gives the correlation extreme value is obtained from the correlation amounts at three points. While three correlation amounts separated by two shifts are used, “corresponding example 3” uses three correlation amounts separated by one shift.

Next, the shift method in "Conventional Example 2" and the method for obtaining the correlation extreme value will be described with reference to FIG.

As in FIGS. 12 and 13, 10a, 1
0b indicates the intensity distribution of the two subject images, and arrows 12a and 12b indicate calculation regions for correlation calculation.

The feature of the shift method of "Conventional Example 2" is that, as shown in the figure, when the shift k = 0, the image signals 10a and 10b
Is to be calculated, and the calculation range is symmetrically reduced from one end of the calculation area as shifting from k = 0.

This shift method is similar to the "mutual shift" in "Conventional Example 5", except that in "mutual shift", the absolute position of the operation area of the two image signals is changed as the shift is performed while the number of operation pixels remains constant. Mutually and symmetrically.
Therefore, the change in the relative position between the two calculation areas is performed in units of two shifts.

On the other hand, in "Conventional Example 2", the number of operation pixels changes with the shift, and the absolute position of one of the operation regions in the two image signals changes depending on whether the shift k is a positive or negative sign. I didn't let it. Therefore, the two operation areas are performed in units of one shift.

The shift method of moving only one of the operation areas while reducing the number of operation pixels of the two image signals in this manner is hereinafter referred to as “operation number variable shift”.

It should be noted here that, in "Conventional Example 2", arrows 12a and 1a in FIG.
That is, the correlation operation is not performed in the operation region itself shown by 2b. FIG. 14A is a diagram for explaining a difference between “Conventional example 2” and another conventional example. In the “Conventional Example 2”, the operation areas are actually shifted by one pixel with a certain shift as shown in FIG. Two operation area (15a, 15b) pairs separated by ± 1 shift about the shift and (16a, 16
b) The amount of correlation is calculated in pairs, and the difference is calculated.

FIG. 17 shows the correlation difference amount D (−
4) to D (4) are plotted. The operator is "absolute value of difference".

Since the object image given as an example is originally shifted by 0.5 pixel, the correlation value should take an extreme value when the shift k = 0.5. As is clear from the figure, the sign of the correlation difference amount changes at shifts k = 0 and k = 1, and the correlation difference amount is zero between shifts k = 0 and k = 1, that is, the original function. This means that there is a decimal shift amount that gives an extreme value of the correlation amount. Therefore, in “Conventional Example 2”, two points of the correlation difference amounts D (0) and D (1) are interpolated to obtain the shift amount kp at which the correlation difference amount is expected to be zero. The interpolation method for obtaining the correlation extreme value in decimal units from the two points of the correlation difference amount is disclosed in detail in “Conventional Example 2”, and thus the description thereof will be omitted.

The method of detecting the image shift disclosed in the “conventional example 4” by the present applicant is the same as the “differential interpolation method” in the same manner as the “conventional example 2”.
Although it is "operable variable shift", it is a method using "large value" or "small value" as an operator.

FIG. 18 shows correlation difference amounts E (−4) to E (−4) showing the correlation difference amounts according to “Conventional example 4”.
(4) shows the case where "large value" is used as the operator, and the correlation difference amounts F (-4) to F (4) show the case where "small value" is used as the operator. The method of obtaining the correlation extremum kp of the shift unit or less may be the same as that of “Conventional example 2”.

As described above, "conventional example 2"
Although the number of calculation pixels in the correlation calculation changes according to the shift, the number of calculation pixels is allowed to change because the correlation extreme value is detected based on the difference amount as described above. In the case of the "direct interpolation method" in which the direct correlation extreme value is detected, if the number of calculation pixels at each shift changes, the amount of information contributing to each correlation amount is not uniform.
There may be cases where the correct extremum cannot be detected. In other words, the "direct interpolation method" that directly interpolates the extreme value from the correlation amount is achieved by implicitly assuming that the correlation amount changes at a fixed rate according to the shift,
When the number of pixels used for the correlation amount calculation changes due to the shift, the correlation amount changes due to the two factors of the shift and the number of calculation pixels, and the interpolation calculation does not hold.

Therefore, when "variable shift of operation number" is used as in "conventional example 2", in combination with "differential interpolation method", as shown in FIG. The number of calculation pixels is kept constant for each shift.

The method of detecting the image shift of the conventional example described above is summarized as follows. (1.1) “Absolute value of difference” (conventional examples 1, 2, 3) (1.2) “large value (or small value)” (conventional example 4) 2 . (2.1) “Direct interpolation method” (conventional example 3) (2.2) “Differential interpolation method” (conventional examples 1, 2, and 4) The correlation operation shift method includes: (3.1) “mutual shift” (conventional example 5) (3.2) “single image shift” (conventional example 3) (3.3) “operation number variable shift” (conventional example) 2.4) However, it is better not to use the “variable operation number shift” of (3.3) in combination with the “direct interpolation method” of (2.1).

[0057]

As described above, various methods of detecting an image shift have been proposed, but each method has a performance difference depending on an object image to be observed.

For any geometric object image free of noise, the correct amount of image shift can be detected by any method. However, when low-frequency noise such as random noise from a photoelectric conversion sensor or ghost of an optical system is mixed in a subject image signal, the S / N ratio depending on the subject image is generated depending on the method.

As an example, if the object image to be observed is a pattern such as a so-called edge pattern that is entirely flat and has only a partial luminance change, the luminance change portion is a calculation area for correlation calculation. , The method using the “large value (or small value)” is more effective than the image shift detection method using the “absolute value of the difference” as the operator. Excellent in N. The reason is described in detail in Japanese Patent Application Laid-Open No. S60-101513 (hereinafter referred to as "Conventional Example 6") by the present applicant.

When low-frequency noise such as a ghost of an optical system is mixed in a subject image signal, an extreme value is generally obtained from a correlation difference amount as an interpolation method of a correlation extreme value, though it depends on the subject. The "direct interpolation method" for directly finding an extremum from the correlation amount tends to have a smaller error than the "differential interpolation method". This is considered to be mainly due to how many correlation amounts are used in the interpolation calculation. "Differential interpolation method" is 2
Although the interpolation operation is performed using the correlation difference amount between the points, the difference amount is calculated from the correlation amount between the two points as shown in Expression (4). Therefore, the interpolation operation is substantially performed using the correlation amount between the four points. It is nothing but performing the operation. On the other hand, the direct interpolation method
Point correlation is used. For this reason, when each shift correlation amount fluctuates due to noise, the "differential interpolation method" using a larger number of correlation amounts is more susceptible to noise.

As a shift method of the correlation operation,
It can be said that "mutual shift" is disadvantageous in S / N compared to other methods. In the case of "mutual shift", as described above, the correlation amount can be obtained only in units of two shifts. In the other two methods “one-image shift” and “variable number of operations”, the shift is obtained in units of one shift. Therefore, in the interpolation operation of "mutual shift", the position of the correlation extreme value to be obtained is calculated using the correlation amount at a shift that is farther away than other methods. When the inter-coordinate distance of the correlation amount used in the interpolation calculation increases, it is easily expected that the error of the interpolation calculation will increase.

The advantages and disadvantages of the image shift detection method can be summarized by the procedure described above. When the operator is “large value (or small value)”, the correlation extreme value interpolation method is “direct value”. The "interpolation method" is generally superior in terms of S / N to the shift method of the correlation operation except for the "mutual shift" and "single image shift" if combined with the "direct interpolation method". Can be.

However, the image shift detecting method combining the above preferred procedures cannot detect a correct image shift amount. FIG. 19 shows a plot of the correlation amount by this combination. The operator uses "large value (or small value)", "direct interpolation method" as the interpolation method of the correlation extreme value, and "single image shift" as the shift method. The subject image signal assumes a case where the geometric edge image is shifted by 0.5 pixel as in the previous examples.

When the "large value" is used as the operator, the correlation amounts B (-4) to B (-4) to A (4)
(4) shows a case where “small value” is used as an operator. As can be seen from the figure, no matter which of the above two correlation amounts is used, the position does not show the extreme value at the position of 0.5 pixel which is the original image shift amount, that is, the shift k = 0.5. Image shift detection is not possible.

However, the subject image is shown in FIG.
If it is a geometric bar pattern as shown in (b), an operator of "large value (or small value)" can correctly detect an image shift. The plot of the correlation amount in this case is shown in FIG. (B). Correlation A (-4) to A (4), Correlation B (-4)
To B (4) are the correlation amounts when the operators are “large value” and “small value”, respectively. Two subject images are shifted by 0.5 pixel as in the previous examples. The case is assumed. In this case, no matter which of the two correlation amounts is used, the shift amount kp of the correlation extreme value represents 0.5 pixel which is the original image shift amount, and the correct extreme value can be detected.

The reason why the operator "large value (or small value)" has such a property can be considered as follows.

The correlation function is a function in which the correlation amount becomes minimum (or maximum) at the shift position where the two signal sequences are most similar, and the correlation amount increases (or decreases) as the distance from the shift position increases. . However, when the "large value" is used as an operator, and the calculation area of one image is fixed, and the correlation amount of a "left-right asymmetric" image signal such as an edge image is calculated, if the shift sign is positive, The amount of correlation changes according to the shift, but in the negative case, the amount of correlation does not change even if the shift is changed. In other words, if the edge portion of the other image enters the “envelope” created by the luminance signal of the fixed nudge image, the “large value” of This is because the operator always extracts only a constant signal along the envelope of the fixed image, and no change appears in the correlation amount.

In the case of "mutual shift", the above-mentioned state does not occur because the calculation area is constantly changed without fixing one image, and therefore, the correlation amount is changed according to the shift regardless of the sign of the shift. Changes, and the detection of the correlation extreme value becomes possible.

After all, the operator “sum of large values (or sum of small values) of corresponding two signals” is combined with a correlation operation in which one operation area of the image signal is fixed, and the correct image shift amount depends on the subject. Is an operator that cannot detect
It turns out that.

[0070]

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problem. Prior to a correlation operation, a subject image signal to be detected for image shift is duplicated to increase the signal amount.
In the correlation calculation, the amount of correlation is obtained by "mutual shift", thereby enabling highly accurate image shift detection excellent in S / N ratio.

[0071]

The present invention will be described below with reference to examples.

FIGS. 1A and 1B illustrate the enlargement of the subject image signal performed prior to the correlation calculation and the setting of the calculation area accompanying the shift.

In FIG. 10, reference numerals 10a and 10b denote the intensity distributions of the two subject images on the surface of the sensor rows 4a and 4b in FIG.
This shows a case where there is a shift of 5 pixels. The number of signals in the image signal sequence is 8 pixels. In the figure, IA (0) to IA (7) indicate each pixel output of the sensor array 4a, and IB (0) to IB (7).
Indicates each image output of the sensor 4b.

The enlargement of the image signal is performed by duplicating each pixel signal of the image signal series 10a and 10b in FIG. 1A and enlarging it to twice the number of signals, respectively, as shown by 11a and 11b in FIG. 1B. ,
The conversion is performed by converting the signal into an image signal sequence having 16 pixels. Specifically, the first pixel signal IA of the image signal sequence 10a
(0) and the first JA (0) and the second JA (0) of the image signal sequence 11a.
And the second pixel signal IA (1) of the image signal sequence 10a is referred to as a third JA (1) of the image signal sequence 11a.
(2) and a fourth JA (3).

The subject image signal 11 thus enlarged
a and 11b are subjected to a correlation operation. 12a, 12 in the figure
The arrow b indicates a calculation area for the correlation calculation, which is the “mutual shift” as in FIG. It should be noted that the “mutual shift” in FIG. 7 is a two-shift correlation operation because two images are moved simultaneously, but in FIG.
Since the image signal is moved after being enlarged twice, it is possible to perform a correlation operation in units of one shift in terms of the original image signal sequence.

FIG. 2 shows a plot of the correlation amount for each shift in the correlation operation according to the present invention. The operator uses "large value (or small value)". Correlation amount A (-
4) to A (4) and correlation amounts B (−4) to B (4) are the correlation amounts when the operators are “large value” and “small value”, respectively.

As is apparent from the figure, the correlation amount can be obtained in units of one shift, regardless of the "mutual shift", and the correlation extreme value can be correctly detected.

FIG. 3 shows a specific circuit block diagram for realizing the focus detection device according to the present invention.

In the figure, reference numeral 40 denotes a sensor device, in which two sensor rows 4a and 4b are arranged. Reference numeral 50 denotes a sensor drive circuit, and reference numeral 60 denotes a microcomputer that controls the entire focus detection device and performs signal processing. Microcomputer 6
The sensor drive circuit 50 is controlled by control signals 61, 62, 63 output from 0, and the control signals 61, 62, 63
Sensor drive signals 51, 52 and 53 generated from 63 are provided to the sensor device 40. The actual control in this area is not directly related to the present invention, and thus a detailed description is omitted.

When a predetermined accumulation of the sensor array composed of accumulation type sensors is performed, the analog image signal 41 is output to the sensor device 4.
0, and the amplification circuit 54 in the sensor driving circuit 50
, The amplified analog image signal 55 is input to the analog input of the microcomputer 60. The microcomputer 60 outputs control signals 61 and 6 output by itself.
A / D conversion of the input analog image signal 55 in synchronization with
The digital image signals are sequentially stored at a predetermined address of M.

When the storage of the image signal is completed, the microcomputer 60 executes the focus detection program stored in the ROM.

FIG. 4 shows a flowchart of the microcomputer program.

When the focus detection operation is started, step (S1)
The "focus detection start" routine N of "00" is called, and the input of the image signal is executed in step (S101). The image signal 10a in FIG. 1A is represented by IA (0) to IA.
(7) and 10b are IB (0) to IB (7).

In the next steps (S102) and (S103), a process of expanding the image signal to twice the signal amount is performed. The enlarged image signals 11a and 11b in FIG. 1B are respectively represented by JA (0) to JA (15) and JB (0) to JB (1
Assuming that 5), JA (2 · i) ← IA (i) (5) JA (2 · i + 1) ← IA (i) (6) JB (2 · i) ← IB (i) (7) JB (2 · i + 1) ← IB (i) (8) where i = 0 to 7. IA in equations (5) to (8)
(I), IB (i) is represented by JA (0) to JA (15), JB
(0) to JB (15). Step (S1)
02) represents a loop process for i, and the equations (5) to (8) are executed in step (S103) in the loop.

Next step (S104), (S105)
Performs a correlation operation in the correlation operation range shown in FIG. In this case, the correlation amount at the shift k is COR (k) using “small value” as an operator, and the following equation (9) is calculated.

[0086]

[Outside 3] However, k = −4 to +4.

The processing in steps (S104) and (S105) means the above-mentioned “correlation calculation in equation (9) using small values” for each calculation range for each k value shown in FIG. That is, when k = -4, the signals JA (0) to J (0) to J
For A (7) and signals JB (8) to JB (15), the smaller right value of JA (0) and JB (8) is expressed by J
A smaller value of A (1) and JB (9)
A pair of A (2) and JB (10), and a pair of JA (3) and JB (1)
1) pair, JA (4) and JB (12) pair, JA
Pair of (5) and JB (13), JA (6) and JB (1
4) and the smaller value of each pair of JA (7) and JB (15) are selected, and the sum of the selected smaller right values is obtained as COR (-4). .

When k = -3, the signals JA (1) to JA (1) to J
For A (8) and the signals JB (7) to JB (14), the sum of the smaller values of the corresponding signals is COR (-
3). Similarly, when k = −2 to +4, the above-mentioned COR (−) is obtained for each range shown in FIG.
2) to COR (+4) are required.

The correlation amount CO when the shift k is from -4 to +4
When the calculation of R (k) is completed, steps (S106) to (S106)
In step (S110), an extreme correlation value is detected. When “small value” is adopted as the operator, the correlation amount COR (k) should be the maximum at the shift where the correlation is the maximum. Therefore, in step (S106), 0 is given as an initial value to the variable CORMAX storing the maximum value of the correlation amount in advance.

Step (S107) represents a loop process for k, and searches for an extreme value of the correlation amount over the range of k = -3 to +3.

As the conditions for the extremum search, "maximum" and "greater than two points before and after" are used. Step (S
At 108), it is determined whether or not the correlation amount COR (k) is maximum.
It is determined whether or not the correlation amounts of the points are larger than COR (k-1) and COR (k + 1). Only when both conditions are satisfied, the process proceeds to step (S110), and the process proceeds to step (S11).
0), the maximum correlation value variable CORMAX is set to COR.
The value of (k) is updated, and at the same time, the current value of shift k is stored in extreme value shift kx. In order to determine the step (S109) in which the correlation amount is compared with the values of two points before and after, the loop processing in the step (S107) sets k to -3 to +3 and sets the shift range (the shift range in which the correlation operation was actually performed). -4 to +
4) It is narrower.

Now, B (-4) to B (4) in FIG. 2 are the correlation values COR obtained in steps S104 and S105.
The extreme value search will be described as representing (−4) to COR (+4).

First, CORMAX is set to 0 in S106, and COR (-3) and CORMAX are compared in S108. Since COR (−3) = B (−3)> CORMAX, COR (−3) and CO (−3) are determined in step S109.
R (−4) and COR (−3) are compared with COR (−2).

In this case, COR (−2) = B (−2)>
Since COR (−3) = B (−3), step S1
Then, k is set to −2 without executing Step 10, and Steps S108 and S109 are compared again. As a result, CO
R (-1) = B (-1)> COR (-2) = B (-2)
Therefore, without executing step S110, k
= −1, and the above steps S108 and S10 are performed again.
9 is executed. As a result, COR (0) = B (0)> C
Since OR (-1) = B (-1), S1 is also used in this case.
Steps S108 and S109 are executed again, with k = 0, without executing step 10. In this case, COR
(0) = B (0)> COR (-1) = B (-1), and COR (0) = B (0)> COR (1) = B
(1), step S110 is executed, and COR
Set COR (0) = B (0) as MAX and set kx
Is set to 0, then k = 1, and the above S
108 and S109 are executed. In the subsequent processing from k = 1 to k = 4, steps 108 and 109 do not satisfy the condition, so that the processing is performed with k = 4 and then step S11
Proceed to 1.

When an extreme correlation value is detected in steps (S107) to (S110), the next step (S11)
In 1), interpolation of the correlation extreme value is performed to obtain the image shift amount on a scale smaller than the shift unit.

As the interpolation method, as described above, the method of performing the interpolation from the shift of the maximum value and the correlation amounts of three points before and after the shift is used. Here, as an example, the interpolation calculation is performed by the method of the conventional example 3.

As shown in FIGS. 5 (a) and 5 (b), the correlation amount COR calculated using the "small value" at the shift kx.
When (kx) becomes the maximum, COR (kx-1) and C
Depending on the magnitude of the value of OR (kx + 1), COR (kx
-1) ≦ COR (kx + 1) (FIG. 5A)
And COR (kx-1)> COR (kx1) (FIG. 5B). As shown in FIG.
If R (kx−1) ≦ COR (kx + 1), CO
The k coordinate at the intersection of a straight line passing through R (kx-1) and COR (kx) and a straight line having the opposite sign and passing through COR (kx + 1) with the opposite sign is defined as an image shift amount kp of a small number or less. With the above processing, the image shift amount kp as the focus detection result is obtained.

As shown in FIG. 5B, COR (kx-1)>
In the case of COR (kx + 1), COR (Kx) and CO
The k coordinate of the intersection of a straight line passing through R (kx + 1) and a straight line passing through COR (kx−1) with the opposite sign to the straight line passing through COR (kx−1) is represented by kp
And This can be expressed as follows. (I) When COR (kx−1) ≦ COR (kx + 1),

[0099]

[Outside 4] (Ii) When COR (kx−1)> COR (kx + 1),

[0100]

[Outside 5]

As described above, the maximum correlation COR (K
x) and two points COR (kx-1) and COR (k
x + 1), the image shift amount kp is obtained.
As shown in FIGS. 5A and 5B, in addition to the method of calculating kp by linear interpolation, the correlation amounts COR (kx−1) and COR (K
x) and COR (kx + 1) may be obtained as extreme values of a quadratic function through which three points pass.

FIG. 6 is an explanatory diagram showing another example of the above-mentioned interpolation calculation. FIG. 6 shows an example in which the image shift amount kp is interpolated by a quadratic function.

In FIG. 6, the maximum correlation COR (Kx) and the two points COR (kx-1) and COR (kx + 1) before and after it are shown.
Is given, the quadratic function COR (K) passing through the three points
= Ak ² + bk + c are obtained, and the extreme value of the quadratic function is obtained.

[0104]

[Outside 6] Is regarded as kp. kp is given by the following equation (12).
Required by

[0105]

[Outside 7]

Now, FIG. 4A shows a case where "small value" is used as a correlation operator, but FIG. 4B shows a part of a flowchart in the case where "large value" is used as an operation. Shown in

Steps (S204) to (S204) in FIG.
210) corresponds to (S104) to (S110) in FIG. Explaining only the difference between the two, the operator "large value" is used when calculating the correlation amount in step (S205). When “large value” is used as an operator, the correlation amount is the maximum correlation and the minimum value.

In the next step, the initial value 9999 is stored in the variable CORMIN representing the minimum value of the correlation amount. This is not limited to 9999, and it is sufficient to store a large value from which the correlation amount cannot be obtained.

Steps (S208), (S209),
In (S210), the extreme value search condition is “minimum” and “smaller than two points before and after”. If the condition is satisfied, the minimum value variable C is determined in step (S210).
Update ORMIN and shift kx.

As for the interpolation of the correlation extreme value, when interpolating with a straight line, the sum of the cases differs slightly depending on whether the maximum value or the minimum value is obtained. That is, COR (kx
-1) ≧ COR (kx + 1), the equation (10) is
When OR (kx−1) <COR (kx + 1), the expression (1)
1) applies.

As shown in FIG. 6, when interpolating with a quadratic function, it is naturally obtained by equation (12) regardless of the operator.

In the present invention, the number of new image signal sequences is two.
Therefore, if the correlation calculation is performed with twice the signal amount as it is, the calculation time will be significantly longer than in the past.

However, if the correlation calculation is devised as shown in FIG. 7, the calculation time can hardly be increased as compared with the related art.

The arrow pairs (13a, 14a), (13
b, 14b) represent the correlation calculation areas corresponding to the arrows 12a, 12b in FIG. 1, respectively. Originally, the enlarged image signal sequences 11a and 11b are respectively image signal sequences 10a and 10b.
b. Therefore, the enlarged image signal sequence 1
1a and the area 12b of the enlarged image signal sequence 11b
The amount of correlation between them is equal to the value obtained by adding the amount of correlation between the regions 14a and 14b to the amount of correlation between the region 13a of the original image signal sequence 10a and the region 13b of the original image signal sequence 10b. In the even-numbered shift, the areas 13a and 14a are equal, and in the odd-numbered shift, the area 13a is an area obtained by adding one stroke at both ends of the 14a.

Therefore, the correlation amount at the even-numbered shift is a value obtained by doubling the correlation amount between the regions 14a and 14b, and the correlation amount at the odd-numbered shift is a value obtained by doubling the correlation amount between the regions 14a and 14b. And the correlation amount of only the signals at both ends of the regions 13a and 13b. By performing the calculation in this manner, the correlation calculation time in the enlarged image signal sequence hardly increases compared to the calculation time in the original image signal sequence.

FIG. 8 shows a part of a flowchart when the calculation method of FIG. 7 is used. Step of FIG. 8 (S301)
4 to (S305) correspond to the steps (S10) in FIG.
1) to (S105).

The step (S301) is the same as the step (S1)
01), image signals IA (0) to IA (7), IB
(0) to IB (7) are obtained.

The next step (S302) represents a loop process for the shift k, where k = -4 in the loop.
To +4.

As apparent from FIG. 7, the method of setting the operation area differs depending on whether the shift k is an even number or an odd number. That is,
When the shift k is an even number, the correlation amount COR (k) becomes

[0120]

[Outside 8] Becomes On the other hand, when the shift k is an odd number, the correlation amount CO
R (K) is

[0121]

[Outside 9] Becomes COR in steps (S303) to (305)
(K) is calculated.

If the correlation amount COR (k) in which the shift k is from -4 to +4 is obtained in this way, then, FIG.
The search for the correlation extreme value and the interpolation of the extreme value may be performed after the step (S106) of (a).

In FIG. 8, “small value” is used as the operator of the correlation operation. However, when “large value” is used as the operator, “Min” in the figure may be changed to “Max”. In this case, the steps (S206) to (S21) in FIG.
0) and step (S111) in FIG. 4A, the image shift amount kp can be obtained.

In the above embodiment, as described in "Conventional Example 6", the image of the object to be observed is an image such as a so-called edge pattern which is entirely flat and has only a partial luminance change. In this case, a method using “large value (or small value)” is more advantageous in S / N than an image shift detection method using “absolute value of difference” as an operator of the correlation operation. For this reason, in the description so far, “large value (or small value)” has been used as an operator. But,
The operator “absolute value of difference” is also applicable to the image shift detection method of the present invention.

As described above, the operator “absolute value of difference” can be used in combination with “shift of one image” and “alternate shift”, which are shift methods of the correlation operation. Although it seems that there is no need to combine with the "mutual shift", the "single image shift" fixes the calculation area of the reference portion of the image signal, and therefore may be inconvenient depending on the subject image to be observed. That is, in the “single image shift”, since the shift k = 2 to 3, and k = −2 to −3, the calculation area of the fixed reference image is changed.
This is because the correlation amount may be discontinuous here.

In the case of a subject image shifted by 0.5 pixel as shown in FIG. 13, kp = 0.5 can be interpolated as shown in the plot of the correlation amount in FIG. It is. However, if the subject image is shifted by 2.5 pixels, the shift k = 2, 3, 4 or k = 1, 2, 3
Since the correlation extreme value in decimal units is obtained by interpolating the correlation extremum of the three points, if the continuity of the correlation quantity is impaired by the shifts k = 2 and k = 3, it may be a considerable error factor.

In "Conventional example 3", the shift range assigned to one calculation region of the reference image partially overlaps with the shift range of another calculation region in consideration of the vicinity.
As a result, the three correlation amounts used in the interpolation calculation can be the same in the calculation area of the reference image, and the continuity of the three correlation amounts can be maintained.

In the image shift detecting method of the present invention, since there is no concern about the continuity of the correlation amount, there is no need to devise the technique described in "Conventional Example 3", and the calculation time does not increase accordingly.

FIG. 9 shows a plot of the amount of correlation for each shift when "absolute value of difference" is applied as an operator to the image shift detection method of the present invention. The subject image is the same geometric edge image as in FIG. 1 shifted by 0.5 pixel.

As is clear from the figure, the correlation amount does not become discontinuous between shifts k = 2 and 3, or k = -2 and -3, and it is possible to detect a correct image shift amount.

[0131]

As described above, according to the present invention,
Prior to the correlation operation, the object image signal to be detected for image deviation is duplicated, the signal amount is enlarged, and the correlation operation obtains the correlation amount by “mutual shift”, thereby detecting an image deviation excellent in S / N. If it is possible, and if a "large value (or a small value)" is used as an operator, the dependence on the subject can be reduced, thereby enabling highly accurate focus detection.

[Brief description of the drawings]

FIG. 1 is a principle explanatory diagram of a correlation operation according to the present invention.

FIG. 2 is a plot diagram of a correlation amount of the first embodiment of the correlation operation according to the present invention.

FIG. 3 is a circuit block diagram of the focus detection device of the present invention.

FIG. 4 is an explanatory diagram showing a flowchart of a microcomputer in the focus detection device.

FIG. 5 is an explanatory diagram of detection of an extreme value of a correlation amount.

FIG. 6 is an explanatory diagram showing another example of detecting an extreme value of a correlation amount.

FIG. 7 is another explanatory diagram of the correlation operation according to the present invention.

8 is an explanatory diagram showing a flowchart of a microcomputer for realizing the correlation operation shown in FIG. 7;

FIG. 9 is an explanatory diagram for explaining another example of a correlation amount interpolation operation.

FIG. 10 is an explanatory diagram for explaining the principle of a focus detection optical system.

FIG. 11 is an explanatory diagram for explaining the principle of the focus detection optical system together with FIG. 10;

FIG. 12 is an explanatory diagram for explaining the principle of a conventional correlation operation.

FIG. 13 is an explanatory diagram for explaining another principle of a conventional correlation operation.

FIG. 14 is an explanatory diagram for explaining the principle of another conventional correlation operation.

FIG. 15 is an explanatory diagram showing a conventional correlation operation result.

FIG. 16 is an explanatory diagram showing another example of a conventional correlation operation result.

FIG. 17 is an explanatory diagram showing a difference amount calculation result of a conventional correlation calculation.

FIG. 18 is an explanatory diagram showing another example of a difference amount calculation result of a conventional correlation calculation.

FIG. 19 is an explanatory diagram showing a correlation calculation amount by a combination of the conventional example.

FIG. 20 is an explanatory diagram illustrating an example of a subject image signal.

FIG. 21 is an explanatory diagram showing another example of the correlation operation using the combination of the conventional example.

[Explanation of symbols]

 Reference Signs List 1 shooting lens 4a photoelectric conversion sensor array 4b photoelectric conversion sensor array 40 sensor device 60 microcomputer

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G02B ^7/ 28-7/40 G03B 3/00-3/12

Claims (2)

(57) [Claims] 1. A light beam is received on first and second sensor arrays via an optical system, and an optical signal is obtained based on a relative displacement amount with respect to the first and second signals corresponding to the light beam received by each sensor array. In a focus detection device that detects a focus state of a system, a signal of each pixel constituting each sensor is duplicated and enlarged to a double signal amount, and a signal corresponding to the enlarged first and second signals is generated. A focus detection device wherein the correlation calculation is performed in such a manner that the calculation areas of the first and second signals are displaced symmetrically with each other for each shift. 2. The correlation operation according to claim 1, wherein the correlation operation is a correlation operation using a large value or a small value of a corresponding signal in the first and second signals as an operator. Focus detection device.