JPH0527162A - Focus detecting device - Google Patents

Abstract

PURPOSE:To accomplish the accurate detection of the deviation of an image, which is excellent in S/N, by duplicating an object image signal, doubly enlarging it, and obtaining a correlation amount by ''mutual shift'' in correlation arithmetic operation. CONSTITUTION:In a device where a luminous flux is received on 1st and 2nd sensor trains 4a, 4b through optical systems 1-3 so that the focusing state of the optical systems 1-3 is detected based on a correlation displacement amount with respect to 1st and 2nd signals by the respective sensor trains 4a and 4b, the signals from plural picture elements which constitute the respective sensors 4a, 4b are duplicated, enlarged to be double signal amounts, and the correlation arithmetic operation is performed to the 1st and the 2nd enlarged signals. In the correlation arithmetic operation, a larger value or a smaller value out of the corresponding signals in the 1st and the 2nd signals is set as an operator. By obtaining the correlation amount by 'mutual shift', the detection of the image, which is excellent in the S/N, is accomplished. Furthermore, by using the larger value or the smaller value as the operator, the dependency of the object is decreased, and the accurate focus detection is accomplished.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a focus detection device used in optical equipment such as a camera, and more particularly to a signal processing method for focus detection.

[0002]

2. Description of the Related Art Conventionally, as one method of a focus detecting apparatus for a single-lens reflex camera, a pupil of a photographing lens is divided into two areas, and two object images formed by a light flux passing through the divided pupil areas are formed. A device that detects the focus state of a photographing lens by observing the relative position displacement is well known. The principle will be briefly described with reference to FIG.

In FIG. 10, a field lens 2 is arranged with the same optical axis as the photographic lens 1 for focus detection, and two secondary imaging lenses 3a and 3b are arranged in parallel behind the field lens 2 and further after that. Photoelectric conversion sensor arrays 4a and 4b for light reception are arranged on each side. Note that 5a and 5b are diaphragms provided near the secondary imaging lens. The field lens 2 substantially forms the exit pupil of the taking lens 1 on the pupil planes of the two secondary imaging lenses 3a and 3b. As a result, the light fluxes incident on the secondary imaging lenses 3a and 3b respectively come from the areas of the same area on the exit surface of the photographing lens 1 which do not overlap with each other and which correspond to the secondary imaging lenses 3a and 3b. It has been ejected.

The aerial image formed near the field lens 2 is transferred to the sensor array 4 by the secondary imaging lenses 3a and 3b.
When re-imaged on the surfaces a and 4b, the two re-imaged images change their positions based on the difference in the position in the optical axis direction where the aerial image is formed.

FIG. 11 shows how this phenomenon occurs. Focusing on the in-focus state of FIG. 11 (a), FIG.
As shown in (b) and (c), the so-called focus state is the front focus,
The two images formed on the surfaces of the sensor rows 4a and 4b move in opposite directions on the surfaces depending on whether or not the rear focus is applied. If the relative displacement of the two images is detected by photoelectrically converting the image intensity distribution and subjecting the obtained photoelectrically converted signal to electrical processing, the focus of the photographing lens can be detected.

A number of methods for detecting the amount of shift between the two images have been proposed, and the correlation is obtained while relatively shifting the two photoelectric conversion image signals, and the image shift occurs at the shift position with the highest correlation. The method of setting the quantity is common.

For example, JP-A-56-75607 (hereinafter referred to as "conventional example 1") and JP-A-57-4 of Japan.
Japanese Patent No. 5510 (hereinafter referred to as "conventional example 2"). JP-A-60-247211 (hereinafter referred to as "conventional example 3"). Alternatively, JP-A-58-14230 by the present applicant
Various methods are disclosed, such as Japanese Patent Laid-Open No. 6 and Japanese Patent Laid-Open No. 59-107313 (hereinafter referred to as "conventional example 4").

Each of the correlation calculation methods disclosed in the above-mentioned conventional examples is characterized by the procedures such as (1) operator of correlation calculation, (2) method of obtaining extreme value of correlation amount, and (3) shift method of correlation calculation. is there.

First, the operator will be described.

In "conventional example 1", "conventional example 2" and "conventional example 3", as the operator of the correlation function, "absolute value of difference between corresponding two signals" or "absolute value of difference between corresponding two signals" is used. Exponentiation "
Is used to calculate the correlation amount.

In "Conventional Example 4", "large value (or small value) of corresponding two signals" is used as an operator of the correlation function.

The meaning of the operator will be briefly described below.

The two object image signal sequences are respectively IA
(I) and IB (i), the operator "absolute value of difference between corresponding two signals" (hereinafter referred to as "absolute value of difference") calculates the correlation amount COR (K) at shift K as follows. The calculation is performed according to the equation (1).

[0014]

[Outer 1]

On the other hand, the "large value of the corresponding two signals" or the "small value of the corresponding two signals" (hereinafter "large value",
The operator "called a small value" calculates the correlation amount COR (K) according to the equations (2) and (3), respectively.

[0016]

[Outside 2]

However, Max {x, y}, Min {x,
y} is an operator for extracting the larger one or the smaller one of the real numbers x and y.

When the correlation amount is calculated according to the above equation,
From the obtained correlation amount for each shift, the shift amount having the largest correlation can be detected. This shift amount corresponds to the image shift amount between the two subject images, and the focus state (defocus amount) of the photographing lens can be detected from this.

By the way, if the image shift amount is simply the shift amount that maximizes the correlation as described above, the detectable image shift amount can be obtained only by one shift of the subject signal sequence, that is, by an integer unit of one pixel. When this is converted into a defocus amount, the resolution becomes relatively coarse, which is an insufficient level for practical use.

Therefore, each of the conventional examples discloses a method of detecting the image shift amount at which the correlation is maximized with a resolution of a decimal unit smaller than an integer unit of one pixel. There are types.

One is a method of directly interpolating a correlation extremum in a decimal unit from a correlation amount in an integer shift unit (hereinafter referred to as "direct interpolation method"). Another method is to interpolate a first-order derivative, that is, a differential unit value of the correlation amount (actually, the correlation amount is a discrete amount of the shift unit, so it is a difference value), but is a decimal unit shift. This is a method of obtaining (hereinafter referred to as "differential interpolation method"). This is based on the principle that the original function takes an extreme value when the first derivative becomes zero.

In the "direct interpolation method", it is expected that the correlation amount will be minimized from the shift amount calculated for each shift and having the minimum correlation amount and the correlation amounts at three points before and after the shift. The shift amount to be obtained is interpolated using a scale smaller than the shift unit, and this is used as the image shift amount.
The “conventional example 3” corresponds to this, and the specific method is disclosed in detail in the “conventional example 3”.

On the other hand, in the "differential interpolation method", the difference amount of the correlation amount is calculated in advance for each shift, the shift in which the sign of the difference amount changes is detected, and the difference between the shift and the adjacent two shift points is detected. From the quantity, the shift quantity that the difference quantity will be zero, that is, the correlation quantity will have an extreme value, is obtained by interpolating with a scale smaller than the shift unit.

For example, when the operator is the "absolute value of difference", the correlation difference amount in shift K is calculated according to the following equation (4).

DIF (K) = Σ | IA (i) -IB (i + K + 1) | -Σ | IA (i + 1) -IB (i + K) | (4) The “conventional example 1”, the “conventional example 2”, The extremal value detection method of "Conventional example 4" corresponds to this, and the specific method is "Conventional example 2".
In detail.

Next, the shift method of the correlation calculation will be described.

The setting method of JP-A-56-50315 (hereinafter referred to as "conventional example 5") by the present applicant will be described with reference to FIG.

Reference numerals 10a and 10b in the figure show the intensity distributions of two object images on the surfaces of the two sensor arrays 4a and 4b in FIG. 10, and in this example, a simple geometric edge image is 0.5. The case where the pixels are displaced is shown. Each sensor has 8 pixels, and the number of signals in the image signal series is also 8 pixels. Arrows 12a and 12b indicate calculation areas for the correlation calculation, and the shift k of the correlation calculation is, for example, 4
In the case of, as shown in FIG.
For the signal 0b, the correlation of the signals of the four pixels on the right side is calculated.
When the shift k is 2, the calculation areas are symmetrically shifted by one pixel and the correlation of the image signals of four pixels is calculated. Similarly, while the number of pixels to be calculated is constant, the calculation area is changed according to the shift as shown in the figure. The sign of shift k corresponds to the so-called front focus and rear focus.

The shift method in which the calculation regions of the two image signals are symmetrically and mutually shifted in accordance with the correlation shift will be referred to as "mutual shift" hereinafter.

In the "mutual shift" method, since two image signals are shifted at the same time, the shift of the correlation is inevitably 2 shift units. Therefore, the number of signals in the image signal sequence is 8
If the number of calculation pixels is 4 pixels, the shift k that can be calculated is -4, -2, 0, 2, 4.

FIG. 15 is a plot of the correlation amount calculated by "mutual shift" using "absolute value of difference" as an operator. C (-4), C (-2), C (0), C
(2) and C (4) are shifted by k = −4, −2,
The amount of correlation at 0, 2, and 4 is shown. From this plot, the correlation amount C (-
2), C (0), and C (2), the shift amount kp that interpolates the local minimum value of the correlation as shown by the broken line in the figure using the "direct interpolation method" and at the same time gives the local minimum value. You should ask. The method of obtaining the correlation extreme value in decimal units from the correlation amount of three points is disclosed in detail in "Conventional Example 3", and therefore the description thereof is omitted here.

In order to explain the "mutual shift", FIG. 15 is a plot of the correlation amount using the operator of "absolute value of difference". In "Conventional Example 5", "mutual shift" is used as the shift method, but the method for obtaining the correlation extreme value is not particularly specified.

Next, the shift method in the "conventional example 3" will be described with reference to FIG.

As in FIG. 12, 10a and 10b are 2 in the figure.
The intensity distribution of one subject image, and the arrows 12a and 12b indicate calculation areas for correlation calculation.

The characteristic of the shift method in "conventional example 3" is 2
The two subject images are characterized as "standard part" and "reference part", and basically the image signal of the reference part is shifted while the standard part is fixed, and correlation calculation is executed. .

As shown in the figure, the operation area 12a of the image signal 10a of the reference portion is fixed and the operation area 12b of the image signal 10a of the reference portion is fixed during the shift of k = -2 to 2.
Only move. In this shift method, only one of the image signals is moved, so that the "mutual shift" in "Conventional example 5" described above is used.
Unlike the case, the correlation calculation for each shift is possible.

Now, at shift k = -2 or 2, the calculation area 12b of the image signal 10b at the reference portion abuts on the end of the entire signal area and cannot be moved any further. Therefore, in order to calculate the shift k = −3 or more, the image signal 10 of the reference portion, which should originally be fixed, is used.
The calculation area 12a of “a” is moved by two pixels so that a relative shift of two images, k = −3, is realized. The same applies to shifts after k = 3.

The shift method in which the calculation region of one image signal is fixed and only the other calculation region is moved in this way is hereinafter referred to as "single image shift".

FIG. 16 shows a plot of the correlation amount calculated by the operator "absolute value of difference" of "conventional example 3" and the shift method "single image shift".

In the figure, C (-4) to C (4) represent the correlation amounts at shifts k = -4 to 4, respectively.

From this plot, the correlation quantities C (-1), C (0), C can be detected in order to detect the correlation extreme value in decimal units.
From (1) or C (0), C (1), and C (2), the local minimum value of the correlation is interpolated as shown by the broken line in the figure by using the "direct interpolation method", and at the same time the local minimum is obtained. The shift amount kp that gives a value may be obtained.

In both "Conventional example 5" and "Conventional example 3", the shift amount in decimal units that gives the correlation extreme value is obtained from the correlation amounts at three points. While three correlation amounts separated by 2 shifts are used, in "Conventional example 3", three correlation amounts separated by 1 shift are used.

Next, the shift method in "Conventional example 2" and the method of obtaining the correlation extreme value will be described with reference to FIG.

Similar to FIGS. 12 and 13, reference numerals 10a and 1 in FIG.
0b indicates the intensity distribution of the two subject images, and the arrows 12a and 12b indicate the calculation areas for the correlation calculation.

As shown in the figure, the characteristic of the shift method of the "conventional example 2" is that the image signals 10a and 10b are obtained at the shift k = 0.
The whole signal range is calculated, and the calculation range is symmetrically reduced from one end of the calculation region as it is shifted from k = 0.

This shift method is similar to the case of "mutual shift" in "Conventional example 5", but in "mutual shift", the absolute positions of the calculation regions of two image signals are changed as the number of calculation pixels remains constant. It changes mutually and symmetrically.
Therefore, the change in relative position between the two calculation areas is performed in units of two shifts.

On the other hand, in the "conventional example 2", the number of operation pixels changes with the shift, and the absolute position of one operation area of the two image signals changes depending on the positive or negative sign of the shift k. I haven't let it happen. Therefore, the two calculation areas are performed in units of one shift.

The shift method in which only one of the calculation regions is moved while the number of calculation pixels of the two image signals is reduced will be referred to as "calculation number variable shift" hereinafter.

It should be noted here that in the "conventional example 2", the arrows 12a, 1 of FIG.
This means that the correlation calculation is not performed in the calculation area itself shown in 2b. FIG. 14A is shown to explain the difference between the “conventional example 2” and other conventional examples. In the “conventional example 2”, since the “differential interpolation method” is actually used as the method for detecting the correlation extreme value, the calculation areas are shifted from each other by one pixel at a certain shift, as shown in FIG. Two operation area (15a, 15b) pairs and (16a, 16) separated by ± 1 shift about the shift
b) The correlation amount is calculated in pairs, and the difference is calculated.

FIG. 17 shows the correlation difference amount D (-
4) to D (4) are plotted. The operator is "absolute value of difference".

Since the subject image given as an example is originally shifted by 0.5 pixels, the correlation amount should take an extreme value at shift k = 0.5. As is clear from the figure, the correlation difference amount changes its sign at shifts k = 0 and k = 1, and the correlation difference amount is zero between shifts k = 0 and k = 1, that is, it is an original function. It means that there is a fractional shift amount that gives the extreme value of the correlation amount. Therefore, in the “conventional example” 2, two points of the correlation difference amount D (0) and D (1) are interpolated to obtain the shift amount kp where the correlation difference amount is expected to be zero. The interpolation method for obtaining the correlation extreme value in decimal units from the correlation difference amount between two points is disclosed in detail in "Prior art example 2", and therefore the description thereof is omitted here.

The image shift detection method disclosed in the "Prior art example 4" by the present applicant is the same as the "prior art example 2" in the "differential interpolation method" for the correlation extreme value interpolation method and the shift method.
Although it is a "variable number shift", it is a method using "large value" or "small value" as an operator.

FIG. 18 shows correlation difference amounts E (-4) to E, which are obtained by plotting the correlation difference amounts according to the "conventional example 4".
(4) represents a case where "large value" is used as an operator, and correlation difference amounts F (-4) to F (4) represent a case where "small value" is used as an operator. The method of obtaining the correlation extreme value kp equal to or less than the shift unit may be the same as that in the "conventional example 2".

As described above, "conventional example 2"
In the above, the number of calculation pixels of the correlation calculation changes according to the shift, but the number of calculation pixels is allowed to change because the correlation extreme value is detected by the difference amount as described above. In the case of the "direct interpolation method" of detecting the direct correlation extreme value, when the number of calculation pixels in each shift changes, the amount of information contributing to each correlation amount is not uniform,
It may happen that the correct extreme value cannot be detected. In other words, the "direct interpolation method" of directly interpolating the extreme value from the correlation amount is based on the implicit assumption that the correlation amount changes at a constant rate according to the shift.
If the number of pixels used for the calculation of the correlation amount changes due to the shift, the correlation amount changes due to two factors, the shift and the number of calculation pixels, and the interpolation calculation cannot be established.

Therefore, in the case of using the "variable operation number shift" as in the "conventional example 2", it is combined with the "differential interpolation method" and the correlation amount of two shifts away is calculated by performing the difference operation as shown in FIG. 14B. The number of calculated pixels is constant for each shift.

To summarize the conventional image shift detection methods described above, The correlation calculation operators are (1.1) "absolute value of difference" (conventional examples 1, 2, 3) (1.2) "large value (or small value)" (conventional example 4) 2 ． As the interpolation method of the correlation extreme value, there are (2.1) “direct interpolation method” (conventional example 3) (2.2) “differential interpolation method” (conventional examples 1, 2, 4). The correlation calculation shift method includes (3.1) "mutual shift" (conventional example 5) (3.2) "one-sided image shift" (conventional example 3) (3.3) "calculation number variable shift" (conventional example) 2.4) However, it is better not to use the "variable number of arithmetic shifts" in (3.3) above in combination with the "direct interpolation method" in (2.1).

[0057]

As described above, various methods for detecting image shift have been proposed, but each method has superior or inferior performance depending on the observed object image.

With respect to a geometrical object image in which no noise exists at all, the correct image shift amount can be detected by any method. However, if low-frequency noise such as random noise from the photoelectric conversion sensor or ghost of an optical system is mixed in the object image signal, S / N superiority or inferiority depending on the object image occurs due to a difference in method.

As an example, when the object image to be observed is a pattern such as an edge pattern that is entirely flat and has a brightness change only partially, the brightness change part is the calculation area of the correlation calculation. If it exists at the end of the image, the method using the “large value (or small value)” is better than the image deviation detecting method using the “absolute value of difference” as the operator. N is excellent. The reason for this is described in detail in Japanese Patent Application Laid-Open No. 60-101513 (hereinafter referred to as "conventional example 6") by the present applicant.

When low frequency noise such as ghost of the optical system is mixed in the subject image signal, the extreme value is generally obtained from the correlation difference amount as an interpolating method of the correlation extreme value, although it depends on the subject. The error tends to be smaller in the “direct interpolation method” that directly obtains the extreme value from the correlation amount than in the “differential interpolation method”. The main reason for this is considered to be how many correlation amounts are used in the interpolation calculation. "Differential interpolation method" is 2
Although the interpolation calculation is performed using the correlation difference amount of the points, since the difference amount is calculated from the correlation amount of the two points as shown in Expression (4), the interpolation amount is practically used using the correlation amount of the four points. It is nothing but a calculation. On the other hand, in the "direct interpolation method", 3
The point correlation is used. Therefore, if each shift correlation amount changes due to noise, the "differential interpolation method" using a larger number of correlation amounts is more susceptible to noise.

As a shift method of the correlation calculation,
It can be said that “mutual shift” is S / N disadvantageous as compared with other methods. In the case of "mutual shift", the correlation amount can be obtained only in units of 2 shifts as described above. In the other two methods, "single-image shift" and "variable operation number shift", one shift unit is obtained. Therefore, in the "mutual shift" interpolation calculation, the position of the correlation extremum to be obtained is calculated using the correlation amount at a shift that is farther away than other methods. It is easily expected that the error of the interpolation calculation will increase if the inter-coordinate distance of the correlation amount used for the interpolation calculation increases.

Summarizing the advantages and disadvantages of the image shift detection method, which have been described so far, according to the procedure, the operator is “large value (or small value)” and the correlation extreme value interpolation method is “direct”. The "interpolation method" is generally superior in terms of S / N to the shift method of the correlation calculation if the "interpolation method" is combined with the "direct interpolation method" other than "mutual shift". You can

However, the image shift detecting method in which the above-mentioned preferable procedures are combined cannot detect the correct image shift amount. FIG. 19 shows a plot of the correlation amount by this combination. "Large value (or small value)" is used as an operator, "direct interpolation method" is used as a correlation extreme value interpolation method, and "single-image shift" is used as a shift method. The subject image signal is assumed to be a case where the geometric edge image is displaced by 0.5 pixel, as in the above-described examples.

When the "large value" is used as the operator, the correlation amounts A (-4) to A (4) are correlation amounts B (-4) to B (B).
(4) represents the case where "small value" is used as an operator. As can be seen from the figure, no matter which of the above two correlation amounts is used, the extreme value is not shown at the position of 0.5 pixel which is the original image shift amount, that is, the shift k = 0.5, which is correct. Image shift detection is impossible.

However, the subject image is as shown in FIG.
In the case of the geometrical bar pattern as shown in (b), correct image shift detection is possible even with the operator of "large value (or small value)". A plot of the correlation amount in this case is shown in FIG. It is shown in (b). Correlation amount A (-4) to A (4), correlation amount B (-4)
B (4) are the correlation amounts when the operator is the “large value” and the “small value”, respectively, and the object image is shifted by 0.5 pixels between the two images as in the previous examples. The case is assumed. In this case, the shift amount kp of the correlation extreme value represents 0.5 pixel which is the original image shift amount, and the correct extreme value can be detected regardless of which of the above two correlation amounts is used.

The reason why the operator "large value (or small value)" has such a property can be considered as follows.

In the first place, the correlation function is a function in which the correlation amount becomes minimum (or maximum) at the shift position where the two signal sequences are most similar, and the correlation amount increases (or decreases) as the distance from the shift position increases. . However, when the “large value” is used as an operator and the calculation area of one image is fixed and the correlation amount of the “left-right asymmetric” image signal such as an edge image is calculated, when the shift sign is positive. A situation occurs in which the correlation amount changes according to the shift, but when the shift amount is negative, the correlation amount does not change even if the shift is changed. In other words, if the edge part of another image enters the "envelope" created by the luminance signal of the fixed nidge image, even if the image that entered is shifted in that state, "large value" This is because the operator always extracts a constant signal along the envelope of the fixed image, and no change appears in the correlation amount.

In the case of "mutual shift", since the calculation area is constantly changing without fixing one image, the above condition does not occur. Therefore, regardless of the sign of the shift, the correlation amount depends on the shift. Changes, and it becomes possible to detect the correlation extreme value.

After all, the operator "large value sum (or small value sum) of the corresponding two signals" is combined with a fixed correlation calculation in one calculation area of the image signal, and the correct image shift amount depending on the subject. Is an operator that cannot detect
It turns out that.

[0070]

SUMMARY OF THE INVENTION The present invention is intended to solve the above-mentioned problems, and prior to the correlation calculation, the object image signal to be subjected to the image shift detection is duplicated and doubled to perform the correlation calculation. Then, by obtaining the correlation amount by the "mutual shift", it is possible to detect the image shift with high accuracy and excellent S / N.

[0071]

EXAMPLES The present invention will be described below with reference to examples.

FIGS. 1 (a) and 1 (b) illustrate the expansion of the subject image signal prior to the correlation calculation and the setting of the calculation region associated with the shift.

In the figure, 10a and 10b show the intensity distributions of two object images on the surfaces of the sensor arrays 4a and 4b in FIG. 10, and in this example, a simple geometric edge image is 0.
The figure shows the case where there is a shift of 5 pixels. The number of signals in the image signal series is 8 pixels. Further, in the figure, IA (0) to IA (7) indicate respective pixel outputs of the sensor array 4a, and IB (0) to IB (7).
Indicates each image output of the sensor 4b.

To enlarge the image signal, each pixel signal of the image signal series 10a and 10b of FIG. 1A is duplicated and enlarged to double the number of signals, as shown in 11a and 11b of FIG. 1B, respectively. ,
The conversion is performed by converting into an image signal sequence having 16 signals. Specifically, the first pixel signal IA of the image signal series 10a
(0) is the first JA (0) and the second JA (0) of the image signal sequence 11a.
The 2nd pixel signal IA (1) of the image signal series 10a is the 3rd JA of the image signal series 11a.
(2) and the fourth JA (3) are the enlargement operations.

The object image signal 11 thus enlarged
Correlation calculation is performed on a and 11b. 12a, 12 in the figure
The arrow of b represents the calculation area for correlation calculation, and is the same "mutual shift" as in FIG. It should be noted that the “mutual shift” in FIG. 7 moves two images at the same time, and therefore, a correlation calculation is performed in units of two shifts, but in FIG.
Since it is moved after being doubled, it is possible to convert the original image signal sequence into one shift unit for correlation calculation.

FIG. 2 shows a plot of the correlation amount for each shift in the correlation calculation according to the present invention. The operator uses “large value (or small value)”. Correlation amount A (-
4) to A (4) and the correlation amounts B (−4) to B (4) are the correlation amounts when the operators are “large value” and “small value”, respectively.

As can be seen from the figure, the correlation amount can be obtained in units of one shift and the correlation extreme value can be detected correctly in spite of the "mutual shift".

FIG. 3 shows a specific circuit block diagram for realizing the focus detection device according to the present invention.

In the figure, reference numeral 40 denotes a sensor device in which two sensor rows 4a and 4b are arranged. Reference numeral 50 is a sensor drive circuit, and 60 is a microcomputer that controls the entire focus detection apparatus and performs signal processing. Microcomputer 6
The sensor drive circuit 50 is controlled by the control signals 61, 62, 63 output from 0, and the control signals 61, 62,
Sensor drive signals 51, 52, 53 generated from 63 are provided to the sensor device 40. The actual control of this area is not directly related to the present invention, and therefore detailed description thereof will be omitted.

When a predetermined accumulation of a sensor array of accumulation type sensors is performed, the analog image signal 41 is transmitted to the sensor device 4
0 is output, and the amplification circuit 54 in the sensor drive circuit 50 is output.
Then, the amplified analog image signal 55 is input to the analog input of the microcomputer 60. The microcomputer 60 has control signals 61, 6 output by itself.
The analog image signal 55 that is input is A / D converted in synchronism with 2, 63 and RA
Digital image signals are sequentially stored in a predetermined address of M.

When the storage of the image signal is completed, the microcomputer 60 executes the focus detection program stored in the ROM.

FIG. 4 shows a flow chart of the program of the microcomputer.

Step (S1
00) "focus detection start" routine N is called, and the input of the above-mentioned image signal is executed in step (S101). The image signal 10a in FIG. 1A is converted to IA (0) to IA
Let (7) and 10b be IB (0) to IB (7).

In the next step (S102) (S103), processing for expanding the image signal to double the signal amount is performed. The magnified image signals 11a and 11b shown in FIG. 1B are set to JA (0) to JA (15) and JB (0) to JB (1), respectively.
5), JA (2 · i) ← IA (i) (5) JA (2 · i + 1) ← IA (i) (6) JB (2 · i) ← IB (i) (7) JB (2 · i + 1) ← IB (i) (8) However, i = 0 to 7. (5) to (8) in IA
(I), IB (i) as JA (0) to JA (15), JB
(0) to JB (15). In addition, step (S1
02) represents a loop process for i, and formulas (5) to (8) are executed in step (S103) in the loop.

Next Steps (S104), (S105)
At, the correlation calculation is performed within the correlation calculation range shown in FIG. In this case, using "small value" as the operator, the correlation amount at shift k is COR (k), and a certain calculation of the following equation (9) is performed.

[0086]

[Outside 3] However, k = −4 to +4.

The processing in steps (S104) and (S105) means the correlation calculation in the equation (9) using the smaller value for each calculation range for each k value shown in FIG. 1 (b). That is, when k = -4, signals JA (0) to J (J)
For A (7) and signals JB (8) to JB (15), the smaller right value of JA (0) and JB (8) is set to J
The smaller value of A (1) and JB (9) is set to J
A (2) and JB (10) pair, JA (3) and JB (1
1) pair, JA (4) and JB (12) pair, JA
Pair of (5) and JB (13), JA (6) and JB (1
4) is selected, and a smaller value is selected from each pair of JA (7) and JB (15), and the addition value of the selected smaller right values is obtained as COR (-4). .

When k = -3, signals JA (1) to JA (1) -J
For A (8) and signals JB (7) to JB (14), the addition value of the smaller values of the corresponding signals is COR (-.
3) is required. Also, when k = −2 to +4, the above COR (− is similarly set for each range shown in FIG. 1B.
2) to COR (+4) are required.

Correlation amount CO from shift -4 to +4
When the calculation of R (k) is completed, step (S106)-
The correlation extreme value is detected in (S110). When “small value” is adopted as the operator, the correlation amount COR (k) should be maximized at the shift with the maximum correlation. Therefore, in step (S106), 0 is given as an initial value to the variable CORMAX which stores the maximum value of the correlation amount in advance.

Step (S107) represents a loop process for k, and searches for the extreme value of the correlation amount over the range of k = −3 to +3.

"Maximum" and "greater than two points before and after" are used as the conditions for the extreme value search. Step (S
108), it is determined whether or not the correlation amount COR (k) is the maximum, and in step (S109), the COR (k) is set to the front or rear two.
It is determined whether or not the point correlation amounts are larger than COR (k-1) and COR (k + 1). Only when both conditions are satisfied, the process proceeds to step (S110), and step (S11)
0), the maximum correlation value variable CORMAX is set to COR.
The value of (k) is updated, and at the same time, the current value of shift k is stored in the extreme value shift kx. In order to perform the determination in the step (S109) of comparing the correlation amount with the values of two points before and after, the loop processing of the step (S107) is performed with k ranging from -3 to +3, and the shift range (corresponding to the actual correlation calculation) -4 to +
4) It is narrower than

Now, B (-4) to B (4) in FIG. 2 are the correlation values COR obtained in steps S104 and S105.
The extreme value search will be described as representing (−4) to COR (+4).

First, after setting CORMAX = 0 in S106, COR (-3) and CORMAX are compared in S108. Since COR (-3) = B (-3)> CORMAX, COR (-3) and CO are obtained in step S109.
The magnitude comparison between R (-4) and COR (-3) and COR (-2) is performed.

In this case, COR (-2) = B (-2)>
Since COR (-3) = B (-3), step S1
Without executing step 10, k = -2 is set, and steps S108 and S109 are compared again. As a result, CO
R (-1) = B (-1)> COR (-2) = B (-2)
Therefore, without executing step S110, k
= -1, and again the above steps S108 and S10.
9 is executed. As a result, COR (0) = B (0)> C
Since OR (-1) = B (-1), S1 in this case as well.
Instead of executing step 10, k = 0 is set and steps S108 and S109 are executed again. In this case, COR
(0) = B (0)> COR (-1) = B (-1) and COR (0) = B (0)> COR (1) = B
Since it is (1), step S110 is executed, and COR
Set COR (0) = B (0) as MAX, kx
Is set to 0, and then k = 1, and the above S is again set.
108 and S109 are executed. In the subsequent processing from k = 1 to 4, the conditions in steps 108 and 109 are not satisfied, so after performing processing with k = 4, step S11 is performed.
Go to 1.

When the correlation extreme value is detected in steps (S107) to (S110), the next step (S11)
In 1), the correlation extreme value is interpolated in order to obtain the image shift amount on a scale smaller than the shift unit.

As the interpolation method, the method of performing the interpolation from the maximum value shift and the correlation amount of a total of three points before and after the shift as described above is used. Here, as an example, the interpolation calculation is performed by the method of the conventional example 3.

As shown in FIGS. 5 (a) and 5 (b), the correlation amount COR in which "small value" is calculated in the shift kx.
When (kx) becomes the maximum, COR (kx-1) and C
Depending on the magnitude of the value of OR (kx + 1), COR (kx + 1
-1) ≤ COR (kx + 1) (Fig. 5 (a))
And COR (kx-1)> COR (kx1) (FIG. 5 (b)). CO as shown in FIG.
If R (kx−1) ≦ COR (kx + 1), then CO
The k-coordinate of a straight line passing through R (kx-1) and COR (kx) and the intersection of the straight line and the straight line passing through COR (kx + 1) with the opposite sign to COR (kx + 1) is set as an image shift amount kp of a small number or less. Through the above processing, the image shift amount kp which is the focus detection result is obtained.

As shown in FIG. 5B, COR (kx-1)>
In the case of COR (kx + 1), COR (Kx) and CO
The k coordinate of a straight line passing through R (kx + 1) and an intersection of the straight line and the straight line passing through COR (kx-1) with the opposite sign to kp is kp.
And The formula is as follows. (I) When COR (kx−1) ≦ COR (kx + 1),

[0099]

[Outside 4] (Ii) When COR (kx-1)> COR (kx + 1),

[0100]

[Outside 5]

As described above, the maximum correlation COR (K
x) and two points COR (kx−1) before and after it, COR (k
The image shift amount kp is obtained from (x + 1),
In addition to the method of interpolating kp with a straight line as shown in FIGS. 5A and 5B, the correlation amounts COR (kx−1) and COR (K
x) and COR (kx + 1) may be obtained as an extreme value of a quadratic function passing through three points.

FIG. 6 is an explanatory view showing another example of the above-mentioned interpolation calculation, and FIG. 6 shows an example in which the image shift amount kp is interpolated by a quadratic function.

In FIG. 6, the maximum correlation COR (Kx) and the two points COR (kx−1) and COR (kx + 1) before and after it.
Is given, a quadratic function COR (K) passing through the three points
= Ak ² + bk + c coefficients are obtained, and the extrema of the quadratic function

[0104]

[Outside 6] Is regarded as kp. kp is the following equation (12)
Required by.

[0105]

[Outside 7]

Although FIG. 4A shows the case where "small value" is used as the correlation operator, a part of the flowchart when "large value" is calculated is shown in FIG. 4 (b). Shown in.

Steps (S204) to (S) of FIG.
210) corresponds to (S104) to (S110) in FIG. Explaining only the difference between the two, the operator "large value" is used when calculating the correlation amount in step (S205). When the “large value” is an operator, the correlation amount is the maximum and the minimum.

In the next step, the initial value 9999 is stored in the variable CORMIN representing the minimum value of the correlation amount. This is not limited to 9999, and a large value that the correlation amount cannot take may be stored.

Steps (S208), (S209),
In (S210), as the extreme value search condition, "minimum" and "smaller than two points before and after" are set. If the condition is satisfied, the minimum value variable C is set in step (S210).
Update ORMIN and shift kx.

Regarding the interpolation of the correlation extremum, the total is slightly different depending on whether the maximum value or the minimum value is calculated when interpolating with a straight line. That is, COR (kx
When −1) ≧ COR (kx + 1), the equation (10) is C
When OR (kx−1) <COR (kx + 1), the expression (1
1) applies.

When interpolating by a quadratic function as shown in FIG. 6, it is naturally obtained by the equation (12) regardless of the operator.

In the present invention, the new number of the image signal sequence is set to 2
Since the number of times is doubled, if the correlation calculation is performed with the doubled signal amount as it is, the calculation time is significantly increased as compared with the conventional case.

However, if the correlation calculation is devised as shown in FIG. 7, the calculation time hardly increases as compared with the conventional case.

Arrow pairs (13a, 14a), (13 in FIG.
b and 14b) respectively indicate the correlation calculation areas corresponding to the arrows 12a and 12b in FIG. Originally, the magnified image signal series 11a and 11b are respectively the image signal series 10a and 10b.
It is generated from b. Therefore, the magnified image signal sequence 1
1a area 12a and magnified image signal sequence 11b area 12b
The correlation amount between the two is equal to the value obtained by adding the correlation amount between the region 13a of the original image signal sequence 10a and the region 13b of the original image signal sequence 10b to the correlation amount between the regions 14a and 14b. Then, in the even shift, the regions 13a and 14a are equal, and in the odd shift, the region 13a is a region obtained by adding one image at both ends of 14a.

Therefore, the correlation amount in the even shift is a value obtained by doubling the correlation amount between the regions 14a and 14b, and the correlation amount in the odd shift is a value obtained by doubling the correlation amount between the regions 14a and 14b. In addition, the correlation amount of only the signals at both ends of the regions 13a and 13b is added. If the calculation is performed in this way, the correlation calculation time in the magnified image signal series hardly increases as compared with the calculation time in the original image signal series.

FIG. 8 shows a part of the flow chart when the calculation method of FIG. 7 is used. Step of FIG. 8 (S301)
~ (S305) is the step (S10) in FIG.
1) to (S105).

The step (S301) is the step (S1)
Image signal IA (0) to IA (7), IB
(0) to IB (7) are obtained.

The next step (S302) represents the loop processing for shift k, and k = -4 in the loop.
Perform processing from to +4.

As is apparent from FIG. 7, the method of setting the calculation area differs depending on whether the shift k is an even number or an odd number. That is,
When the shift k is an even number, the correlation amount COR (k) is

[0120]

[Outside 8] Becomes On the other hand, when the shift k is odd, the correlation amount CO
R (K) is

[0121]

[Outside 9] Becomes COR in steps (S303) to (305)
(K) is calculated.

In this way, if the correlation amount COR (k) with the shift k of −4 to +4 is obtained, then FIG.
It suffices to search for correlation extreme values and interpolate extreme values after the step (S106) in (a).

In FIG. 8, “small value” is used as the operator of the correlation operation, but when “large value” is used as the operator, “Min” in the figure may be replaced with “Max”. In that case, steps (S206) to (S21) of FIG.
0) and step (S111) of FIG. 4A, the image shift amount kp can be obtained.

In the above-described embodiment, as described in "Conventional example 6", the observed object image is an image in which the image is so flat that there is only a partial change in luminance like an edge pattern. In terms of S / N, the method using "large value (or small value)" is more advantageous than the image shift detection method using "absolute value of difference" as the operator for correlation calculation. Therefore, in the above description, "large value (or small value)" has been used as an operator. But,
The operator "absolute value of difference" is also applicable to the image shift detection method of the present invention.

As described above, the operator "absolute value of difference" can be used in combination with "one-image shift" and "alternate shift", which are shift methods of correlation calculation, and therefore, the "difference of absolute value" of the present invention is intentionally used. It seems that it is not necessary to combine it with "mutual shift", but since "single image shift" fixes the calculation area of the reference portion of the image signal, it may be inconvenient depending on the observed object image. This is because in the "single image shift", the calculation area of the fixed reference image is changed from shift k = 2 to 3 and k = -2 to -3.
This is because the correlation amount may be discontinuous here.

When an object image with a 0.5 pixel shift as shown in FIG. 13 is targeted, kp = 0.5 can be interpolated as in the plot of the correlation amount in FIG. 16, and correct image shift detection is possible. Is. However, when the subject image is displaced by 2.5 pixels, shift k = 2,3,4 or k = 1,2,3
Since the correlation extremal value in decimal units is interpolated and calculated from the correlation amounts of the above three points, if the continuity of the correlation amount is lost at shifts k = 2 and k = 3, it may cause an error to some extent.

In the "conventional example 3", in consideration of this, the shift range covered by one calculation area of the reference image partially overlaps with the shift ranges of the other calculation areas.
As a result, the correlation amount of the three points used for the interpolation calculation can be the same in the calculation region of the reference image, and the continuity of the correlation amount of the three points is maintained.

In the image shift detecting method of the present invention, since there is no concern about the continuity of the correlation amount, the device of "conventional example 3" is not necessary and the calculation time does not increase accordingly.

FIG. 9 shows a plot of the correlation amount for each shift when the “absolute value of difference” is applied as an operator to the image shift detection method of the present invention. The object image is the same geometric edge image as that in FIG. 1 shifted by 0.5 pixels.

As is apparent from the figure, the correlation amount does not become discontinuous between shifts k = 2 and 3 or k = -2 and -3, and the correct image shift amount can be detected.

[0131]

As described above, according to the present invention,
Prior to the correlation calculation, the object image signal to be detected for image shift is duplicated and enlarged twice, and the correlation amount is obtained by the "mutual shift" in the correlation calculation, so that image shift detection superior in terms of S / N can be performed. By using “large value (or small value)” as an operator, it is possible to reduce subject dependency, which enables highly accurate focus detection.

[Brief description of drawings]

FIG. 1 is a principle explanatory diagram of a correlation calculation according to the present invention.

FIG. 2 is a plot diagram of the correlation amount of the first embodiment of the correlation calculation according to the present invention.

FIG. 3 is a circuit block diagram of a focus detection device of the present invention.

FIG. 4 is an explanatory diagram showing a flowchart of a microcomputer in the focus detection device.

FIG. 5 is an explanatory diagram of extreme value detection of a correlation amount.

FIG. 6 is an explanatory diagram showing another example of detecting the extreme value of the correlation amount.

FIG. 7 is another explanatory diagram of the correlation calculation according to the present invention.

8 is an explanatory diagram showing a flowchart of a microcomputer that realizes the correlation calculation in FIG. 7. FIG.

FIG. 9 is an explanatory diagram for explaining another example of the correlation amount interpolation operation.

FIG. 10 is an explanatory diagram for explaining the principle of a focus detection optical system.

11 is an explanatory diagram for explaining the principle of the focus detection optical system together with FIG.

FIG. 12 is an explanatory diagram for explaining the principle of a conventional correlation calculation.

FIG. 13 is an explanatory diagram for explaining the principle of another conventional correlation calculation.

FIG. 14 is an explanatory diagram for explaining the principle of another conventional correlation calculation.

FIG. 15 is an explanatory diagram showing a conventional correlation calculation result.

FIG. 16 is an explanatory diagram showing another example of a conventional correlation calculation result.

FIG. 17 is an explanatory diagram showing a difference amount calculation result of a conventional correlation calculation.

FIG. 18 is an explanatory diagram showing another example of the difference amount calculation result of the conventional correlation calculation.

FIG. 19 is an explanatory diagram showing a correlation calculation amount according to a combination of conventional examples.

FIG. 20 is an explanatory diagram showing an example of a subject image signal.

FIG. 21 is an explanatory diagram showing another example of the correlation calculation by combining the conventional examples.

[Explanation of symbols]

1 Shooting lens 4a photoelectric conversion sensor array 4b photoelectric conversion sensor array 40 sensor device 60 microcomputer

Claims (3)

[Claims] 1. A light flux is received on a first and a second sensor array via an optical system, and an optical signal is obtained from a relative displacement amount with respect to the first and second signals corresponding to the light flux received by each sensor array. In a focus detection device for detecting the focus state of a system, the signals from a plurality of pixels forming each sensor are duplicated and expanded to a doubled signal amount, and the expanded first and second signals are A focus detection device characterized by performing a correlation calculation. 2. The correlation calculation is a correlation calculation using a large value of corresponding signals in the first and second signals or a small value of corresponding two signals as an operator. The focus detection device according to item 1. 3. The focus detection apparatus according to claim 1, wherein the correlation calculation is a correlation calculation using an absolute value of a difference between corresponding signals in the first and second signals as an operator.