JPH0473564B2 - - Google Patents

Description

[Detailed description of the invention] Industrial fields of use For example, in movie cameras, still cameras, or single-lens reflex cameras that use a solid-state image sensor or image pickup tube as an image sensor, an auto camera that automatically focuses on a desired subject is used. Focus devices are known.

The present invention relates to an image processing device suitable for use in obtaining an evaluation value of the sharpness of an image for focus detection in, for example, such an autofocus device.

BACKGROUND TECHNOLOGY AND PROBLEMS This type of autofocus device includes an active method such as a triangulation method using ultrasonic waves and infrared rays, and a passive method using contrast detection, coincident image detection, etc. Among these, the following is known as an example of a passive contrast detection autofocus device.

Figure 1 shows the overview, and this is TTL
This is the type used in single-lens reflex cameras.

In FIG. 1, light passing through a lens 1 is incident on a sub-mirror 2, and the transmitted light is incident on a film surface 3. On the other hand, the light reflected by the sub-mirror 2 enters the beam splitter 4. For this beam splitter 4, one row of picture elements (sensor ) are provided with a sub-imaging device S ₁ and a sub-imaging device S ₂ each consisting of a one-dimensional array of solid-state imaging devices, such as CCDs. In the figure, broken lines 6 and 7
The sub image sensor S ₁ for the film surface 3 is shown by
At the equivalent position of S ₂ , they are spaced an equal distance l from the front and back of the film surface 3 .

Although not shown, a main mirror is provided between the sub-mirror 2 and the lens 1 so as to be perpendicular to the surface direction of the sub-mirror 2, and incident light is directed to the sub-mirror 2 from the sub-imaging elements S ₁ and S. ₂ , the light is made to enter the focusing plate provided on the opposite side.

By the way, whether or not the image is in focus can be evaluated by whether or not the image is clear. It is well known that whether or not an image is clear can be determined by determining the spatial frequency of the image and evaluating the MTF (modulation transfer function) with respect to the amplitude. Considering the conditions that it can be done and is inexpensive, it is difficult in practice.

On the other hand, in general, whether an image is clear or not can often be determined by whether the outline of the subject is sharp.

This contour information is obtained by differentiation, but in computer processing, it is obtained by using the difference in output between picture elements (sensors) of the sub-imaging device.

Here, for the sake of simplicity, half of the screen is white,
If we consider a subject whose brightness changes gradually, with the remaining half being black, as shown in Figure 2,
Contrast E is strongest where the object is accurately focused (focus point), and contrast E decreases even if the object is shifted from this position to the front or back.

Considering the evaluation function that represents this contrast,
This means that when an image is obtained using the one-dimensional solid-state image sensors S ₁ and S ₂ as described above, the n-th image and the n+
It can be expressed as the difference between the output In of the first sensor and In+1. That is, = _N-1 〓 ⁿ⁼⁰ (In+1-In)...(1). Considering a sufficiently long array of one-dimensional image sensors, the array direction is set to x, and E=∫dl/dxdx...(2) This simply expresses the integral of I and has nothing to do with sharpness. I get used to it. In order to avoid this drawback, some method must be used to perform nonlinear processing on In+1-In in equation (1). For this nonlinear processing, analog calculations are performed,
It takes advantage of the nonlinearity of the image sensor.

Assume that the contrast evaluation function E is obtained by performing the above-described nonlinear processing. This evaluation function E
is applied to the outputs of the above-mentioned sub-imaging devices S ₁ and S ₂ , and the function curve is obtained by plotting the focus shift on the horizontal axis and the contrast on the vertical axis, as shown in FIG. 3. In FIG. 3, curve 8 is the contrast C ₁ curve of element S ₁ , and curve 9 is the contrast C ₂ curve of element S _2.
Each curve is shown.

That is, since the sub-imaging elements S ₁ and S ₂ measure the contrast before and after the original focus, as the lens 1 is moved and focusing is performed, two contrast curves whose phases are shifted are obtained.
As described above, this device is configured so that the correct focus position is located halfway between the front focus side image sensor S ₁ and the rear focus side image sensor S ₂ , so that the contrast curves 8 and 9 The in-focus position is when the values are the same. Further, if contrast C ₁ >contrast C ₂ , it indicates that the current focus is on the front, and if contrast C ₁ <contrast C ₂ , it indicates that the current focus is on the back.

In this way, autofocus is possible by using the contrast information of the outputs of the two image sensors S ₁ and S ₂ placed at positions shifted forward and backward relative to the original focus.

However, in the case of conventional devices, including the device shown in Figure 1, the image sensors S ₁ and S ₂ are line sensors, that is, they are one-dimensional arrays, so they are not suitable for images where the contrast changes in the direction of the sensor array. However, in the case of an image in which there is no change in contrast in the array direction and only in a direction intersecting the array direction, the focus position cannot be detected.

Furthermore, conventionally, as mentioned above, when introducing a contrast evaluation function, arithmetic processing using analog calculations or the nonlinearity of the image sensor itself has been performed, but such calculation processing does not have a dynamic range. When the distance became narrower and the subject became slightly darker, distance measurement became impossible.

Purpose of the Invention In view of the above points, the present invention makes it possible to obtain an evaluation value of image sharpness that has no direction dependence on the contrast of the image and can expand the dynamic range. It seeks to provide something.

Summary of the Invention In this invention, for example, a surface image sensor is used as a sub-image sensor for distance measurement in the above example, and a logarithmic compression conversion is performed to logarithmically compress the image output from the surface image sensor and the surface image sensor. In order to obtain the evaluation value of the sharpness of the image based on the image output logarithmically compressed by the logarithmic compression conversion means, Using adjacent picture element c, vertically adjacent picture element b, and diagonally adjacent picture element a, the Laplacian at picture element d is calculated based on |3d-(a+b+c)|, and this is calculated using multiple pixels of the surface image sensor. calculation means for calculating the sum of the Laplacians of the picture elements; focus control means for changing the focus to obtain focus position information corresponding to the maximum value of the output value from the calculation means;
By providing focus selection means for selecting a desired focus position when a plurality of focus position information is obtained corresponding to the maximum value of the output value from the calculation means by changing the focus, By using a manual switch as the focus selector means, it is possible to perform accurate contrast detection and widen the dynamic range of the output. This is what I did.

Embodiment Hereinafter, an embodiment of the present invention will be described with reference to FIG. 4 and subsequent figures. This embodiment is an example of a novel autofocus device that also overcomes other drawbacks of the prior art described above.

That is, in the case of the above-mentioned autofocus device, since the lens system is moved in conjunction with distance measurement, the operating speed is slow and the movement is unnatural.

Also, if the focus is greatly out of focus and it becomes the undetectable area in Figure 3,
The drawback is that distance measurement is no longer possible.

Also, since a beam splitter is used,
Another drawback is that the amount of incident light decreases depending on the number of stages, and the operating range of autofocus is narrow.

This example is a new autofocus device that has fast distance measurement speed and can respond quickly even when the distance suddenly changes from infinity to close-up.

That is, FIG. 4 shows an optical system according to an example of the present invention, in which 11 is a lens, 12 is a main mirror, and 13 is a lens.
14 is a main image pickup device made of, for example, a CCD, and 15 is a sub-image pickup device also made of, for example, a CCD, and the sub-image pickup device 15 is used for distance measurement in autofocus.

The sub-imaging device 15 is a near-field imaging device in which sensors are arranged two-dimensionally. Note that this sub-image sensor 15 is not the same as the main image sensor 1 due to its purpose of use.
It does not have to be the same size as 4, and may be a partial one that cuts out a part of the angle of view of the main image sensor 14. The number of picture elements (sensors) of this sub-imaging device 15 is, for example, 64 vertically by 64 horizontally.

Light that has passed through the lens 11 using the TTL method enters the main image sensor 14 via the main mirror 12. A part of the light passing through the lens 11 is also reflected by the main mirror 12 and guided to the viewfinder. Further, a part of the light that has passed through the main mirror 12 is reflected by the sub mirror 13 and is transmitted to the sub image sensor 1.
5.

In this invention, the lens 1 is used for distance measurement for autofocus as in the conventional case.
Instead of moving the sub-imaging device 15, the sub-imaging device 15 is moved by ±Δd in the optical axis direction. For this reason,
In this example, the sub-imaging device 15 is mounted on an actuator as shown in FIG. This actuator is, for example, a cylinder 1 magnetized to the S pole.
An annular body 17 magnetized to N pole is formed at a certain distance from the cylinder 16 so as to surround the cylinder 16, and a voice is formed in a groove 18 formed between the cylinder 16 and the annular body 17. Coil 19 is inserted. This voice coil 19 is attached to a base 20 to which the sub-imaging device 15 is fixed, and the base 20 and therefore the sub-imaging device 15 change depending on the current supplied to the voice coil 19. The piston moves in the direction shown by the middle arrow (the optical axis direction shown by the arrow in FIG. 4).

In this case, the origin position of the sub-image sensor 15 is at the same position as the focal length of the main image sensor 14, and the actuator causes the sub-image sensor 15 to scan a movement amount D corresponding to the front focal length of the optical system. make a piston motion as if Scanning in this case is performed in a step-driven manner using a step wave as shown in FIG.
5 is made to take a position of 16 steps in steps during its movement amount D. In other words, if it takes time T for the amount of movement D, the sub-imaging device 15 will move by T/
It moves its position in the optical axis direction every time 16=τ, and there are 16 positions d ₀ , d ₁ , d ₂ ...d ₁₅
, and it is stopped at each position for a period of time τ. Here, d ₀ corresponds to the closest focal position of the optical system, and d ₁₅ corresponds to the focal position at infinity.

Then, at each focal position of d ₀ to _{d 15} , the sharpness of the image of the sub image sensor 15 at each position is determined at each stop time τ, and information representing the sharpness at each position is obtained. is stored in memory. Then, among the sharpness information stored in the memory, a maximum position, that is, a position with high sharpness compared to other positions, is found, and based on this, the main image sensor 14 or lens 11 is moved to take appropriate action. The focal point is placed at the focal point. In this way, autofocus is achieved. In addition, the amount of movement D
The scanning speed of the above-described distance measurement by the sub-imaging device 15 to find the focal point position of maximum sharpness in the image is, for example, about 3 times/second.

By the way, in the case of this invention, the arithmetic processing for determining the sharpness of the image of the sub-imaging device 15 takes advantage of the fact that a surface imaging device is used, and expands the conventional one-dimensional processing to a two-dimensional surface. The total amount of the absolute value of the Laplacian for each picture element of the sub-imaging element for all the picture elements of the sub-imaging element 15 is used as the evaluation of the degree.

Now, consider an image in which the left half is black and the right half is white, as shown in FIG. This is the solid line 51 in the same figure.
The step function is as shown by , but in reality, it is somewhat blurred as shown by the solid line 52 due to the spatial filter of the image sensor. Here, suppose that two blurs,
Considering this, the convolution integral of the spread function gives an image of characteristics as shown by solid lines 53 and 54 in the figure. If a gradient is taken from this image by difference, the result will be as shown in FIG. In other words, the step function is shown as 61, and the blur caused by the spatial filter and the solid lines 53 and 54 are shown as solid lines 62, 63, and 64, depending on the degree of blur. The shape of the rectangular part changes. However, when this is integrated, the area does not change even if the degree of blur changes. Therefore, it is inappropriate as an evaluation function for image sharpness.

Therefore, by further calculating the difference and taking the absolute value of the Laplacian, values corresponding to the blur are obtained as shown by solid lines 71 to 74 in FIG. When this is integrated over the entire screen, an evaluation function as shown in FIG. 10 is obtained, which is large when the image is clear and small when the image is blurred. The maximum value of this function is the focal point, and focusing can be obtained by applying feedback to the optical system.

That is, take the absolute value of the Labrasian of the image output, find the total amount for the entire image,
If this is used as a sharpness evaluation function, good autofocus can be achieved.

Now, when considering the Laplacian of a certain picture element, as shown in Figure 11, find the Laplacian for the 8 picture elements A, B, C, D, F, G, H, and I around that picture element E. However, the differential calculation process is E-(A+B+C+D+F+G+H+I)/8
...(3) is known. However, if we pay attention to the fact that what is needed here is the total Laplacian for the entire image, for example in Fig. 11, the Laplacian is calculated and added sequentially for the picture elements E, F, etc., so the point is , 3 picture elements A surrounding picture element E,
It turns out that it is sufficient to find the Laplacian for B and D.

In other words, when the picture element currently being read is δ as shown in Figure 12, the left adjacent picture element is γ, the upper adjacent picture element is β, and the adjacent picture element diagonally above the left is α. The Laplacian 〓 ₁ for the element δ is given by 〓 ₁ = | δ − (α + β + γ) / 3 | ... (4) This is obtained by scanning the entire picture element of the sub-imaging device 15 and finding it for the entire image, 〓= 〓 ⁱ 〓i ……(5) If 〓 is obtained, the evaluation value of the sharpness of the image can be obtained.

In this case, the pixels in the uppermost row and the left column of the sub-imaging device 15 shown with diagonal lines in FIG. 13 are different from each other because there are no surrounding pixels. , when scanning these picture elements, no calculations are performed.

Also, when the picture element α is scanned, using the surrounding picture elements β, γ, and δ, |α−(β+γ+δ)/3|
You may also perform the calculation as follows; in this case,
In the image sensor shown in FIG. 13, surrounding pixels do not exist in the pixels in row m and column n, so when scanning these pixels in row m and column n, the above calculation is performed. Make sure not to.

Alternatively, these picture elements may not be scanned.

The image sharpness evaluation value is obtained in the above manner, and this value is obtained at each focal position of the optical system from d ₀ to d ₁₅ mentioned above, and is recorded in memory as sharpness information at each position. , the focus is controlled based on this.

By the way, for example, if a person 22, a pillar 23, and a bookshelf 24 are lined up sequentially in front of the camera 21 as shown in FIG.
For example, when a clear image is formed at positions d ₃ , d ₇ , and d ₁₀ , the total Laplacian amount 〓 becomes maximum at each position of d ₃ , d ₇ , and d ₁₀ as shown in Figure 15. . In other words, if there are multiple objects within the surface angle of the imaging lens 11, the total differential amount becomes maximum at each focal position of the camera's optical system according to the number and position thereof, and is stored in the memory. , lens 1 is placed at each focal point that becomes this maximum point.
1 or by moving the image sensor 14, the image is brought into focus.

Normally, when a plurality of objects come within a plane angle, it is desired to focus on a desired one of them. In this way, in order to focus on a desired subject among a plurality of local maximum points, the following procedure may be used.

In other words, since the desired subject to be focused is often the closest to the camera, first focus on the closest subject, the person 22 in the example of Fig. 14, and then If the object is not there, you can move it to another maximum point so that you can always focus on the desired object.

FIG. 16 shows a block diagram of an example of the control system of the device of the present invention in consideration of the above.

In FIG. 16, 30 is a focus control circuit, which consists of an arithmetic unit, a register, a digital comparator, a counter, etc.

The focus control circuit 30 obtains a stepped signal in the form of a digital signal for causing the sub-imaging element 15 to scan for distance measurement as shown in FIG. The counter described above is used for this purpose and is supplied with a clock having a period τ. The output of this counter is then supplied to the D/A converter 31 and converted into an analog signal, which is then supplied to the voice coil 19 of the actuator shown in FIG. is brought to the focal position. At this position, the image output obtained from the sub-imaging device 15 is transmitted to the logarithmic compression circuit 3 via the preamplifier 32.
3. This logarithmic compression circuit 33 is for converting the distance characteristic of light into a linear one since it is logarithmic. The output of the logarithmic compression circuit 33 is converted into a digital signal by the D/A converter 34, and then supplied to the arithmetic circuit 35 to obtain the Laplacian total amount of each picture element of the above-mentioned image, which is stored in the memory 36. , is stored as an evaluation value of the sharpness of the image at that position at an address that corresponds 1:1 to that position.

The arithmetic circuit 35 includes a shift register 351, a Laplacian arithmetic circuit 352, an adder 353, and a register 354. Sub image sensor 15
If the horizontal scanning of the imaging surface is performed sequentially from the top to the bottom, then when the number of pixels in the horizontal direction of the sub-imaging device 15 is n, the number of stages of the shift register 351 is has (n+1) stages. Outputs are taken out from the first, nth, and (n+1)th stages of this shift register 351 and supplied to the Laplacian calculation circuit 352, and the current output of the sub-image sensor 15 is Laplacian calculation circuit 3
52.

Now, assuming that the picture elements of the sub-image sensor 15 are sequentially scanned as described above and the current position of the picture element is δ, the first stage of the shift register 351 at this time,
From each stage of the nth stage and (n+1)th stage, γ,
Information on picture elements at positions β and α can be obtained. Then, in the circuit 352, the above-mentioned formula (4) is calculated, and the output thereof is supplied to the adder 353,
The summed output is stored in register 354. The output of this register 354 is supplied to an adder 353 each time the position of the picture element changes as the picture element is scanned, and is added to the calculation output of the Laplacian of the next picture element. That is, the adder 353 and the register 354 perform the calculation (formula (5)) for determining the total amount of Laplacian on the screen.

Then, the total Laplacian amount 〓 obtained in the register 354 after the scanning of all pixels of the sub-imaging device 15 is completed is written to the corresponding address in the memory 36.

In this example, when scanning the picture elements of the sub-image sensor 15, when scanning the picture element positions indicated by diagonal lines in FIG. 13, an inhibit signal is supplied to the addition circuit 353, and the addition process will be prevented from occurring.

Next, when the time τ has elapsed, the focus control circuit 30 increments the counter,
The count value changes. Then, the output of the D/A converter 31 changes, and the position of the sub-image sensor 15 is moved in the optical axis direction. Then, at this changed position, the same operation as described above is performed, and the total Laplacian amount of the image output, which is the evaluation value of sharpness at that position, is written into the corresponding address of the memory 36. The above operations are performed at all of the above-mentioned 16 positions d ₀ to d ₁₅ , thereby obtaining the stored content of the memory 36 corresponding to that shown in FIG. 15. This is repeated below.

A local maximum value is detected for the 16 pieces of information in the memory 36, and if it is a local maximum value, a marker is attached to the information. That is, memory 3
6, when writing information on the sharpness of each position, a digital comparator installed in the focus control circuit 30 compares the information on the previous and subsequent positions with the information on that position and determines the maximum value. When it is detected, a marker is added to the information and written. Alternatively, after all the information is written in the memory 36, read it out sequentially and use a digital comparator to find the maximum value,
It is also possible to attach a marker and write again at the original position.

When there is a subject like the one shown in Figure 14 relative to the camera position, the sharpness information at positions d ₃ , d ₇ , and d ₁₀ becomes maximum, and as mentioned above, the camera automatically focuses on the subject in the foreground. When focusing, information on this position is obtained from the focus control circuit 30 according to the maximum value marker added to the information on the position of _d3 , and this is converted into an analog signal by the D/A converter 37. , this analog signal causes the imaging lens 11 in this example to be moved so that the optical system is at the focal position of _d3 . this d ₃
is not the focus position for the desired subject, the focus select switch 38
When operated, the memory position is moved to the next maximum value marker, information on that position is obtained from the focus control circuit 30, and the lens 11 is moved to a position that follows it.

The focus select switch 38 is placed near the lens of the camera body 40, for example, as shown in FIG.
When the switch 38 is turned down, the focus position sequentially moves to the subject position farther from the camera, and when the switch 38 is turned to the F side in the figure, the focus position is sequentially moved closer to the camera. .

Of course, instead of moving the lens system using the output of the D/A converter 37, the main image sensor 14 may be moved.

Note that when looking at the positional relationship of the picture elements in FIG. 12, the distances between the picture element δ and the picture elements β and γ are the same, but the distance of the diagonal picture element α is different from the others.
Therefore, in order to obtain the evaluation value more accurately, it is necessary to add a certain weight to the calculation for the diagonal picture element α according to the distance.

In order to avoid this, the picture elements of the image sensor may be arranged in a hexagonal shape as shown in FIG. In other words, in this case, the Laplacian of picture element d is found by using the surrounding picture elements a, b, c, e, f, and g, and the distances to picture element d are all almost the same, and the weight of each picture element is There is no need to think about the attachment. In this case as well, the actual calculation is to find |d-(a+b+c)/3| for each picture element, add this for all picture elements, and use the sum as the evaluation value.

According to the example described above, instead of moving the lens system with distance measurement as in the past, the image sensor with a small inertial mass is moved until the in-focus point is found.
Since the other lens systems are fixed, the distance measurement speed is increased and unnatural movements are avoided. Further, in the case of the present invention, since the in-focus point is detected at the same time for all focal lengths, there is an effect that the response is quick even if the subject position suddenly changes from infinity to close-up.

It also moves the sub-imaging device for focusing.
Since the lens system is not used for distance measurement, any interchangeable lens can be used as the lens system.

Effects of the Invention According to the present invention described above, the total sum of the Laplacian of picture elements is calculated by the calculation means, and the focus is changed by the focus control means to obtain focus position information corresponding to the maximum value of the output value from the calculation means. By changing the focus and selecting a desired focus position by the focus selection means at a location where a plurality of focus position information is obtained corresponding to the maximum value of the output value from the calculation means, By using a manual switch as the focus selection means, it is possible to perform accurate contrast detection and widen the dynamic range of the output. can.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an optical system of an example of a conventional autofocus device, FIGS. 2 and 3 are diagrams for explaining the same, and FIG. 4 is a diagram showing an example of an optical system of the present invention. Figure 5 is a diagram showing an example of the configuration of the main part.
FIG. 6 is a diagram for explaining the same, FIGS. 7 to 13 are diagrams for explaining the main parts of this invention, and FIGS. 14 and 15 are partial configurations of one embodiment of this invention. 16 is a block diagram of an example of the control system of the present invention, FIG. 17 is a diagram illustrating a part of the configuration, and FIG. 18 is a diagram of the sub-imaging device used in the present invention. FIG. 7 is a diagram showing another example of a picture element arrangement. 15 is a sub-imaging device; 33 is a logarithmic compression circuit; 35
is an arithmetic circuit, 351 is a shift register, 352 is a Laplacian arithmetic circuit, 353 is an adder, 354
is a register.

Claims (1)

[Scope of Claims] 1. A surface image sensor, a logarithmic compression conversion means for logarithmically compressing an image output from the surface image sensor, and an image based on the image output logarithmically compressed by the logarithm compression conversion means. In order to obtain the sharpness evaluation value of a certain picture element d of the above-mentioned surface image sensor, the horizontally adjacent picture element c, the vertically adjacent picture element b, and the diagonally adjacent picture element a of this picture element d are The above picture element d
The Laplacian in |3d−(a+b+c)|
calculation means for calculating the sum of the Laplacian of these picture elements, and a calculation means for calculating the sum of the Laplacian of these picture elements, and a calculation means for calculating the sum of the Laplacian of these picture elements, and a calculation means for changing the focus to reach the maximum value of the output value from the calculation means. a focus control means for obtaining corresponding focus position information, and selecting a desired focus position when a plurality of focus position information is obtained corresponding to the maximum value of the output value from the calculation means by changing the focus; An image processing device characterized by comprising: focus selection means for 2. The image processing apparatus according to claim 1, wherein the focus selection means is a manual switch.