JP2001141981A - Range finder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a CPU operating at high speed or a correlation operation device must be mounted to perform complicated processing in order to perform picture processing at high speed when distance measurement using a conventional artificial retina LSI having a parallax is applied for a camera, so that costs of individual parts increase and a range finder itself is possibly large-sized because of the increase of the number of parts, and it cannot be mounted on a small-sized and inexpensive device like a compact camera though being able to be mounted on a specific single-lens reflex camera or the like. SOLUTION: The range finder is provided with an AF area sensor, where an imaging device arranged so as to receive two images having as parallax ad a light reception signal processing circuit for pressing of a light reception signal of this imaging device are formed on the same semiconductor substrate, and detects a principal subject in a photographic picture on the basis of sensor data (outline data) integrated in the AF area sensor in the outline detection mode and sets a distance measurement area including this principal subject to perform operation for distance measurement.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring device usable for a camera or the like.

[0002]

2. Description of the Related Art In general, a camera is equipped with a distance measuring device for measuring (ranging) a distance to a subject at the time of photographing in order to focus a photographing lens on the subject.

As the distance measuring apparatus, there are an active method in which infrared light is projected on a subject and distance measurement is performed by reflected light, and a passive method in which external light from the object is integrated to measure distance. Are known. In these systems, a line sensor or an area sensor is generally used as a photoelectric conversion element that receives external light and converts it into an electric signal.

As an image processing apparatus using the area sensor, an area sensor having two light receiving regions arranged so as to have parallax, an image processing circuit for performing various information processing on image signals from the area sensor, and But the same IC
2. Description of the Related Art An artificial retinal LSI device mounted on a chip is known. This image processing circuit performs processes such as edge enhancement (detection), smoothing, and one-dimensional projection.

[0005] For example, Japanese Patent Application Laid-Open No. 8-178637 discloses an artificial retinal LSI device.
An image processing apparatus employing I has been disclosed.

In this image processing apparatus, the contour of an image is extracted from an input image by an artificial retinal element, and the correlation between the contour information of the scanning lines of the same height in the two images is calculated by a correlation operation apparatus using a synapse connection operation circuit. The calculation is performed at high speed, and information such as distance measurement to the object and detection of the movement of the object is obtained using the result of the correlation.

[0007]

However, the image processing apparatus described in Japanese Patent Application Laid-Open No. Hei 8-178637 requires a high-speed image processing when a distance measurement using an artificial retinal LSI is applied to a camera. In this method, a high-speed CPU and a correlation operation device for performing complicated processing are mounted, and the cost of each component is high, and the number of components is large. Therefore, the distance measuring device itself may be large.

Further, two images are captured using two cameras equipped with an artificial retinal LSI, and the distance to an object is measured, that is, the distance is measured based on the correlation between the two images based on the stereo method. Therefore, the device itself becomes very large and the cost is high.

Therefore, even if it can be mounted on a specific single-lens reflex camera or the like, it cannot be mounted on a small and inexpensive device such as a compact camera.

Accordingly, the present invention provides an AF in which an image sensor for receiving two input images having parallax and a light receiving signal processing circuit for generating contour data of a subject from a light receiving signal of the image sensor are formed on the same semiconductor substrate. With an area sensor,
It is an object of the present invention to provide a distance measuring device having a wide range of distance measurement area, capable of detecting a main subject, reducing a time lag, reducing size, and reducing cost.

[0011]

In order to achieve the above object, the present invention provides two optical systems having parallax, an image sensor for capturing two images formed by the optical system, and an image sensor. Provided is a distance measuring device that is formed on the same semiconductor substrate and includes a processing unit that performs image processing on an output of the image sensor and a distance measuring unit that performs distance measurement based on an output of the processing unit.

[0012] Further, two optical systems having parallax, an image pickup device for picking up two images formed by the optical system, and an image formed on the same semiconductor substrate as the image pickup device, and an image output to the image pickup device. Processing means for performing processing; main subject detecting means for detecting a main subject based on the output of the processing means; and distance measuring means for measuring the distance based on the output of the processing means and the output of the main subject detecting means. And a distance measuring device comprising:

In the distance measuring apparatus having the above configuration, two images formed by two optical systems having parallax are picked up by the image pickup device, and the light receiving signal of the image pickup device is processed by the processing means on the same semiconductor substrate. Then, the data is output to the main subject detecting means as sensor data (contour data). The main subject detecting means detects a main subject in a shooting screen based on the sensor data, sets a ranging area including the main subject, performs a ranging calculation, and controls a position of a focus lens of a mounted camera. Is performed.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows a schematic configuration example of a camera equipped with a distance measuring device as a first embodiment of the distance measuring device of the present invention.

This camera has a control unit 1 comprising a microcomputer for controlling the components described below and performing arithmetic processing, and an AF area sensor 2 used for distance measurement.
A focus lens driving unit 3 for driving the focus lens 4; a focus lens encoder 5 for generating a pulse signal corresponding to the amount of movement of the focus lens 4; and a photometric signal by processing a photocurrent signal generated by the photometric light receiving element 6. A photometric unit 7 for outputting a result, a shutter driving unit 8 for driving a shutter (not shown), a strobe circuit unit 10 for emitting a strobe light unit 9 as auxiliary light for photographing or AF auxiliary light for distance measuring operation, A viewfinder display section 11 for superimposing and displaying information relating to the present invention on a viewfinder screen in addition to a shooting screen, and a camera display section including an LCD and the like provided on the exterior of the camera for displaying the number of film frames and a shooting mode. And a display circuit unit 13 for controlling display on the finder display unit 11 and the camera display unit 12.
Then, the zoom lens 14 is moved to perform a zoom operation (changing the focal length), and the focal length information is output to the control unit 1, and the automatic loading of film loading, one frame winding, winding are performed. It is composed of a film feeding section 16 for feeding the film such as returning the film, and an attitude detecting section 21 for detecting the attitude (vertical and horizontal) of the camera.

The controller 1 is further connected to a first release switch (1RSW) 17 and a second release switch (2RSW) 18,
When W17 is turned on, AF and photometry operations are performed, and when 2RSW18 is turned on, exposure operation and film winding operation are performed.

The control unit 1 includes a central processing unit (CPU)
1a, a ROM 1b for storing a series of sequence programs related to photographing, a RAM 1c for rewritably storing information as needed, and an A / D converter 1 for converting analog signals such as photometric output from the photometric unit 7 into digital signals.
d, and an EEPROM 1e for storing correction data for each camera relating to autofocus (AF), photometry and exposure calculation, and various parameters for detecting a main subject in a shooting screen described later.

The AF area sensor 2 includes a light receiving element group 2a in which pixel units including a plurality of photodiodes and the like are two-dimensionally arranged in a horizontal direction and a vertical direction of an imaging area.
, A light receiving signal processing circuit 2b, and a stationary light removing unit 2c. In this configuration, under the control of the integration operation by the control unit 1, a subject image formed by a distance measuring optical system, which will be described later, is imaged, and charges generated by light incident on the light receiving element group 2a are converted into pixel amplification circuits for each pixel. The constant light removing unit 2c removes the steady light component from the sensor data and outputs it to the control unit 1.

The control unit 1 outputs a drive signal to the focus lens drive unit 3 based on the distance measurement calculation result calculated based on the input sensor data, monitors the output of the focus encoder 5 and monitors the output of the focus lens 4. A distance measurement operation for performing position control is performed. The main routine in shooting will be described with reference to the flowchart shown in FIG. When a power switch (not shown) is turned on or a battery is inserted, the control unit 1 starts up and starts operating according to a sequence program stored in the ROM 1b in advance. First,
After initializing each block in the camera,
The adjustment and correction data relating to auto focus (AF), photometry, and the like stored in e are developed in the RAM 1c (step S1).

Next, it is determined whether or not the 1RSW 17 has been turned on (step S2). If the 1RSW 17 remains off (NO) in this determination, another switch (1RSW 17)
Then, it is determined whether or not any of the buttons (other than 17, 2RSW 18) has been operated (step S3). If another switch is operated (YES), processing according to the switch input, for example,
When the ZUSW 19 or the ZDSW 20 is operated, the zoom lens 14 is moved up and down (step S4), and the process returns to step S2. If no other switch has been operated (NO), the process returns to step S2 and waits.

In the determination in step S2, 1
If the RSW 17 is turned on (YES), distance measurement is performed (step S5), and photometry / exposure calculation is performed (step S6). Thereafter, it is determined whether or not the 2RSW 18 has been turned on (step S7). In this determination, 2
When the RSW 18 is turned on (YES), the film is exposed by the shutter operation (step S8). After the end of the exposure, the film is wound up by one frame (step S).
9) Return to step S2 and wait for the next photographing.
However, if the 2RSW 18 is not turned on in step S7 (NO), the process returns to step S2.

FIGS. 3A and 3B show a conceptual configuration of a distance measuring optical system based on an external light passive system and will be described. In this configuration, the light receiving element group 2 of the AF area sensor 2
The distance measuring light is guided to the light receiving area where
The light receiving lenses 31 and 32 for measuring the distance to are arranged with a base line length B therebetween.

The light receiving lenses 31 and 32 divide the image of the subject 33 into two images and form an image on the light receiving element group 2a of the AF area sensor 2. Relative positional difference x between the two images
Is the focal length f of the light receiving lens according to the principle of triangulation.
And the base length B, the subject distance L is given by the following equation. L = (B · f) / x The distance calculation is performed by the control unit 1. More specifically, a distance measurement block is set in the light receiving area 2a of the AF area sensor 2, and a correlation operation is performed using sensor data corresponding to the two images to detect a relative positional difference x between the two images. I do.

FIG. 4 shows a specific configuration example of the AF area sensor 2 described above.

The AF area sensor 2 has a pixel area (within the light receiving element group 2a) 3 for receiving the distance measuring light imaged by the light receiving lenses 31, 32 as shown in FIG.
4, 35, a vertical control circuit (VC) 36, 37 functioning as a control scanner, a horizontal control circuit (HC) 38 functioning as a multiplexer, an output circuit OC39, and an AF based on a command from the control unit 1. A control circuit (SC) 40 for controlling the operation of the area sensor 2 is formed on the same silicon substrate by a CMOS process, and light receiving lenses 31 and 32 (not shown) are integrally mounted. .

Each light receiving element in each of the pixel regions 34 and 35 includes a photodiode PD as a light receiving portion and a differential amplifier S
A, and a reset transistor RT. Sensitivity control input terminals DP, DN, and a reset terminal RS are connected to vertical control circuits 36, 37. An output terminal Nout commonly connected to each column is connected to the horizontal control circuit 38. Here, the sensitivity control input terminals DP and DN and the reset terminal RS
Are connected in common in each row.

The reset terminal RS is for resetting the PD potential to a constant potential. After being reset to a constant potential by the reset pulse Vr, a signal according to light intensity is accumulated during a charge accumulation period until a readout pulse is input, and the potential decreases according to the amount of incident light.

FIG. 5 shows a specific configuration example of the differential amplifier circuit SA.

This differential amplifier circuit SA is configured by using current mirror connection by transistors TA, TP, TN, T0 and TI composed of NMOS. The photodiode PD and the reset transistor RT are connected in series, and the cathode of the photodiode PD is connected to the gate of the current control transistor TA.

The potential of the photodiode PD of each pixel is
The data is read by the operation of the gate of the current control transistor TA of the differential amplifier SA. Sensitivity control terminals DP, DN
, A positive and negative current output proportional to the potential of the photodiode PD is obtained from Nout.

If no voltage is applied to the sensitivity control terminals DP and DN, the current output of the pixel becomes zero. this is,
The sensitivity control terminal DP that controls the positive output current
When "1" is given to the sensitivity control terminal DN to turn on the transistor TP and turn off the transistor T0, a current corresponding to the potential of the sensitivity control terminal PD is output from the output Nout through the transistors TA and TP. You.

On the other hand, "1" is given to the sensitivity control terminal DN for controlling the negative output current and "0" is given to the sensitivity control terminal DP to turn on the transistor TN, and the transistor TP
Is turned off, a current corresponding to the potential of the sensitivity control terminal PD flows through the transistors TA and TI, and after being turned back by the current mirror, a current corresponding to the potential of the photodiode PD is drawn from the output Nout by the transistor T0. .

Here, the output terminal N from each pixel in the same row
out are connected to each other, the current output Nout from each pixel is added. Accordingly, as shown in FIG. 6, which will be described later, in a state where the pulse VP in the third and fourth columns is applied and the pulse VN in the second and fifth columns, the four pixels are added to the image information incident thereon by a weighting factor. A result similar to that obtained by performing filtering with (−1, 1, −1, 1) is obtained.

The current output Nout is supplied to the horizontal control circuit 38.
Is selected by scanning, is converted into a voltage output from the output circuit 39, is output, and is input to the AD converter 1d in the control unit 1. Assuming that the sensitivity control signal vector is S and the two-dimensional matrix W of the input image information, a correlation signal of WS is obtained at the same time as the output.

Since the sensitivity control data is transferred by the vertical control circuits 36 and 37, the sensitivity control signal S is sequentially shifted one bit at a time.

As a result, various pulse voltages as VP and VN are applied to the vertical control circuits 36 and 37, and scanning is performed.
Image processing such as compressing two-dimensional information into one dimension can be performed.

For example, the sensitivity control signal vector S is expressed as S
(1, 0, 0,..., 0), that is, S (0, 1, 0,..., 0), S (0, 0,
1,..., 0), a normal image (normal image) is obtained. If S (1, -1, 0,..., 0) is shifted, the difference between adjacent pixels in the horizontal direction is obtained, and horizontal contour extraction can be performed.

Alternatively, if S (1, 1, 0,..., 0) is shifted, an output obtained by adding data of two pixels is obtained, and the resolution can be changed. S (1,1,
1, 0,..., 0), three pixels are added, and the resolution can be varied.

In the case of S (1, 1, 1,..., 1), the output signals in the horizontal direction are added to obtain one-dimensional projection data. The control unit 40 sets the sensitivity control signal in accordance with a command from the control unit 1.

Hereinafter, a specific outline detection operation will be described. 6A and 6B show a conceptual relationship between the AF area sensor 2 and an input image. The light incident on the AF area sensor 2 irradiates the upper right area as shown in FIG. 6 and does not irradiate the lower left shadow area.

In this configuration, 1 is used as the sensitivity control signal.
When a voltage “1” is input to the row and a voltage “−1” is input to the second row, S (1, −1, 0,..., 0) is applied to the sensitivity variable element in the first and second rows. In the same light portion, outputs are added in the column direction, and the output becomes "0" (first, third to fifth rows).

Further, an output "1" which is not "0" appears only in a portion where the light intensity is changed between the first row and the second row (output of the second row). That is, contour detection is performed.

Next, the applied voltage (sensitivity control signal) is shifted downward in FIG. 6A, and as shown in FIG. 6B, the voltage “1” is set in the second row and the voltage “1” is set in the third row. Add -1 ". In this case, outputs are added in the column direction at the same portion of the light irradiated to the sensitivity variable element in the second and third rows, and the output becomes "0" (first, second, fourth, and fifth rows). In addition, only the portion where the light intensity is changed between the second and third rows is “output that is not“ 0 ””.
1 "appears (output of the third row).

As described above, the scanning is repeated and the control signal voltages are sequentially shifted and applied, and when the output signals at that time are arranged, a matrix as shown in FIG. 6C is formed.

Thus, the outline of one scanning line is simultaneously
In addition, detection can be performed in parallel, and a matrix relating to contour information can be obtained.

FIG. 7 is a diagram showing the relationship between the size of the screen at the wide-angle end and the telephoto end in the shooting screen and the distance measurement area.

In this embodiment, since the external light distance measuring method is employed, there is a parallax between the photographing screen by zooming and the distance measuring area.

Therefore, the area used for distance measurement is limited according to the focal length information (zoom information) of the photographing optical system. The distance measurement area position correction data corresponding to such a change in the focal length is stored in the EEPROM 1e in advance, and is read out and expanded in the RAM 1d when the control unit 1 is initialized.

According to the zoom operation by the zoom lens driving unit, the AF area sensor 2 is referred to by referring to the correction data.
The distance measuring area used for the distance measuring operation within the light receiving area is determined. Then, a distance measurement calculation is performed using the sensor data within the range of the distance measurement area.

Further, the control section 1 outputs a control signal to the AF area sensor 2 so as to generate a peak monitor for integration control corresponding to the AF area sensor 2. And AF
The area sensor 2 outputs a peak signal within the range of the designated ranging area to the control unit 1. The control unit 1 refers to this monitor signal so that the integration amount becomes a predetermined level.
Control.

In this way, the influence of the subject outside the photographing screen is prevented. Also at the time of reading the sensor data, unnecessary sensor data outside the shooting screen is skipped and stored in the RAM 1c with reference to the distance measurement area correction data corresponding to the shooting screen. Alternatively, a read range setting signal is output to the AF area sensor 2 so that only the sensor data within the set range is output.

With reference to the flowchart shown in FIG. 8, the AF routine in the distance measuring apparatus according to the present embodiment will be described. First, operating the operation unit provided in the camera, A
The contour detection mode in the F area sensor 2 is set (step S11).

Then, the controller 1 sends the AF area sensor 2
To output an integration control signal to perform an integration operation (step S12). A monitor signal corresponding to a peak (brightest pixel) output within a predetermined range is output from the AF area sensor 2. While referring to this monitor signal, the integration time is adjusted so that the amount of light received by the light receiving section 2a of the AF area sensor 2 becomes appropriate. Then, predetermined integrated sensor data (contour data) is read from the AF area sensor 2 in accordance with a read clock signal from the control unit 1, and
After the AD conversion by the AD converter 1d, the data is stored in the RAM 1c.

Next, the central processing unit 1a performs a process of extracting a main subject (step S13). It is determined whether or not the extracted object is a main subject (step S14), a ranging area including the main subject is set, and a ranging calculation is performed (step S15).

The focus lens 4 is driven by the focus lens drive unit 3 based on the distance measurement data obtained by the distance measurement calculation (step S16).

The main subject detection routine will be described with reference to the flowchart shown in FIG. In this routine, for example, a procedure for assuming a person as a main subject and detecting the person will be described. Note that two images are obtained by the AF area sensor 2, and the image data (sensor data) used for detecting the main subject may be either one of the images or both images.

The sensor data of the AF area sensor 2 is stored in the RAM 1c in the control section 1, and the following processing is performed based on the sensor data.

First, a smoothing process is performed on the read sensor data (step S21). This smoothing process is a process for removing random noise in an image, and is performed by filtering or Fourier transform (step S21). This random noise is generated by random noise of the AF area sensor 2 itself or external noise such as power supply voltage fluctuation of the AF area sensor 2.

Next, a part below a certain value is extracted from the binarized image by threshold processing to obtain a binary image (step S22).

Here, the smoothing process is a process for removing random noise mixed in the image. There are various methods for this processing. For example, a median filter for finding a median (median) of pixel values in a neighboring region,
An edge preserving filter or the like that divides the neighboring area into small areas, obtains the variance for each small area, finds the small area having the smallest variance, and outputs the average value thereof is effective. The median filter has a side effect of dulling the edge of the image, but the edge preserving filter is more effective because the edge is not dull. In addition, there is a method using Fourier transform.

Thereafter, the central processing unit 1 a
To perform labeling and figure fusion processing (step S2).
3) A graphic having a certain width corresponding to the thinning processing edge is obtained (step S24). The line width is reduced to about 1 by applying a thinning algorithm to this figure.

Then, the shape of the image is determined by a shape determination process described later, and a main subject is extracted (step S).
25).

Next, the binarization processing (threshold processing) will be described with reference to the flowchart shown in FIG. First,
A histogram representing the frequency of appearance of pixel values representing each luminance in the image is created (step S31).

Next, a threshold value is set (step S3).
2). This threshold is set based on a threshold setting processing histogram. There are various methods, for example, in the case where the mode method is used, FIG.
In the example shown in (a), the luminance value with the lowest frequency is set as a threshold (threshold level), and the binarization process is performed (step S33).

Other methods for setting the threshold include a p-tile method effective when the area of the figure to be extracted is known to some extent, a differential histogram method in which a threshold is set at the boundary of the figure, and a set of density values. The parameter t is set so that the separation between the classes when divided into two classes is the best.
, And a variable threshold method that changes the threshold value depending on the image position. These methods are appropriately selected and used depending on the situation.

For example, by determining the shape of the histogram,
It is determined whether there is a clear minimum value, and if so, the modal method is adopted. On the other hand, when it is not clear, a discriminant analysis method is adopted. Thus, the shape of the histogram is determined, and the threshold setting method is changed according to the result.

In the histogram shape discrimination method, as shown in FIG. 11B, for example, an extreme value (valley), a minimum frequency value a, and a second smallest value b are obtained, and the difference b−a Is compared with the determination value dth, and when it is larger than the predetermined value dth, the luminance value of the minimum value a is adopted as the threshold value. On the other hand, if the value is equal to or less than the predetermined value, a variable threshold method of changing the threshold according to the image position is employed. The threshold value setting processing will be described with reference to the flowchart shown in FIG.

First, the minimum value a and the second lowest frequency b are obtained (step S41). Next, it is determined whether the difference (ba) is larger than a predetermined determination value dth (step S42).

In this judgment, if the difference (ba) is larger than the judgment value dth (YES), the luminance value Ba corresponding to the minimum value a is adopted as the threshold (step S4).
3). However, when the difference (ba) is equal to or smaller than the determination value dth (NO), the variable threshold method is adopted (step S4).
4).

According to the present embodiment, in the case of binarization of an image corresponding to the entire photographing screen, first, a threshold value is set by a mode method and binarization processing is performed. If the result of evaluating the binarized image is not good, the image may be divided into a plurality of blocks, a histogram may be created without any divided blocks, and a threshold may be set again for each divided block.

Next, labeling and figure fusion processing will be described. Labeling is performed on a group of connected portions where pixels having the same luminance value are connected to each other in the image. That is, as shown in FIG. 14B, different labels 1 to 3 are attached to different connection portions to separate and separate regions (connection regions).

Further, in the graphic fusion processing, a graphic having a small area such as a hole or a point-like graphic included in an image is not only essentially ineffective, but may have a bad influence on subsequent processing as noise. Need to be removed.
This removes noise components by expanding or reducing the original figure.

Next, thinning will be described.

The thinning is a process of narrowing the obtained binary image to a line figure having a line width of 1 without deteriorating the connectivity to each of the connected regions included therein. In a linear graphic having an arbitrary thickness, the center line of the linear graphic is obtained by sequentially removing pixels in the width direction.

The area of the shape determination connection region is the number of pixels belonging to the connection region, and the perimeter is the number of pixels located at the boundary around the connection region. However, the angle in the diagonal direction is corrected to √2 times the horizontal and vertical directions. The following coefficient e is used to determine the shape of the image.

E = (perimeter) ＾ 2 / (area) e indicates a minimum value when the shape is circular, and indicates a larger value as the shape becomes more complicated. Since the person's face is considered to be almost circular, it is determined whether or not the symmetric image is a person's face by comparing the above e with a predetermined value. The area of the connected area is also compared with a predetermined value to determine whether or not the face is a symmetric image person.

Prior to the shape determination, if the area is outside the predetermined range by comparing the area with the value of the predetermined range, it may be determined that the image is not a person and the shape determination process is not performed. In this way, the amount of calculation can be reduced, and the AF time lag can be reduced.

Referring to the flowchart shown in FIG.
The shape determination processing will be described. First, it is determined whether or not there is an extraction area. If there is no extraction area (NO), the process returns (step S51). However, if there is an extraction area (YES), the area S of the extraction area is determined, and it is determined whether or not the area is within a predetermined range (step S52).

If the extraction area S is out of the predetermined range (NO), it is determined that the subject is a person other than a person (step S53), and the process returns. However, if it is within the predetermined range (YES), the shape determination value e is calculated, and it is determined whether or not the shape determination value e is within the predetermined range (step S54).
If the shape determination value e is within the predetermined range in this determination (YE
S), it is determined that the person is a person (step S55). However, if the shape determination value e is out of the predetermined range (NO), the process proceeds to step S53, and it is determined that the subject is other than an object.

Next, it is determined whether or not the shape has been determined for all the extraction regions (step S56). If the determination has been completed (YES), the process returns. On the other hand, if not completed (NO), the next extraction area is set (step S5).
7) Return to step S52 and execute repeatedly.

FIG. 14A shows a photographing scene photographing screen in a person determination, FIG. 14B shows contour data after binarization processing, and FIG. 14C shows the photographing scene photographing screen. FIG. 4D shows low-resolution contour data on a photographing scene photographing screen.

This shooting scene shooting screen has a corresponding AF
This is an image area of the area sensor 2 and includes a person as a subject. The outline data output from the AF area sensor 2 is subjected to binarization processing. By this processing, only the edge portion (contour) 53 as shown in FIG. 14B becomes an extracted image. These extraction areas include:
Labels 1 to 3 are respectively attached by labeling processing. Then, a label 3 area determined to be a person's face is extracted, and a distance measurement area 51 including the person determination area label 3 is set. Then, the distance measurement calculation is performed based on the contour data (before the binarization processing) in the distance measurement area 51.

As described above, since the contour data is processed by the circuit provided in the AF area sensor 2, the contour data can be obtained at a very high speed as compared with other arithmetic processing such as difference processing. A distance measuring device with a small time lag can be realized.

Next, a second embodiment according to the distance measuring apparatus of the present invention will be described with reference to the flowchart shown in FIG. This embodiment has a configuration equivalent to that of the above-described first embodiment, and a description of the configuration will be omitted, and a different distance measurement routine will be described.

First, the operation unit provided in the camera is operated to set the y projection detection mode in the AF area sensor 2 (step S61). Then, the controller 1 sends the AF
An integration control signal is output to the area sensor 2 to perform an integration operation (step S62). A monitor signal corresponding to a peak (brightest pixel) output within a predetermined range is output from the AF area sensor 2. While referring to the monitor signal, the integration time is adjusted so that the light receiving amount of the AF area sensor 2 light receiving unit becomes appropriate.

Then, as shown in FIG. 16, predetermined integrated sensor data (y projection data) is read from the AF area sensor 2 in accordance with a read clock signal from the control unit 1, and is subjected to an AD converter. After the AD conversion in 1d, it is stored in the RAM 1c (step S63).

Similarly, the x projection detection mode is set for the AF area sensor 2 (step S64), and the control unit 1 outputs an integration control signal to the AF area sensor 2 to perform the integration operation (step S64). Step S65). Then, as shown in FIG. 16, predetermined integrated sensor data (x
The projection data) is read from the AF area sensor 2 in accordance with the read clock signal from the
After the AD conversion by the converter 1d, it is stored in the RAM 1c (step S66).

Next, a process of extracting a main subject is performed (step S67). For the AF area sensor 2,
The normal mode is set (step S68). Integral control is performed based on the detected monitor signal of the main subject position (step S69). Then, normal sensor data (normal mode data) is read from the AF area sensor 2 (step S70). At this time, it is not necessary to read out the data of all the pixels, but only the data around the main subject position.

Next, a distance measurement area including the extracted main subject position is set, and a distance measurement operation is performed within the distance measurement area (step S71). Based on the obtained distance measurement data, a focus sync lens drive is performed. (Step S7)
2).

FIG. 16 is a diagram showing the relationship between the shooting screen and the projection output by the AF routine shown in FIG. 15, and the main subject detection by the projection output will be described. As described above, the x projection output is the normal pixel output (normal sensor data)
Are added in the x direction, and the y projection output is data added in the y direction. Note that the long side direction of the screen is X and the short side direction is y.

As shown in FIG. 16, when the background 52 is dark and the main subject 56 is bright, the projected output of the area where the main subject 56 exists is large. Step S6 described above
In step 7, an area including a range exceeding predetermined determination values Sx and Sy in the x and y directions as a main subject area is set as the distance measurement area 54.

Then, in step S69,
The control unit 1 outputs a command to the AF area sensor 2 so as to set a monitor range in an area corresponding to a pixel in the distance measurement area 54. By performing an integration operation based on the monitor signal in this area, the distance measurement area 5 which is a person determination area is obtained.
Thus, it is possible to obtain the optimum normal sensor data for 4 and to perform a highly accurate distance measurement calculation. In the case of a backlight scene, as shown in the integral control area 55, by setting the area smaller than the main subject range and performing the integral control, the sensor data in the person determination area is affected by the high luminance background and is blurred. Can be prevented.

Next, a third embodiment according to the distance measuring apparatus of the present invention will be described with reference to the flowchart shown in FIG. This embodiment has a configuration equivalent to that of the above-described first embodiment, and is a modified example of the AF routine shown in FIG. 8, in which the contour detection is also performed on a background scene other than the main subject. is there.

First, the operation section provided in the camera is operated to set the contour detection mode in the AF area sensor 2 (step S81). Then, the control unit 1 outputs an integration control signal to the AF area sensor 2 to perform the pre-emission / stationary light removal integration operation (step S82). This is because when the AF routine described with reference to FIG. 8 is applied to a shooting scene as shown in FIG.
Contour detection may be performed on background scenes other than 6, which may result in extra processing and a large time lag. In order to solve this, the stationary light removal integration of the AF area sensor 2 is performed while performing the multiple pre-emissions by the strobe 9. Since the amount of reflected light from the subject due to the pre-emission is greater for a short-distance subject, if integration control is performed at the peak of the reflected light quantity, the output of a long-distance subject with a small reflected light quantity is removed, as shown in FIG. Contour data is obtained.

Thereafter, the sensor data (contour data) obtained by predetermined integration is read from the AF area sensor 2 in accordance with the read clock signal from the control section 1, and is subjected to A / D conversion by the A / D converter 1d and then to the RAM 1c. Is stored in

Next, in the central processing unit 1a, a process of extracting a main subject is performed (step S83). The main subject extracted here is determined (step S84).
Distance measurement calculation is performed for an area based on the main subject position in the contour data (step S85). In this distance measurement operation, the influence of the background can be removed, and more accurate detection of the main subject with a small time lag and distance measurement can be performed.

Then, the focus lens 4 is driven by the focus lens driving section 3 based on the obtained distance measurement data (step S86).

According to the above embodiment, even in a photographing scene in which an object 57 which can be a subject exists around the main subject 56 as shown in FIG. Processing can be prevented, and a time lag can be prevented from increasing. FIG.
(B) shows the pre-flash in the shooting scene shown in FIG.
3 shows contour data obtained by stationary light removal integration.

Next, a fourth embodiment according to the distance measuring apparatus of the present invention will be described with reference to the flowchart shown in FIG. This embodiment has a configuration equivalent to that of the above-described second embodiment. In the AF routine shown in FIG.
It is a modified example of the AF routine that prevents projection detection even for a background scene with high luminance.

First, the operation unit provided in the camera is operated to set the y projection detection mode in the AF area sensor 2 (step S91). Then, the controller 1 sends the AF
An integral control signal is output to the area sensor 2 to perform pre-flash /
A stationary light removal integration operation is performed (step S92).

This is because, when the AF routine shown in FIG. 15 is applied to a photographic scene as shown in FIG. 20, projection detection is performed also on a background scene having a high luminance other than the main subject 56, and the main subject 56 is detected. Detection is hindered,
There is a problem that extra processing occurs and a time lag increases.

The following processing is performed to solve this.

That is, while pre-emission is performed by the strobe light 9, first, stationary light removal integration of the AF area sensor 2 is performed. Since the amount of reflected light from the subject due to the pre-emission becomes larger for a short-distance subject, if integration control is performed at the peak of the reflected light amount, the output of a long-distance subject with a small reflected light amount is removed. Therefore, the Sy projection data (Sx
Projection data), the influence of the background can be removed, and more accurate distance measurement with a small time lag can be performed.

The predetermined integrated sensor data (y projection data) is read from the AF area sensor 2 in accordance with the read clock signal from the control unit 1.
After being AD-converted by the AD converter 1d, it is stored in the RAM 1c (step S93).

Next, a process of detecting a main subject is performed (step S94), and it is determined whether a main subject has been detected (step S95). In this determination, if it is detected (YES), the process in the x projection mode is omitted, and the normal mode is set for the AF area sensor 2 (step S96). Since the background scene is removed only by the y projection data, if the main subject is detected only by the y projection data, the processing in the next x projection mode can be omitted, and the time lag such as the integration time can be reduced.

Then, in the normal mode, integration control is performed based on the monitor signal of the detected main subject position (step S97). Then, normal sensor data (normal mode data) is read from the AF area sensor 2 (step S98). At this time, it is not necessary to read out the data of all the pixels, but only the data around the main subject position.

Next, a distance measurement area 54 including the extracted main subject position is set, and a distance measurement operation is performed within the distance measurement area (step S99). Based on the obtained distance measurement data, a focus sync lens is set. Drive (Step S1)
00).

On the other hand, if the main subject is not detected in step S95 (NO), the x projection detection mode in the AF area sensor 2 is set by the same operation (step S101). And pre-flash
A stationary light removal integration operation is performed (step S102).

The predetermined integrated sensor data (x projection data) is read out from the AF area sensor 2 and subjected to AD conversion by the AD converter 1d.
c (step S103).

Next, processing for detecting the main subject is performed (step S104), and it is determined whether or not the main subject has been detected (step S105). In this determination,
If it is detected (YES), it indicates that the main subject can be detected by using both the y projection data and the x projection data, and the flow shifts to step S96. However, if the main subject cannot be detected (NO), first, the AF area sensor 2 is set to the normal mode (step S).
106).

Then, integral control is performed for each block (step S107). This integral is, as shown in FIG.
The pixel area of the AF area sensor 2 corresponding to the shooting screen (or the light receiving unit of the AF sensor corresponding to the shooting screen) is divided into a plurality of blocks (E1, E2,...), And a monitor signal for each block is referred to. While performing integral control.

Next, sensor data is read from the AF area sensor 2 under the control of the control section 1 (step S).
108) Then, a distance measurement operation is sequentially performed for each block (step S109), and suitable distance measurement data is determined from the obtained distance measurement data (step S110). As the distance measurement data, the closest distance measurement data is selected and adopted from the distance measurement results of the detectable blocks. As described above, when the main subject cannot be detected by the x, y projection data, distance measurement is performed on the entire screen, and distance measurement data is created based on a predetermined algorithm (closest selection, average, etc.).

Then, the flow shifts to step S100, where the focusing lens is driven based on the determined distance measurement data.

According to the present embodiment described above, FIG.
The projection detection is performed only on the main subject even in a shooting scene in which there is a high-luminance object that can be a subject other than the main subject as shown in (1), unnecessary processing can be prevented, and the time lag can be prevented from increasing.

Referring to the flowchart shown in FIG.
A fifth embodiment according to the distance measuring apparatus of the present invention will be described.

First, a low resolution mode in the AF area sensor 2 is set by operating the operation unit provided in the camera (step S111). Then, integral control is performed according to a command from the control unit 1 (step S112).

The predetermined integrated sensor data (low-resolution data) is read from the AF area sensor 2 in accordance with the read clock signal from the control section 1, and is subjected to AD conversion by the AD converter 1d. , RAM1
c (step S113). The stored sensor data is, for example, low-resolution data as shown in FIG. 14C in the shooting scene shown in FIG.

Next, a main subject is detected based on the low resolution data (step S114).

Next, a high resolution mode (normal mode) is set for the AF area sensor 2 (step S1).
15). Integral control is performed based on the monitor signal of the detected main subject position (step S116). And AF
Sensor data (high-resolution data) from area sensor 2
Is read (step S117).

Next, a distance measurement area including the extracted main subject position is set, a distance measurement operation is performed within the distance measurement area (step S118), and a focus sync lens drive is performed based on the obtained distance measurement data. (Step S11)
9).

Next, the main subject detection in step S114 will be described with reference to the flowchart shown in FIG.

This main subject detection is the same routine as the main subject detection described in the flowchart of FIG. 9 described above, except that difference processing is performed on low-resolution sensor data.

In detecting the main subject, the sensor data of the AF area sensor 2 is stored in the RAM 1c in the control section 1, and the following processing is performed based on the sensor data.

First, a smoothing process is performed on the read sensor data (step S121). This smoothing process is a process for removing random noise in an image, and is performed by filtering or Fourier transform.

Next, a difference process is performed on the obtained low-resolution data (step S123). This difference processing
This is a process of performing edge detection, in which an edge candidate region and its strength are given.

In the edge detection processing, the edge detection is performed by performing the following processing on the sensor data s (i, j) as shown in FIG.

In the method using the first derivative operator, the derivative in the x direction and the derivative in the y direction are calculated by the following equations.

Δxs (i, j) = s (i, j) −s (i−1, j) Δys (i, j) = s (i, j) −s (i, j−1) , Data as shown in FIG.

In the method using the second derivative operator, the value is obtained by the following equation.

Δ ＾ 2xs (i, j) = s (i−1, j) −2s (i, j) −s (i + 1, j) Δ ＾ 2ys (i, j) = s (i−1 , j) -2s (i, j) -s (i, j + 1) The Laplacian operator, which is a kind of the second derivative operator, emphasizes the shoulder portion of the edge. Transition. Then, as shown in FIG. 25B, an edge is obtained by obtaining a portion that becomes "0".

As a specific processing method, a product-sum operation with a spatial filter table (weight table) is performed. FIG.
(A) to (d) show examples of the spatial filter table.

FIGS. 26 (a) and 26 (b) show a primary differential operator (horizontal and vertical directions), FIG. 26 (c) shows a Laplacian operator, and FIG. 26 (d) shows a Sobel operator ( x-direction, y-direction first derivative, absolute value conversion, addition).

The following shows the arithmetic expression.

[0135]

(Equation 1)

S (x, y): data before processing, S '(x, y):
Data after processing, W (x, y): spatial filter, n: constant.

Such a spatial filter is appropriately selected and used according to the situation. When the difference processing is performed on all the images, a first-order differential operator and a Laplacian operator, which are relatively simple and fast to operate, are used.

On the other hand, when performing the difference processing on a part of the images in the photographing screen, a sobel operator is selected and used, which has a relatively complicated operation and a long operation time but has a large effect.

When the integration time of the AF area sensor 2 is low at low brightness, the first derivative operator or the Laplacian operator is used. On the other hand, when the integration time is high at low brightness, the Sobel operator is used. A
The balance as the F time lag may be taken.

As described above, the contour detection is performed on the low resolution data by the difference processing.

Next, in the same manner as in step S23 of FIG. 9, the difference processing image is subjected to threshold processing to extract a portion equal to or less than a certain value to obtain a binary image (step S12).
3).

Thereafter, the central processing unit 1 a
To perform labeling and figure fusion processing (step S1).
24), a figure having a certain width corresponding to the thinning processing edge is obtained (step S125). The line width is reduced to about 1 by applying a thinning algorithm to this figure.

Then, the shape of the image is determined by a shape determination process described later, and a main subject is extracted (step S).
126).

Since low-resolution data can be obtained at a high speed in the low-resolution mode in this manner, processing at a higher speed than in the case of lowering the resolution by arithmetic processing such as addition can be performed.

Since the main subject is detected by reducing the number of data based on low-resolution data, high-speed main subject detection processing is possible. Further, since the distance measurement calculation is performed based on high-resolution data, highly accurate distance measurement can be performed.

Referring to the flowchart shown in FIG.
A sixth embodiment according to the distance measuring apparatus of the present invention will be described.

In this embodiment, the normal integration in FIG. 22 is changed to the pre-emission / steady light removal integration. Even in the low resolution mode, the strobe 9 is pre-emitted and the steady light is removed to perform the integration operation. As a result, the influence of the background subject not at a short distance can be removed, and the main subject can be detected with higher accuracy.

First, a low resolution mode in the AF area sensor 2 is set by operating the operation unit provided in the camera (step S131). Then, in accordance with a command from the control unit 1, pre-flash / steady-light removal integration control is performed in which the strobe light 9 is pre-flashed and stationary light is removed and integration operation is performed (step S132).

The predetermined integrated sensor data (low-resolution data) is read from the AF area sensor 2 in accordance with the read clock signal from the control unit 1, and is subjected to AD conversion by the AD converter 1d. , RAM1
c (step S133).

Next, a main subject is detected based on the low resolution data (step S134). During ranging,
The high resolution mode is set for the AF area sensor 2 (step S135). Then, it is determined whether or not the detected main subject has low contrast (step S13).
6) If the contrast is not low (YES), the steady light removal integration operation is performed based on the main subject position while pre-flashing the strobe 9 (step S13).
7).

Thereafter, normal integration control is performed based on the monitor signal of the main subject position (step S138), and AF is performed.
The sensor data (high-resolution data) is read from the area sensor 2 (step S139). In addition, the above step S
If it is determined in 136 that the contrast is low, (N
O), the process directly proceeds to step S139.

Next, a distance measurement area including the extracted main subject position is set, a distance measurement operation is performed within the distance measurement area (step S140), and the focusing lens is driven based on the obtained distance measurement data. (Step S14)
1).

As described above, the situation of the main subject is determined by detecting the main subject, and the integration mode (normal integration, pre-emission / steady light removal integration) is changed according to the situation. And the detection capability is improved.

Referring to the flowchart shown in FIG.
A seventh embodiment according to the distance measuring apparatus of the present invention will be described.

In the present embodiment, the AF area sensor 2 is set to the low resolution / contour mode, and the main subject is detected using the obtained low resolution / contour data.

First, the operation section provided in the camera is operated to set the AF area sensor 2 to the low resolution / contour detection mode (step S151). Then, the control unit 1 outputs an integration control signal to the AF area sensor 2 to perform an integration operation (step S152).

The predetermined integrated sensor data (low-resolution / outline data) is read from the AF area sensor 2 in accordance with the read clock signal from the control unit 1, and is subjected to AD conversion by the AD converter 1d. After, RA
It is stored in M1c (step S153). The stored sensor data is, for example, low-resolution data as shown in FIG. 14C in the shooting scene shown in FIG.

Next, a process for detecting a main subject based on the low resolution / contour data is performed (step S154).
At the time of distance measurement, a high resolution and
The contour mode is set (step S155).

Thereafter, integration control is performed based on the main subject position (step S156), and sensor data (high-resolution data) is read from the AF area sensor 2 (step S157). Next, a distance measurement area including the extracted main subject position is set, a distance measurement operation is performed within the distance measurement area (step S158), and a focusing lens is driven based on the obtained distance measurement data (step S158). S1
59).

As described above, low-resolution data can be obtained at high speed in the low-resolution mode, so that higher-speed processing can be performed as compared with the case of lowering the resolution by arithmetic processing such as addition. The main subject is detected by reducing the number of data based on low-resolution data, so that high-speed main subject detection processing can be performed.

Furthermore, since contour data can be obtained at high speed in the contour mode, processing can be performed at a higher speed than when contour data is obtained by arithmetic processing such as a difference. In addition, since the main subject is detected based on the contour data, highly accurate main subject detection processing is possible. Since the distance measurement calculation is performed based on the high resolution / contour data, highly accurate distance measurement is possible.

Although the above embodiments have been described, the present invention includes the following inventions.

(1) An AF area sensor that captures two images formed so as to have parallax and integrates sensor data based on an appropriate amount of received light, and an AF area sensor formed on the same semiconductor substrate as the AF area sensor. A light-receiving signal processing circuit that generates contour data from sensor data from the AF area sensor; a main subject in a shooting screen is detected from the contour data output from the light-receiving signal processing circuit; And a control unit configured to set within a shooting screen, and perform a distance measurement operation by measuring a distance within a distance measurement area set by the control unit.

(2) The AF area sensor includes a pixel area in which light receiving element groups for receiving distance measuring light having parallax formed by two light receiving lenses disposed in front thereof are arranged in a matrix, and A horizontal / vertical control circuit for scanning and outputting the amount of distance measurement light accumulated from each light receiving element in the pixel area by the command of the unit, an output circuit for amplifying a predetermined output signal from the horizontal / vertical control circuit, The distance measuring apparatus according to the above mode (1), wherein the sensor control circuit for controlling the light receiving accumulation and the output operation based on a command from the control unit is formed on the same silicon substrate by a CMOS process.

(3) The horizontal / vertical control circuit of the AF area sensor sequentially applies a sensitivity control signal one row at a time in the row direction of the light receiving element group arranged in a matrix, and accumulates in each light receiving element in the column direction. The light amount of the measured distance measuring light is extracted, only the portion where the light amount is changed is extracted, and the outline of one scanning line is detected simultaneously and in parallel and output as outline data.

[0166]

As described above in detail, according to the present invention, an imaging device for receiving two input images having parallax, and a light receiving device for generating data obtained by performing image processing on an image of a subject from a light receiving signal of the imaging device. A signal processing circuit is provided with an AF area sensor formed on the same semiconductor substrate, has a wide ranging area, enables detection of a main subject, reduces time lag, reduces size, and reduces cost. A ranging device can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram showing a schematic configuration example of a camera equipped with a distance measuring device as a first embodiment of the distance measuring device of the present invention.

FIG. 2 is a flowchart illustrating a main routine in photographing.

FIG. 3 is a diagram showing a conceptual configuration of a distance measuring optical system based on an external light passive system.

FIG. 4 is a diagram illustrating a specific configuration example of an AF area sensor.

FIG. 5 is a diagram illustrating a specific configuration example of a differential amplifier circuit SA.

FIG. 6 is a diagram showing a conceptual relationship between an AF area sensor and an input image.

FIG. 7 is a diagram illustrating a relationship between the size of a screen at a wide end and a tele end in a shooting screen and a distance measurement area.

FIG. 8 is a flowchart for explaining an AF routine in the distance measuring apparatus of the first embodiment.

FIG. 9 is a flowchart for describing a main subject detection routine.

FIG. 10 is a flowchart illustrating a binarization process (threshold process).

FIG. 11 is a diagram illustrating a relationship between luminance and frequency for setting a threshold.

FIG. 12 is a flowchart illustrating a threshold setting process.

FIG. 13 is a flowchart illustrating a shape determination process.

FIG. 14 shows an example of a photographing scene in person determination, and part 2
It is a figure which shows the outline data after a value process, low-resolution data, and low-resolution outline data.

FIG. 15 is a flowchart for explaining a second embodiment according to the distance measuring apparatus of the present invention.

FIG. 16 is a diagram illustrating a relationship between a shooting screen and a projection output in an AF routine according to the second embodiment.

FIG. 17 is a flowchart for describing a third embodiment according to the distance measuring apparatus of the present invention.

FIG. 18 is a diagram illustrating a shooting scene including a person and outline data of the person.

FIG. 19 is a flowchart illustrating a fourth embodiment of the distance measuring apparatus according to the present invention.

FIG. 20 is a diagram illustrating an example of projection in the x and y directions with respect to a shooting scene.

FIG. 21 is a diagram illustrating an example in which a pixel area of an AF area sensor corresponding to a shooting screen is divided into a plurality of blocks.

FIG. 22 is a flowchart for describing a fifth embodiment according to the distance measuring apparatus of the present invention.

FIG. 23 is a flowchart for describing main subject detection in a fifth embodiment.

FIG. 24 is a diagram for describing edge detection processing.

FIG. 25 is a diagram illustrating characteristics of an original image and a processed image captured by a first derivative operator method for performing edge detection.

FIG. 26 is a diagram illustrating an example of a spatial filter table.

FIG. 27 is a flowchart for describing main subject detection in a sixth embodiment.

FIG. 28 is a flowchart for describing main subject detection in a seventh embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Control part (microcomputer) 1a ... Central processing unit (CPU) 1b ... ROM 1c ... RAM 1d ... A / D converter 1e ... EEPROM 2 ... AF area sensor 2a ... Light receiving element group 2b ... Light receiving signal processing circuit 2c ... Stationary Light removal unit 3 Focus lens drive unit 4 Focus lens 5 Focus lens encoder 9 Strobe light emission unit 10 Strobe circuit unit 14 Zoom lens 15 Zoom lens drive unit 17 First release switch (1RSW) 18 Second release Switch (2RSW)

Claims (3)

[Claims] 1. An optical system having parallax, an imaging device for capturing two images formed by the optical system, and an image formed on the same semiconductor substrate as the imaging device, the image being output to the imaging device. A distance measuring device comprising: processing means for performing processing; and distance measuring means for measuring distance based on an output of the processing means. 2. An optical system having parallax, an image sensor for capturing two images formed by the optical system, and an image formed on the same semiconductor substrate as the image sensor and output to the image sensor. Processing means for performing processing; main subject detection means for detecting a main subject based on the output of the processing means; and distance measurement means for performing distance measurement based on the output of the processing means and the output of the main subject detection means. And a distance measuring device comprising: 3. The measuring apparatus according to claim 2, wherein said processing means has a plurality of processing modes, and outputs to said main subject detecting means and said distance measuring means can be set independently of each other. Distance device.