JPH09318870A - Range finder, photographing device, and background processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately extract a main object inside a rangefinding visual field at a high speed and to detect the image deviation amount or the distance of the main object part even though the object three-dimensionally exists in a wide field. SOLUTION: An image information detecting part 1 has a pair of photoelectric conversion element columns 1L and 1R. The columns 1L and 1R respectively photoelectrically convert a pair of object images formed by a photometric optical system arranged while having base length. Right and left image information is nearly simultaneously outputted corresponding to each other from each end part. The image information output from the part 1 is given to a main object detecting part 2. The part 2 detects the main object image based on the image information output from the part 1, and the image deviation amount of the main object image is calculated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring device suitable for a camera, and more particularly to a CCD (Charge Coupled Device:
The present invention relates to a passive distance measuring device using a photoelectric conversion element such as a charge-coupled device), a photographing device using the same, and a background processing device.

[0002]

2. Description of the Related Art Recently, 3D (3-Dime) using not only a still camera but also a video camera and two area CCDs has been used.
In most cases, various cameras including nsional (three-dimensional) image pickup devices incorporate an autofocus system that automates focusing. Such an autofocus system is mainly configured by a combination of a distance measuring device that measures the subject distance and a lens control device that controls the extension of the photographing lens. Conventionally, a distance measuring system in which a photographer designates a desired point in a wide distance measuring field and calculates a distance at that point, and the distance measuring field is divided into a plurality of distance measuring areas, and these distance measuring areas are A multi-point distance measuring system or the like has been proposed in which distance calculation is performed by selectively using.

Examples of conventional distance measuring devices are as follows. (1) Japanese Patent Application Laid-Open No. 7-287161 discloses that in a multipoint distance measuring device having a plurality of divided distance measuring areas, adjacent distance measuring areas are partially overlapped and set. By overlapping the distance-measuring areas in this way, the possibility of a so-called hollow-out phenomenon in which the distance-measuring object enters the blank area of the distance-measuring area, that is, the dead zone, is reduced.

(2) Japanese Patent Application Laid-Open No. 7-181010 discloses that
It has been shown that, in a multi-point distance measuring device, for a point for which the distance measurement result is uncertain, the distance is calculated based on the distance measurement result of a point located near that point and for which the distance measurement result has already been confirmed. There is. As described above, the time required for the distance measurement calculation is shortened by obtaining the distance measurement result of the indefinite point based on the known distance measurement result of the adjacent point.

(3) Japanese Patent Application Laid-Open No. 61-245124 discloses an image output for focus detection by an A / D (Analog-to-Di).
A focus detecting device is shown which shortens the time required for the distance measurement calculation by performing a predetermined calculation for the focus detection during the conversion from digital to analog-digital).

[0006]

However, the above-described conventional distance measuring system has the following problems. (1) Japanese Patent Laid-Open No. 7-287161 discloses a distance measuring device in which a plurality of divided distance measuring areas overlap each other. However, this distance measuring device merely proposes a method of eliminating the dead zone when the distance measuring points are simply subdivided to make multiple points. Therefore, no consideration is given to the main subject relating to actual photography, and the discrimination and distance measurement of the main subject depend on the photographer's judgment and operation. In addition, the distance measuring device does not take into consideration the increase in the calculation time due to the number of distance measuring points.

(2) Japanese Patent Laid-Open No. 7-181010 discloses that
If the distance measurement result is confirmed at a certain point, after that,
A range finder that correlates distances only in the vicinity is shown. This distance measuring device shortens the calculation time by estimating the similarity with an adjacent point. That is, based on the natural assumption that the subject straddles a plurality of distance measurement points, it is estimated that the adjacent points will have a result close to the already calculated result, and the correlation calculation will be performed in the vicinity of the result, and the distance measurement will be further performed. The calculation time is shortened by substituting the calculation result of the possible part for the part that cannot be calculated.

However, since the subject and the background coexist three-dimensionally, a sufficient distance measurement result cannot be obtained by simply simplifying the distance measurement calculation based on the distance measurement results in the vicinity. It is presumed that. Further, there is no disclosure about a method of limiting the search width of the correlation calculation. It should be noted that this publication also makes no mention of extraction of the main subject or separation of the subject and the background.

(3) Japanese Patent Application Laid-Open No. 61-245124 discloses a focus in which other processing is performed during A / D conversion of an image output so that the A / D converter and the arithmetic circuit move simultaneously. A detection device is shown. However, this publication describes, as an operation to be performed simultaneously with A / D conversion, not an important operation such as a correlation operation but an operation such as removal of an unnecessary frequency component, which is a kind of preprocessing, and an actual measurement. No special consideration has been given to the distance calculation itself. As described above, in most conventional distance measuring devices, the distance measuring field of view is divided into a plurality of distance measuring areas, the conventional correlation calculation is performed for each distance measuring area, and one of the results is calculated. I choose to focus on that distance.

The present invention has been made in view of the above-mentioned circumstances, and even when a subject is three-dimensionally present in a wide field of view, the main subject in the range finding field can be accurately and rapidly detected. An object of the present invention is to provide a distance measuring device that is capable of extracting and detecting the image shift amount or distance of the main subject portion. Particularly, it is a first object to detect the main subject by sequentially processing images right and left. Has an aim. A second object of the present invention is to accurately perform distance measurement calculation by detecting and using relative block shift and block reliability across a plurality of image information blocks.

A third object of the present invention is to separate the background and the main subject so that distance measurement can be accurately performed. A fourth object of the present invention is to configure the main subject detection unit with a microcomputer or the like, terminate the processing during the reading time, and enable high-speed distance measurement calculation.

A fifth object of the present invention is to finally determine the relative shift amount by updating the left and right blocks according to the result of processing, so that it is not necessary to calculate a plurality of shift amounts each time. As shown in Japanese Patent Publication No. 7-181010, the complicated judgment is not necessary, and the shift amount of the adjacent points is not arbitrarily determined and calculated, but the shift amount is changed based on the previous result. Accurate and high-speed calculation is performed by calculation. A sixth object of the present invention is to detect an appropriate relative shift amount by creating left and right blocks shifted by a plurality of relative shift amounts and performing a comparison operation.

A seventh object of the present invention is to set an aperture value and a focal length on the basis of the distribution of the image shift amount of the extracted main subject so that all the extracted main subjects can be focused. To provide a device.
An eighth object of the present invention is to provide a photographing device that allows a photographer to confirm the main subject extracted by the distance measuring device. A ninth object of the present invention is to provide a background processing device capable of detecting a distance for each part of a subject, extracting a main subject, and electrically blurring a background other than the main subject such as a portrait photograph. is there.

[0014]

In order to achieve the above-mentioned first object, a distance measuring device according to the present invention as set forth in claim 1 has a pair of left and right measuring devices arranged with a base line length. A distance optical system and a pair of left and right photoelectric conversion element rows for photoelectrically converting the subject optical image formed by the pair of distance measurement optical systems, respectively, and a pair of images photoelectrically converted by the pair of photoelectric conversion element rows. Information is sequentially processed from the end portions of the left and right photoelectric conversion element arrays, and a pair of left and right image information outputs from the image information detection section are sequentially processed to detect the main subject, and the main subject is detected. It is characterized by comprising a main subject detecting section for detecting an image shift amount.

According to a second aspect of the distance measuring device of the present invention, in order to achieve the above-mentioned second object, in the distance measuring device of the first aspect, the main subject detecting section is provided with a pair of provided main subject detecting parts. The image information is divided into a plurality of blocks, respectively, and the pair of image information block rows is relatively shiftable, and a block of the pair of image information blocks relatively shifted by the block forming unit. A block shift detecting unit that detects a shift, a reliability detecting unit that detects reliability evaluation information of the pair of image information blocks output from the block forming unit, and a block shift detecting unit and the reliability detecting unit. From the output, the block shift and the reliability of the pair of image information are evaluated over a plurality of image information blocks, and the main object in the pair of image information is evaluated. It is characterized in that it comprises a window evaluation unit for detecting at least one of the distance to the image shift and said principal subject's body.

According to a third aspect of the present invention, there is provided a range finder according to the second aspect, wherein the window evaluation unit is provided with a relative shift. Based on the amount and the block reliability evaluation information, a plurality of blocks within a predetermined range of subject distance are detected as continuous blocks, and a continuous block corresponding to the shortest distance among continuous blocks of a predetermined length is detected. Accordingly, it is characterized in that it includes means for detecting the main subject.

In order to achieve the above-mentioned fourth object, the distance measuring device according to the present invention described in claim 4 is characterized by
In the range finder according to any one of 3 above, the main subject detection unit performs a calculation process for detecting the main subject and its image shift amount during an image reading cycle from the image information detection unit. It is characterized by including a computer.

According to a fifth aspect of the distance measuring device of the present invention, in order to achieve the above-mentioned fifth object, a pair of right and left distance measuring optical systems are arranged with a base line length, A pair of left and right photoelectric conversion element arrays for photoelectrically converting a subject optical image formed by a pair of distance measurement optical systems, and a pair of left and right image information output from the pair of photoelectric conversion element arrays are divided into a plurality of blocks, respectively. And a relative shift unit that relatively shifts the pair of image information blocks formed by the block formation unit by a predetermined amount, and the pair of image information blocks shifted by the relative shift unit are compared and calculated. Along with this, a calculation unit for controlling the relative shift unit based on the calculation result to sequentially update the relative shift amount and detect the image shift amount of the subject is provided. There.

According to a sixth aspect of the distance measuring device of the present invention, in order to achieve the above-mentioned sixth object, in the distance measuring device of the fifth aspect, the arithmetic unit is provided with a plurality of different relative shift amounts. It is characterized in that it includes means for updating the relative shift amount as a reference based on the comparison of the comparison operation results of the image information blocks shifted to.

In order to achieve the above-mentioned seventh object, an image-taking device according to the present invention according to claim 7 comprises the distance measuring device according to claim 1, and is extracted by the main subject detecting section. It is characterized by including depth adjusting means for setting at least one of the aperture value and the focal length so that the plurality of distances calculated from the block shift amounts of the plurality of windows fall within the depth of field.

In order to achieve the above-mentioned eighth object, an image-taking device according to the present invention according to claim 8 comprises the distance measuring device according to claim 1 and is extracted by the main subject detecting section. It is characterized by including a distance measuring field display means for displaying a distance measuring field range corresponding to the window. The background processing apparatus according to the present invention according to claim 9 is
In order to achieve the above-mentioned ninth object, the distance measuring device according to claim 1 is provided, and image information of a subject corresponding to a distance area different from the main subject extracted by the main subject detecting unit is displayed. It is characterized in that it further comprises means for electrically blurring.

[0022]

That is, since the distance measuring device according to the present invention processes a pair of image outputs to discriminate the main subject and detects at least one of the distance and the image shift amount of the main subject image itself, a wide range of images can be obtained. Even when a subject exists three-dimensionally in the field of view, it is possible to accurately and quickly detect the image shift amount or distance of the main subject in the distance measuring field of view.
In the distance measuring apparatus according to the first aspect, the main subject detection unit sequentially processes a pair of left and right image outputs to detect the distance or the image shift amount of the main subject. In the past, the distance measuring field was divided into multiple parts, and the focus was on the distance measuring result of the distance measuring field in which the closest subject exists, whereas this distance measuring device extracts the image of the main subject itself. However, since the image shift of the main subject image is detected, the main subject is accurately captured,
The image shift can be detected by the subject information of only the main subject that does not include other subject information, and distance measurement with a small distance measurement error can be performed.

According to another aspect of the distance measuring apparatus of the present invention, the block shift detecting unit detects the block shift and the reliability detecting unit detects the reliability by using the image information blocked by the block forming unit, and further the window evaluation is performed. The unit evaluates reliability and block shift over a plurality of blocks to detect the image shift amount of the main subject. In this range finder, the reliability and the block shift are evaluated over a plurality of blocks, the image shift is calculated, and it is also possible to weight the information by using the weighted average by the reliability.
It is possible to perform accurate distance measurement without the error component.

In the distance measuring device according to the third aspect, since the continuity of blocks is evaluated by the window evaluation unit, only the main subject can be captured, and the photographer does not need to purposely designate the main subject. You can easily focus on the main subject. In the distance measuring device according to the fourth aspect, the calculation is performed using a microcomputer or the like during the read cycle of the image information. This distance measuring device can be easily realized by using a microcomputer or the like, and the calculation can be performed during the reading cycle. Therefore, it is necessary to separately secure the calculation time for the distance measuring calculation. Without
Further, it is not necessary to store and hold all pixel data in a storage element or the like.

According to the fifth aspect of the present invention, the distance measuring device updates the relative shift amount of the pair of image information blocks by using the sequentially processed calculation result, so that the relative shift amount is close to the image shift amount of the main subject. Since the values can be gradually approached, the relative shift amount becomes a value close to the image shift amount of the main subject, and the distance measurement calculation can be accurately performed. The distance measuring apparatus according to claim 6 obtains a plurality of block shift amounts or a plurality of reliability by performing a comparison operation on a relatively shifted block with respect to a plurality of different relative shift amounts, and uses the result as a reference. Since the relative shift amount is updated, the relative shift amount can be determined more accurately and easily.

For example, with respect to the reference relative shift amount S, S
If the block shift and the block reliability are obtained by +1 and S-1, if the reference relative shift amount S needs to be increased, S + 1 is more reliable than S-1 and the block shift is increased. The quantity also decreases. In this way, the relative shift amount can be easily updated appropriately. According to a seventh aspect of the present invention, the image capturing apparatus obtains the image shift amount or the subject distance from the block shifts of a plurality of windows, and even if the main subjects are not all present at a constant distance and are distributed over a certain distance range By setting the aperture value or the focal length of the photographing optical system, it is possible to bring all the main subjects into the depth of field.

According to the eighth aspect of the present invention, the main subject is detected by the main subject detection unit, and the distance measuring field of view corresponding to the main subject is displayed to notify the photographer of the detected main subject. Therefore, the difference in recognition between the photographing device side and the photographer becomes clear, and photographing due to erroneous distance measurement can be prevented in advance.

The background processing apparatus according to claim 9 extracts the main subject and measures the distance between the main subject and the two areas CC.
By combining with a photographing device such as a 3D camera using D or the like or a video camera using the contrast method, it is possible to obtain distance measurement information for each part in all parts in the photographing screen, and to obtain a subject image as a main subject ( It is possible to separate into a subject of the same distance) and other subjects, and by processing the video signal of the portion other than the same distance as the main subject,
A subject image other than the main subject can be made into a blurred image by electrical processing. With this background processing device, it is possible to obtain a portrait effect that emphasizes the main subject or to cut out only the main subject from the entire image, and it is possible to easily perform processing such as editing after shooting.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION The distance measuring device of the present invention will be described in detail below with reference to the drawings based on the embodiments. Figure 1
FIG. 4 shows a configuration of a main part of the distance measuring device according to the first embodiment of the present invention. The distance measuring device shown in FIG. 1 includes an image information detecting section 1 and a main subject detecting section 2.

The image information detecting section 1 has a pair of left and right photoelectric conversion element arrays 1L and 1R. As shown in FIG. 2, the pair of left and right photoelectric conversion element arrays 1L and 1R includes a pair of left and right distance measuring optical systems 3L arranged with a predetermined base line length.
And a pair of left and right subject images formed by 3R are photoelectrically converted. These pair of left and right photoelectric conversion element arrays 1
The result of photoelectric conversion of the pair of left and right subject images by L and 1R is output from the image information detection unit 1.

The outputs of the photoelectric conversion elements forming the left photoelectric conversion element array 1L are respectively L ₁ , L ₂ , L ₃ , ..., L.
and _n, the output of each R ₁ in the photoelectric conversion elements forming the right photoelectric conversion element arrays _{1R, R 2, R 3,} ..., when the R _n, from the image information detecting unit 1, the photoelectric conversion element arrays 1L
From the information of the end portions of the columns 1R and 1R, the image information outputs are sequentially output alternately in the left and right direction, for example, L ₁ , R ₁ , L ₂ , R ₂ , L ₃ , R ₃ ,. Thus, the outputs L _{1 to} L _n and R _{1 to} R _n of the pair of left and right photoelectric conversion element arrays 1L and 1R are the image information outputs L ₁ and L ₁ of the image information detection unit ₁ , respectively.
By sequentially outputting R ₁ , L ₂ , R ₂ , L ₃ , R ₃ , ... Alternately on the left and right, information on the left and right corresponding positions of the pair of left and right object images can be obtained almost at the same time.

For example, the information on the left end of the left subject image and the information on the left end of the right subject image can be obtained almost simultaneously.
Similarly, the information on the positions of the left and right subject images corresponding to each other is obtained almost simultaneously from one end to the other end, and when the information reaches the central portion, the information of each central portion of the left and right subject images is obtained almost simultaneously. By the way, in a conventional distance measuring device, it is general that image information is output for each photoelectric conversion element array. For example, after all the information of the photoelectric conversion element array 1L on the left side is output, photoelectric conversion on the right side is performed. Information of the element array 1R is output, and image information is output after R ₁ , L ₂ , ..., L _n , and then R
₁ , R ₂ , ..., R _n . Therefore, the left and right image information cannot be compared until all the image information of at least one photoelectric conversion element array is output.

The image information detecting section 1 in FIG. 1 outputs the left and right image information in this manner from the respective end portions substantially corresponding to each other. In this case, the image information detection section 1 is supposed to output the outputs of the photoelectric conversion element arrays 1L and 1R alternately to the left and right to one common signal line, and to output the same, but two independent signal lines are used. The L side and the R side may be output independently and information of each corresponding position may be output at the same time.

The image information output from the image information detecting section 1 is given to the main subject detecting section 2. The main subject detection unit 2 detects the main subject image based on the image information output from the image information detection unit 1 and calculates the image shift amount of the main subject image. FIG. 3 shows a detailed configuration of the main subject detection unit 2. The main subject detection unit 2 includes a block formation unit 21, a block shift detection unit 22, a reliability detection unit 23, and a window evaluation unit 24.

The block forming section 21 includes the image information detecting section 1
The image information input from is divided into small blocks and relatively shifted in the left and right directions. Block shift detection unit 2
2 calculates the interval between the blocks of the left and right image information formed by the block forming unit 21. The reliability detecting unit 23 detects the reliability of the blocks of the left and right image information formed by the block forming unit 21. The window evaluation unit 24 connects a plurality of blocks using the results of both the block shift detection unit 22 and the reliability detection unit 23, evaluates them as windows, and finally outputs the image shift amount of the left and right windows. .

The block forming section 21 will be described more specifically. In order to make the description easy to understand, in this embodiment, the description will be made assuming that the analog output is converted to digital and then processed. However, it is also possible to realize the processing corresponding to the following embodiments without changing the analog output. it can. The block forming unit 21 accumulates image information sequentially input in time series in a buffer to form a block composed of a plurality of pixels, and the blocks on the left and right of the block shift detecting unit 22 and the reliability detecting unit in the subsequent stage. 23.

That is, the block forming section 21 converts the given analog output into the A / D converter 2 as shown in FIG.
5, the data is converted into digital data, and the left and right data are input to the L side (left side) buffer 26 and the R side (right side) buffer 27, respectively. Whenever new data is input, this input data is sequentially shifted and finally discarded. That is, the L-side buffer 26 and the R-side buffer 27 hold new data of the input data as many as the number of buffers.

Next, a block is selected from these buffers by the block selection unit 28 and output to the subsequent stage. Here, the selected block is determined by the number of blocks and the relative shift amount. In this embodiment,
When the number of blocks is 4, when the relative shift amount is 0, L ₁ , L ₂ , L ₃ , and L ₄ and R ₁ , R ₂ , R ₃ , and R ₄ are selected, and the relative shift amount is In the case of +2, the R side is shifted to L ₁ , L ₂ , L ₃ , L ₄ and R
₃ , R ₄ , R ₅ , and R ₆ are selected, and the relative shift amount is −
In the case of 2, the L side is shifted to select L ₃ , L ₄ , L ₅ , and L ₆ and R ₁ , R ₂ , R ₃ , and R ₄ .

Therefore, if the subsequent processing is completed while the left and right four images are output, the buffers 26 and 2
As for 7, only the number corresponding to the number of blocks + the relative shift amount is required. For example, the number of blocks is 4, the relative shift amount is ± 8
If shifting is performed, a buffer that stores 12 image outputs is enough. In this case, both the block shift detection unit 22 and the reliability detection unit 23 in the subsequent stage need to perform the detection process while the left and right four images are input to the memory.

If these conditions are met, it is not necessary to store all the input image outputs in the memory, but it is sufficient to store 12 image outputs in each memory, and the block selection based on the relative shift amount is unique to the buffer. You can go to In this embodiment, the analog image data is digitally converted and stored in the memory, but the analog data may be stored in the CCD or the like as it is, or the block may be selected as it is in the analog data. .

Next, the block shift detecting section 2 for evaluating each of the left and right blocks formed by the block forming section 21.
2 and the reliability detection unit 23 will be specifically described.
The block shift detection unit 22 detects a block shift between the selected left and right blocks. This block shift detection unit 22
, The blocks already shifted by the relative shift amount are compared with each other. Therefore, on the actual pixel, the block of the subject image which has been shifted by, for example, d ₀ is even formed by the relative shift amount S ₀ to form a block. In this case, the block shift detector 21 detects the block shift of d ₀ −S ₀ . The reason for doing this will be described later, but it is because a rough image shift is detected by the relative shift amount, and the block shift detection unit 22 detects a precise image shift amount. Here, how to calculate the shift between the blocks will be described.

As shown in FIG. 5, the left and right blocks are Δd.
Is shifted only by To detect this Δd, for example, as shown in FIG. 6, (R _n −L _n ): Δd = (L _{n + 1}
-L _n ): 1, ΔL = R _n -L _n , C = L _{n + 1} -L
_{If n} , then Δd = (ΔL) / C can be obtained. In this way, L _n , R _n , L _{n + 1} , R _{n + 1} ,
The shift between the blocks can be detected by calculating the three points L _{n + 2} and R _{n + 2} and taking the average. Further, as is conventionally done, an evaluation function for shifts may be created by using a general correlation method, and the shift between the left and right blocks may be detected by using an interpolation method. That is, the block shift detection unit 22 may be configured using a digital circuit that performs the above calculation.

In the case of obtaining in an analog manner, the same operation as described above may be performed in an analog manner, or one of the left and right blocks is delayed by delay means or the like so that the left and right signal waveforms are superposed and coincide with each other. It may be detected from the delay amount. Next, the reliability detector 23 will be described. The reliability detection unit 23 detects the reliability of the image of these blocks or the reliability of the calculated block deviation amount for the left and right blocks similar to those for which the block deviation is detected as described above.

The reliability detecting section 23 detects the reliability based on the characteristics shown in FIGS. 7 to 9, the upper characteristic (a) of each drawing shows an example in which the reliability is high, and the lower characteristic (b) shows an example in which the reliability is low. FIG.
The output slopes of the left and right blocks are compared, and FIG.
In FIG. 7A, the inclinations on the L side and the R side are similar and the reliability is high, but in FIG. 7B, the inclinations on the L side and the R side are different, and the reliability is low. FIG. 8 compares the contrasts and FIG.
When the contrast is high as shown in (a), the reliability of the calculated block shift is high, and when the contrast is low as shown in FIG. 8B, the reliability of the calculated block shift is low.

In FIG. 9, the reliability is judged based on the continuity of the inclination in the block. When the variation of the inclination is small as shown in FIG. 9A, the reliability is high, and FIG. When the fluctuation of the slope is large as shown in), the reliability is low. With such a method, the reliability of the block used in the above-described calculation of the block shift amount is detected. From such left and right blocks, the block shift detection unit 22 and the reliability detection unit 2
3 detects the block shift and the block reliability, and gives them to the window evaluation unit 24. That is, by passing through both the block shift detecting unit 22 and the reliability detecting unit 23, 8 data of each 4 pixels on the left and right sides are converted into two data of a block shift amount and reliability. These two data are input to the window evaluation unit 24.

The reliability detecting section 23 can also calculate the reliability in an analog manner instead of digitally. For example, the left and right signals are input to a differentiating circuit to obtain the signal slope, and in the case of FIG. 7, the left and right differential outputs of the differential output may be obtained. In the case of FIG. 8, the magnitude of the differential output may be compared. In FIG. 9, it is sufficient to obtain the temporal variation of the differential output or the frequency. Therefore, it does not matter here whether it is processed digitally or analogically.

Next, the window evaluation section 24 will be described. The window evaluation unit 24 forms a window based on the input block shift amount and reliability, and calculates the image shift amount. First, the block shift amount input to the window evaluation unit 24 does not accurately represent the image shift amount of the main subject due to various factors. That is, the first factor is that the input block shift amount is not always the image shift amount of the main subject image, but may be the image shift amount of the background image of the subject.

That is, the images formed on the photoelectric conversion elements of the photoelectric conversion element rows 1L and 1R of the image information detecting section 1 are:
Not all images are of a constant distance, and generally, images of various distances are input three-dimensionally, so it is necessary to separate them to extract the main subject. The second second factor is that not all input block shift amounts are correct values, but errors are included due to various causes such as variations in photoelectric conversion elements. The block shift amount input to the window evaluation unit 24 mainly includes errors due to these two factors, and the image shift amount must be calculated by forming a window of the main subject in consideration of the errors. An accurate image shift amount cannot be detected.

First, consideration will be given to the removal of distance-measuring errors such as variations due to the latter second factor. For this, the result of the reliability detection unit 23 described above can be used. That is, the result of block shift including a large error is calculated for a block with low reliability, and the error is small for a block with high reliability. In other words, the reliability of the block is evaluated such that the window is formed by using only the block shift that does not include much error and the image shift amount is calculated. In addition, the degree of reliability is used as a parameter to weight the block shift amount,
A weighted average may be used. As a result, blocks having a larger error component can be excluded and a block shift amount having a smaller error component can be calculated.

Next, the extraction of the main subject relating to the former first factor will be described. First, it is necessary to define what the main subject is. This is because the main subject in a strict sense is determined only by the will of the photographer such as a camera or a video camera, and varies depending on the individual photographer. A special input device is required to cover all of these. In the present invention, the main subject is not discriminated by such a special human-powered device, but the generally defined main subject is extracted. First, as shown in FIG. 10, with respect to the viewfinder field that determines the composition of the shooting,
It is assumed that the distance measuring field of view, which is the range for detecting the distance to the subject, is set in a horizontal strip shape.

Generally, the person to be photographed may be a person photographed as shown in FIG. 11 or a landscape photographed as shown in FIG. In the case of FIG. 11, the main subject is a person, and in the case of such person photographing, it suffices to measure the distance closest to the subject in the distance measuring field of view. In the case of FIG. 12, the main subject is a landscape, and the distance may be measured anywhere in the distance measuring field of view. Therefore, in the case of photographing a general person and landscape as shown in FIGS. 11 and 12, it is sufficient to measure the closest object in the distance measuring field of view. On the other hand, in the case of a subject that does not apply to FIGS. 11 and 12, FIGS.
This will be described with reference to FIGS. 15 and 16.

FIG. 13 shows a case where an obstacle such as a fence or a net exists in front of the main subject. In such a case, when the obstacle in front of the person as the main subject is not so disturbing and is at an acceptable level, the photographer focuses on the person as it is and photographs. Also, if the obstacle on the front is at an unacceptable level, that is, if the fence or the like is thick and the person is hidden, the image is not taken. Therefore, in such a case, it is possible to determine whether this is the main subject or another image by determining the size of the image in the distance measuring field.

Next, as shown in FIG. 14, when the main subject stands adjacent to a wall or the like, the wall or the like is included in the distance measuring visual field. Further, as shown in FIG. 15, when the camera is tilted by 90 ° from the horizontal and held in a so-called vertical position, the ground will enter. In such cases, the distance between the wall and the ground is not constant, and in the example shown in the figure, the distance increases from the outside to the inside of the finder, so the detected distance changes. Therefore, the main subject is the only subject within a certain distance range. That is, the wall in FIG. 14 is not a constant distance but is gradually changing. The same applies to the ground in FIG.

Next, as shown in FIG. 16, in a scene where a person comes out from a narrow exit, there is an exit front surface at a short distance, and the front surface of the exit is likely to be measured. In such a case, there is contrast only at the exit edge of FIG.
Since there is often no contrast at other places at the exit, only the edge can be detected as a constant distance. As described above, the main subject in the case of FIGS. 11 to 16 can be captured by detecting the distance of the closest portion of the portion determined to have a certain distance in a certain continuous range.

Therefore, when distance measurement is performed on the main subject in FIGS. 11 to 16, a continuous subject having a certain size, that is, a portion where the distance measurement results are substantially the same in a certain size are continuously present. The main subject can be extracted by selecting the closest subject among them, that is, the portion that continuously exists. However,
Even with such a method, it is not always possible to completely extract the main subject. Therefore, a device such as a display described later may be applied.

Next, how the window evaluation section 24 performs such processing will be described. The above operation requires detection of the distance information of each point (each block) in the distance measurement area and the authenticity of the distance information, that is, the reliability of the information. 22 and the reliability detection unit 23 can detect them sequentially. Based on these pieces of information, blocks at a distance within a certain range are extracted and their continuity is detected. The distance within the certain range may be a range in which the subject is in focus, and may be all the objects having substantially the same distance as the main object. So the focal length,
It may be determined by the depth of field based on the aperture F value (F _NO ) or the like, or may be a constant value for easy detection.

Next, of the detected continuous blocks, a block having a certain length or more is extracted. In other words, this means ignoring short contiguous blocks,
This length may also be constant, but in conjunction with the focal length of the photographing device, it may be shorter for a wide angle and longer for a telephoto, or may be changed depending on the distance in which continuous blocks exist, The short distance may be long and the long distance may be short. Since the main subject is likely to be a person, the length may be set according to the width of the person. In this way, after removing short blocks,
The closest block among the remaining continuous blocks may be extracted to obtain the image shift or the distance.

This situation will be described with reference to the example shown in FIG. 17. As shown in FIG. 18, the subject of FIG. 17 is (a) a background a at a long distance, (b) a main subject b, and (main) It is formed by the foreground c that is a fence that blocks the subject.The output of the block shift detection unit 22 for such composition is illustrated in Fig. 19. In Fig. 19, the vertical axis represents the subject. 20 is the distance or image shift information, and the horizontal axis is the position of the block corresponding to the position of the ranging field of view of the subject that is sequentially output. is there.
In FIG. 20, the main subject is extracted by selecting the subject b corresponding to the short distance from the subjects a and b,
The image shift or the distance of that portion may be detected.

However, in this embodiment, since the output of the block shift detecting section 22 is sequentially output, the window evaluating section 24 can be constructed as shown in FIG. The window evaluation unit 24 shown in FIG. 21 includes a continuity check unit 31, a block counter 32, a window calculation unit 33, a candidate window shift amount storage unit 34, and an adopted window shift amount storage unit 35. First, the block shift amount and the reliability evaluation result calculated from the block shift detection unit 22 and the reliability detection unit 23 from the image information corresponding to the end portion of the photoelectric conversion element array are the window evaluation unit 24.
Are sequentially input to the continuity check unit 31 of.

The continuity checking unit 31 checks the continuity of the block based on the input block shift amount and the reliability evaluation result, and if there is continuity, increments (up-counts) the block counter 32.
At this time, the value of the block deviation amount having continuity is given to the window calculation unit 33, and is weighted averaged using the average of the block deviation amounts calculated up to that point or the reliability evaluation result. The result is the candidate window shift amount storage unit 3.
4 is stored.

Such an operation is repeated each time the block shift amount is input, and when the count value of the block counter 32 is a certain value or more, that is, when the continuous blocks are a certain value or more as described above, the candidates are selected. Only when the candidate window shift amount stored in the window shift amount storage unit 34 is sent to the adopted window shift amount storage unit 35 and the calculated result is a distance closer than the previously stored adopted window shift amount. The adopted window shift amount stored in the adopted window shift amount storage unit 35 is updated.

In this way, the block counter 3
Due to the length of consecutive blocks reflected in the count value of 2, it is possible to eliminate consecutive blocks of short length. Moreover, since the adopted window shift amount stored in the adopted window shift amount storage unit 35 is updated only when a result of a closer distance is input, only continuous blocks of a shorter distance can be selected. Next, the continuity check unit 31 will be described. The continuity may be judged by whether or not the change of the block deviation amount is within a certain range. However, as shown in FIG. 22, by using the reliability evaluation result of the block, the continuity of the block deviation can be closely examined. Sex can be evaluated.

First, when the reliability is high and the block shift is within a predetermined range (compared to the candidate window shift amount, etc.), the subject is a continuous subject, so the block counter 32 is incremented to reduce the block shift. It is used to calculate the candidate window shift amount. When the reliability is low and the block shift amount is within the predetermined range,
Although there is a high possibility that they are continuous, the accuracy of the distance measurement calculation is low, so the block counter 32 is incremented but not included in the candidate window shift amount.

When the reliability is high and the block shift amount is out of the predetermined range, it is highly likely that the block has a distance different from the previously calculated candidate window. Therefore, the block counter 32 is reset. And prepare for setting a new candidate window. In addition,
When the reliability is low and the block shift amount is out of the predetermined range, the reliability is low for setting a new candidate, so that nothing is done. In this way, the candidate window is set based on the input block deviation amount and the evaluation result of the reliability, and the deviation amount is obtained by the sequentially input deviation amount, so that the block counter 32 becomes a certain value or more. In this case, when the distance is shorter than the stored adopted window shift amount, the adopted window shift amount is updated.

In this way, by continuing until the sequentially input data is completed, the finally effective window shift result is stored as the adopted window shift amount, and this result is the image shift amount of the main subject. Correspond. This part can be easily configured by a digital arithmetic circuit, but can also be implemented by analog signal processing. That is, in order to remove the short blocks from FIG. 19 to obtain the one shown in FIG. 20, it can be realized by passing the analog low-pass filter 41 as shown in FIG. 23, and only the highly reliable portion is output. Finally, the level determination unit 43 may perform the level determination through the analog gate 42, and output only the short distance portion.

As described above, the image information corresponding to the other end portions is sequentially output from the respective one end portions of the pair of left and right photoelectric conversion element arrays 1L and 1R, and the sequentially output image information is formed into blocks. Input to the section 21. The block forming unit 21 selects right and left blocks according to the set relative shift amount and inputs them to the block shift detecting unit 22 and the reliability detecting unit 23. Based on the blocks input to the block shift detection unit 22 and the reliability detection unit 23, the block shift amount and the reliability evaluation value are obtained and input to the window evaluation unit 24. Window evaluation unit 24
From these values, evaluate the length of consecutive blocks, that is, consecutive blocks whose block shift amount is within a certain range, remove short blocks, To adopt.

With this configuration, each time the image information is sequentially input, each component operates to determine the distance or the image shift amount of the main subject when the image information transfer is completed. That is, as shown in FIG. 24, conventionally, all the information obtained by the left and right photoelectric conversion elements was once fetched and then processed, but in the present invention, it is calculated every time input information is sequentially input. Therefore, the input and the calculation are performed almost simultaneously in parallel, and the calculation is finished immediately after the end of the data input.

Therefore, in order to realize the present invention, it is essential to calculate the image shift amount or distance of the blocks divided into a plurality of blocks and to extract the main subject from the plurality of calculated results. Therefore, in the above, the extraction of the main subject has been mainly described. Next, the calculation of the image shift amount of the block will be described. As described above, the block shift detecting unit 22 detects the block shift from the block formed by the block forming unit 21, and the relative position of the block to be compared is indicated by the relative shift amount.

Here, the relative shift amount will be described.
It is assumed that the image information on the L side and the image information on the R side are input to the block forming unit 21 as shown in FIG. As shown in FIG. 25, it is assumed that the relative image shift amount between the L side and the R side is shifted by d = 2.8 sensor pitch. Where the first part S
With respect to a, the block forming unit 21 sets the relative shift amount to 0.
26 forms a left-right block of 4 pixels, the result is as shown in FIG. When such left and right blocks are input to the block shift detection unit 22, the difference between the information on the L side and the information on the R side is too large to accurately obtain the block shift. Further, the reliability detecting unit 23 also determines that there is no reliability.

Next, when a block is formed with the relative shift amount being 2, the result is as shown in FIG. 27, the correlation between the L side and the R side is high, and the block shift detecting section 22 can detect the block shift easily. Theoretically, the image shift amount is 2.9−the relative shift amount 2 = 0.9, and the block shift detection unit 22 can detect a value of 0.9. Further, when the relative shift amount is 3, it becomes as shown in FIG. 28, and the correlation between the left and right becomes stronger, and the image shift amount 2.9−the relative shift amount 3
= -0.1. Further, the reliability of the left and right blocks also becomes high, and the reliability detecting unit 23 also determines that the reliability is high.

That is, when comparing left and right blocks,
By forming a block with a relative shift amount close to the image shift amount of the left and right blocks and comparing the blocks, the image shift amount, that is, the relative shift amount + the block shift amount can be detected more accurately. Comparing blocks with a relative shift amount close to the image shift amount means comparing almost the same parts of the subject image, and comparing with different relative shift amounts compares the displaced parts of the subject image. Since it means that, it goes without saying that the above is the case. In the above description, the relative shift amount is not particularly mentioned and is treated as a constant amount, but it is impossible to detect an accurate image shift. The problem is how to set this relative shift amount.

Therefore, each time image information is sequentially input, the relative shift amount is changed, for example, from -n to n, and reliability and block deviation are checked to detect the best relative shift amount, and the relative shift amount is detected. If the value obtained by adding the shift amount and the block shift is input to the window evaluation unit 24, the best relative shift amount can be detected for each block. However, since the relative shift amount is changed from −n to n for each block to detect the block shift amount and the reliability, the calculation scale becomes very large. Further, a relative shift amount may be set to be close to the image shift amount of the distance at which the presence of the main subject is predicted, but this is effective when the image shift amount from infinity to the closest distance is small.

However, these methods are not very practical. Here, a second embodiment of the present invention will be described. As a premise of the second embodiment, the image shift amount when the subject distance is infinity is defined as 0, and the image shift amount when the subject exists at the shortest distance of the shooting range is defined as n. First, the relative shift amount as an initial value is 0. Sequentially
The image information is input, the block creation unit 21 creates a block based on the relative shift amount 0, and the block shift detection unit 2
2 detects the amount of block shift, and the reliability detector 23 detects reliability. Here, when the reliability is higher than a certain value and the block shift amount is a certain value (for example, 1 or more), the image shift amount of that portion is likely to be larger than the relative shift amount at that time, and thus the relative shift amount is large. Increase the amount. Then, each time information is sequentially input, the block forming unit 21
Forms a block with the updated relative shift amount, and the block shift detection unit 22 includes the formed left and right blocks.
The reliability detection unit 23 detects the block shift and the reliability, and outputs a value obtained by adding the relative shift amount and the block shift amount at that time to the window evaluation unit 24.

By doing so, the relative shift amount increases in accordance with the image shift amount. In this case, since the relative shift amount only increases, if there is even one short-distance subject, the detection accuracy of the long-distance subject in the subsequent blocks deteriorates. However, in general, the main subject has a high probability of being the closest subject, and since only the closest subject in the distance measuring field can be extracted, this is not a serious problem. However, the present invention also aims to remove a foreground that is sufficiently small with respect to the main subject as shown in FIG. Therefore, first, in the case where the relative shift amount is shifted only in the short-distance direction, FIGS.
This will be described with reference to FIG.

FIG. 29A shows a case where the subject is only a long-distance subject such as a landscape. In this case, the image shift amount of the subject is uniformly distributed with respect to the visual field position as shown in FIG. 29B. are doing. On the other hand, the relative shift amount does not change, and
It is constant like 9 (c). Next, when the main subject is in front of the background as shown in FIG.
As shown in (b), the image shift amount changes at the position where the main subject exists. Therefore, as shown in FIG. 30C, the relative shift amount is close to the image shift amount in the background portion on the left side of the screen, and the relative shift amount increases at the central main subject portion, and the image shift amount is increased. Get closer to the value.

Next, in the background portion on the right side of the screen, the relative shift amount does not change even if the image shift amount corresponds to the background.
When the block shift created based on such a relative shift amount is calculated and input to the window evaluation unit, R shown in the figure is displayed.
The window is adopted in the section a to measure the distance. Next, when there is an unnecessary foreground as shown in FIG. 31 (a), the distribution of the image shift amount with respect to the visual field range is as shown in FIG. 31 (b). In this case, when the shift amount is updated only for the short distance as described above, the relative shift amount is updated closer to the main subject in the foreground of the left fence portion as shown in FIG. It becomes impossible to extract the main subject.

Therefore, the relative shift amount is updated not only for the increase but also for the decrease. By the way, since the distance measuring device does not need to detect a subject (background) farther than the main subject, it is not necessary to reduce the relative shift amount farther than the main subject. That is, since it is not necessary for the window evaluation unit 24 to detect continuity of blocks to detect an object farther than the adopted window, the relative shift amount is a value whose initial value corresponds to infinity (here, 0). As a result, increase and decrease. In this case, the limit value is 0, but the window evaluation unit 24
When a continuous block in a certain range is adopted, the value may be increased or decreased to the limit value.

That is, in the initial state, the range of the relative shift amount increases or decreases from 0 at infinity to the closest n, but a continuous block having the window evaluation unit 24 is adopted, and the relative shift amount at that time is adopted. Is m (0 ≦ m ≦ n), the range that the relative shift amount can take after that is from m to n, the range that the relative shift amount can take is narrowed, and the main subject can be more accurately captured. Can be extracted. The change in the relative shift amount by this method is shown in FIG.
1 (d), the window is adopted in the Qa portion, the window is also adopted in the Qb portion, and the main subject is finally extracted with the relative shift amount corresponding to the Qb portion. At this time, the method of reducing the relative shift amount is performed based on the detection value of the block shift detection unit 22 as in the case of increasing it, and when the block shift is equal to or less than a certain value (for example, -1), the image shift of that portion is performed. The amount is likely to be less than the relative shift amount there, thus reducing the relative shift amount.

Since the relative shift amount is obtained by following the image shift amount of the subject by the method as described above, even if the subject is three-dimensionally present in the visual field range, the image shift amount of each subject is determined. Following this, the relative shift amount changes and is finally updated to a relative shift amount close to the image shift of the main subject, and the left and right blocks are created with this relative shift amount, so the block shift detection unit 22 detects an accurate value. can do. In the above, the relative shift amount has been increased / decreased based on the magnitude of the block shift amount. Next, a method of increasing / decreasing the relative shift amount more reliably will be described.

In the above-mentioned method, the actual image shift and the relative shift amount are far from each other, and the block shift is obtained by the left and right blocks created thereby, that is, FIG.
In the case of 6, the block shift cannot be accurately obtained, and in some cases it may not be estimated. In such a case, naturally, it may not be known how to update the relative shift amount. Therefore, the left and right blocks are created with a large relative shift amount (for example, +2) and a small relative shift amount (for example, −2) with respect to the reference relative shift amount, and the block shift amount and reliability are provisionally obtained for that block. . When the image shift is larger than the reference relative shift amount, the reliability of the small relative shift amount (-2) becomes poor.

Further, when the image shift is smaller than the reference relative shift amount, the reliability of the large relative shift amount (+2) becomes poor. Therefore, it is sure which of the reference relative shift amounts should be updated. Can be detected. This is just a method similar to the so-called mountain climbing method in the contrast method. That is, in the contrast method, the photographing lens or the image pickup element is vibrated to move the focus in the direction in which the contrast is increased, but a similar idea is applied to the update of the relative shift. By using such a method, the relative shift amount can be evaluated more accurately by changing the shift amount more reliably than by increasing or decreasing the shift amount according to the block shift amount while fixing the reference relative shift amount. Can be updated.

By changing the relative shift amount by the above method,
The block shift and the reliability are obtained, and the window can be evaluated by this value to detect the image shift of the main subject. Further, when the distance measurement operation is repeatedly performed on a moving subject, the distance can be measured accurately and at high speed by performing the distance measurement again using the finally adopted relative shift amount of the window.

Further, by applying the present invention,
The main subject is extracted, and the distribution of multiple block shifts of the extracted main subject (distance distribution of the main subject) can be known. Therefore, even if the main subject is not flat but rather uneven in the front and back, the entire range is It is possible to provide a photographing device, that is, a camera, that can enter the depth of field. In addition, since only the main subject can be extracted, when the field of view range corresponding to the main subject is displayed or when a 3D electronic image pickup device is created using two area sensors, the information of this area sensor is used. A pseudo telephoto lens effect or the like can be created by extracting a main subject and measuring the distance, or electrically blurring a subject other than the same distance as the main subject. In particular, the present invention is suitable for a 3D electronic image pickup device using two area sensors or the like because it can sequentially process left and right images.

The distance measuring method according to the present invention is mainly effective for the external light triangulation and the TTL phase difference method. However, by determining the window of the closest distance in continuous blocks, Is not only a distance measurement method that uses the image shift, but is not a distance measurement method that has a baseline length by applying the contrast method that finds the distance to the subject by detecting the size of the contrast in block units. It is also effective as a contrast method. In such a case, the part corresponding to the reliability corresponds to the contrast of the block, the part corresponding to the relative shift amount corresponds to the position of the photographing lens or the image pickup element, and the block shift detection uses the previous partial contrast in the contrast method. Corresponds to detection of the amount of extension or the subject distance calculated by predicting the amount of extension that maximizes the contrast.

In this way, one area sensor is divided into parts, the contrast is obtained, and the amount of extension is predicted by the time-series change of the contrast, and the contrast of that part is used as reliability information. By doing so, the present invention can be applied to a normal video camera or the like. Next, using a microcomputer etc.,
A method of calculating during the image reading period will be described.

As shown in FIG. 32, the photoelectric conversion elements L ₁ and L _{2 are}
A photoelectric conversion element array 1L composed of L _n and a photoelectric conversion element R
The image output of the photoelectric conversion element array 1R composed of ₁ , R ₂ ... R _n is alternately output like L ₁ , R ₁ , L ₂ , R ₂ ... L _n , R _n , and the output is A / D. The converter 51 quantizes the data into digital data, which is taken into a CPU (Central Processing Unit) 52 for processing. In this embodiment, when the object is at infinity, the image shift amount of the object image formed by the object is zero, at that time, L _1, L as shown in FIG. 33
_{It is} assumed that the image outputs of R ₁ , R ₂ , R ₃ , and R ₄ match with ₂ , L ₃ and L ₄ .

Also. As shown in FIG. 33, when the image shift amount is 1.
Is L₁ , L_Two , L_Three , And L_Four And R_Two , R_Three , R_Four ,
And R_Five The image output of the
When the image with the image shift amount n matches,₁ , L_Two , L_Three , And
And L_Four And R_{1 + n} , R_{2 + n} , R _{3 + n} , And R_{4 + n} Image of
Suppose the outputs match. Next, the overall flow shown in FIG.
The chart will be described. This flow chart is
It is a flow chart during the project period and is performed before that.
Control of storage time of photoelectric conversion element and distance measurement
The lens extension calculation after calculation is not directly related to the present invention.
Therefore, the description is omitted.

First, when the processing is started, necessary variables and the like are initialized (step S1). Next, wait until new data for A / D conversion is input (step S2), and the input image output is A / D converted (step S3). Here, when the A / D converter 51 that takes a long time for A / D conversion is used, the subsequent processing may be performed in parallel and input to the buffer after the conversion is completed. If the converted data is L side image information, it is stored in the L side buffer, and if it is R side image information, it is stored in the R side buffer (step S4). This is necessary to form blocks.

The state of the buffer is shown in FIG. In FIG. 35, a portion indicated by an arrow is a place where the latest data is stored, and L is sequentially arranged from the top row to the bottom row.
_It is shown that data is accumulated as ₁ , L ₂ , L _3, ... In this example, 7 data from the latest data to 7 data before are stored. In addition, the order of the stored data is that, starting from the pointer indicating the latest data, 7 data are cyclically stored in the left direction as shown in the figure. Can be taken out.

Further, if there is a buffer of 7 data and the number of blocks is 4 pixels, the relative shift amount in the R side buffer can be 3 pixels. Since a buffer similar to this can be provided on the R side as well, it is possible to store L and R image information and create a block. Next, it is checked whether the buffer can be formed into blocks (step S5). This is to determine whether the image information is not sufficiently stored in the buffer as in the top row, the second and third rows of FIG. 35, and if the buffer does not have the necessary data, the subsequent processing is performed. Not proceed to step S
Returning to step 2, the image output is output and the process of inputting the image information into the buffer is returned to.

If the buffer is sufficient to create a block, it is judged whether n is an even number or an odd number (step S6), and the steps S7 and S8 are performed alternately, or steps S10 and S11 are performed. Get the work done. Block misregistration detection (step S7) and block reliability detection (step S8) can be calculated only after the left and right blocks have been created, and are therefore more efficient after L and R have been read.

Since the work of steps S10 and S11 is performed using the results of the work of steps S7 and S8, the procedure is that after the L and R blocks can be created, the block shift amount and the block shift amount are calculated in steps S7 and S8. The reliability is calculated, and the window evaluation (step S1
0), and the relative shift amount is updated (step S11), the calculation amount can be dispersed, and one block can be processed left and right each time 2 data (a pair of L and R) is read. Both L and R of the block are 1
If the relative shift amount does not change by processing the block that becomes a pixel, the buffer size does not change.

Further, when the processing is more complicated and the processing of steps S7 and S8 or the processing of steps S10 and S11 cannot be performed in one cycle, the steps S7, S8, S10 and S1 are thus performed in two cycles.
If the processes of steps S7, S8, S10 and S11 are completed in four cycles instead of ending the process of one, the amount of calculation per cycle can be reduced. In this case, in order to prevent the buffer amount from increasing as the cycle progresses, by selecting the left and right blocks from 2 cycle 1 processing to 4 cycle 1 processing as shown in FIG. Since the number of unnecessary data becomes the same, the buffer amount does not increase.

Next, as described above, n is incremented (step S9), and the above work is performed until the image output is completed (step S12). However, strictly speaking, since the information remains in the buffer even after the image output is completed, the processing may be continued until the information in the buffer is exhausted. Since the subject extraction and the subject image shift amount have been detected by the above read operation, the remaining calculation and the like are performed (step S13). here,
Step S, which is the main processing step in the present invention
7, S8, S10 and S11 will be described in detail. First, the prerequisite variables are listed below.

S: Relative shift amount BL _{1 to} BL ₄ : L side block BR _{1 to} BR ₄ : R side block For example, when the relative shift amount S = 0, in the case of two cycle processing, BL _{1 to} BL ₄ are _{L 1 ~L 4, BR 1 ~BR} 4 corresponds to R ₁ to R _4, the BL ₁ to BL _4, at the next processing L
_{2 to} L ₅ and BR ₁ to BR ₄ correspond to R _{2 to} R ₃ . Further, when the relative shift amount S = 1, when BL _{1 to} BL ₄ correspond to L _{m to} L _{m + 3} , BR _{1 to} BR ₄ are R _{m + 1.}
Corresponds to ~ Rm _{+ 4} . Actually, since the buffer shown in FIG. 35 is directly operated, BL _{1 to} BL ₄ and BR ₁ to
No storage is needed for variables such as BR ₄ .

L: variable used for calculation of block shift amount, sum of difference between L side and R side C: variable used for calculation of block shift amount, total sum of inclination between images d: calculated block displacement amount CH ₁ ~CH _3: L-side, the larger the slope between R side of the image CL ₁ -CL _3: smaller inclination first, the block shift amount detection and the block reliability detection, 37 and 38 Will be described with reference to. The block shift amount is calculated by the method described with reference to FIG.
When using pixels pixel by pixel, there are four differences between the L and R sides,
Since the inclination between pixels is 6 on the L side and 3 on the R side, the inclination between the pixels is doubled at the central two pixels for consistency, and the central portion is focused. 8
By forming the parts, the difference between the L side and the R side is doubled to be equivalent to 8 parts, and the matching is achieved.

First, in the block shift detection of FIG. 37, L
The output difference between the pixels on the right side and the pixel on the R side is calculated in four parts and summed (step S21). Next, in calculating the inclination between pixels, BL ₃ −BL ₂ and BR ₃ −BR ₂ corresponding to the central portion are calculated as 2
It is multiplied and the sum is obtained (steps S22 and S23). L and C obtained from these steps S22 and S23
Then, the block shift amount is calculated (step S24).
In this embodiment, the difference between the L side and the R side and the sum of the inclinations between the pixels are obtained to obtain the block shift amount. However, it is also possible to obtain each block individually and use the average thereof.

Next, the block reliability detection of FIG. 38 will be described. In this embodiment, the comparison of the left and right inclinations in FIG. 7 and the continuity of the inclinations in FIG. 9 are used. First, determine the left and right tilt of the three by three lateral than by 4 pixel data of the block, the more the inclination large CH ₁ to CH _3, substituted inclination of smaller to CL ₁ -CL ₃ (step S3
1, S32, S33). Next, (CL) / (CH) is calculated (step S34). When the absolute values of the left and right inclinations are close to each other, (CL) / (CH) becomes a value close to 1, and when they are different from each other, it approaches 0. Therefore, the magnitude of (CL) / (CH) becomes the magnitude of reliability, and CL _{1 to} CL ₃ and CH _{1 to} C
If H ₃ becomes 0 or the sign is different,
Since the continuity is poor, the reliability H is set to 0.

Therefore, when the reliability is high, H = {(C
L ₁ ) / (CH ₁ )} + {(CL ₂ ) / (CH ₂ )} +
{(CL ₃ ) / (CH ₃ )} approaches 3 and becomes 0 when it is low. A window is extracted using the block shift amount d and the reliability H obtained in the processing of FIGS. 37 and 38, and the window evaluation in step S10 and the relative shift amount update in step S11 will be described.

First, the variables used will be described. S
d and SH are variables for calculating the window shift amount by the weighted average based on the results of the block shift amount d and the reliability H, d ₀ is the calculated window shift amount of the candidate candidate window, and K is Corresponding to the depth of field, the calculated block shift amount and the candidate window shift amount are compared with each other to determine whether or not it is within the K range. W represents the length of the window, WL is a constant that determines whether or not the window is adopted, dd is the window shift amount of the adopted window, and ss is the relative shift amount of the adopted window. N is the maximum value of the relative shift amount and corresponds to the closest distance.

The window evaluation detection of FIG. 39 will be described. This flowchart was created based on the example of FIG. First, a candidate window is calculated in order to determine what value the newly calculated block shift is with respect to the current candidate window (step S41). In step S41, the candidate window deviation amount do is calculated by weighted averaging the reliability H integrated value SH and the block deviation amount integrated value sd. Next, the reliability is checked (step S42). If it is determined in step S42 that the reliability is high, whether the newly calculated block shift amount d and the relative shift amount s at that time are within the predetermined range K with respect to the candidate window shift amount do. It is determined whether it is out of the range (step S43).

If it is determined in step S43 that it is within the range, it is added to the variables Sd and SH for calculating the candidate window, and w representing the length of the window is increased (step S44). In step 43, if it is out of the range, w that represents the length of the window is cleared to set a new candidate window, and the new block shift amount d and reliability H are substituted into Sd and SH (step S4).
5). If it is determined in step S42 that the reliability is low (small), it is similarly compared with the candidate window (step S42).
46), when the data is out of the range, it is highly likely that the data with low reliability is accidentally input, so this data is ignored.

When the value is within the range, the reliability is low, but there is a high possibility that the blocks are continuous. Therefore, the window length w
Only Sd and SH are not included (step S
47). To judge whether or not the candidate window judged up to this point can be sufficiently adopted as a main subject, and to judge whether or not the reliability is sufficiently large as a main subject because the reliability has been sufficiently evaluated at the time of the candidate window. It is compared with the constant WL (step S48). If it is determined to be the main subject, substitute it for the image shift amount dd of the main subject,
The relative shift amount at that time is substituted into SS (step S
49). This SS is a limiter for the range in which the relative shift amount described below is updated. Therefore, the window evaluation is repeated until the image reading is completed, and the finally remaining image shift amount d
d corresponds to the closest object among the objects having reliability and size considered to be the main object.

Next, updating of the relative shift amount shown in FIG. 40 will be described. First, the reliability value H is checked (step S51). If the reliability is low, the relative shift amount cannot be updated, so this routine ends. If it is determined in step S51 that the reliability is high (large), the block shift amount d is checked (step S52), and if the absolute value d is greater than 1, the relative shift amount S is updated (step S53 and S54). The updated relative shift amount S is checked (step S55), and when it is determined that the relative shift amount SS is smaller than the relative shift amount SS at the time of determining the adoption window that has been determined previously, the relative shift amount S is set to the relative shift amount SS. It is restricted so that it does not become smaller (step S56).

Also, in step S55, the update of the relative shift amount S is restricted even when it is determined that it is larger than 0 to n (relative shift amount of the closest distance) in the distance measurement interlocking range (step S57). By doing so, unnecessary fluctuations in the relative shift amount are prevented. In other words, it is not necessary to detect a window that is closer than that window after a certain window is adopted and this becomes the adopted window and the relative shift amount SS at this time is determined.

By thus executing the program of the flowchart of FIG. 32 using a microcomputer or the like, the main subject can be extracted and calculated in the image output read cycle, and a low-speed A / D converter is used. However, even if a photoelectric conversion element such as a CCD having a low transfer rate is used, the image shift amount can be calculated during the cycle, so that the overall speed can be increased. Further, in this embodiment, it is not necessary to store the entire image data in a storage area such as a RAM (random access memory) only by providing a certain buffer, and it is not necessary to perform nondestructive reading of the CCD.

[0107] In addition, 3 using two area CCD
Also for the D camera, since it is not necessary to store and calculate the entire data of the two area CCDs, three-dimensional object information is calculated by the present invention from the output two area CCD outputs, and the three-dimensional information and the two-dimensional image information are calculated. By storing only the three-dimensional information and the two-dimensional information, a system that reproduces three-dimensional image information can be easily realized.

[0108]

As described above, according to the present invention, even when a subject exists three-dimensionally in a wide field of view,
Accurately and quickly extract the main subject in the ranging field of view,
It is possible to provide a distance measuring device, a photographing device, and a background processing device capable of detecting the image shift amount or the distance of the main subject portion.

According to another aspect of the distance measuring apparatus of the present invention, the main subject detecting section sequentially processes the pair of image outputs to detect the distance or image shift amount of the main subject. In the past, the distance measuring field was divided into multiple parts, and the focus was on the distance measuring result of the distance measuring field in which the closest subject exists, whereas this distance measuring device extracts the image of the main subject itself. However, since the image shift of the main subject image is detected, the main subject can be accurately captured, and the image shift can be detected by the subject information of only the main subject that does not include other subject information. Distance measurement with a small error can be performed.

According to another aspect of the distance measuring apparatus of the present invention, the block shift detecting unit detects the block shift and the reliability detecting unit detects the reliability using the image information blocked by the block forming unit, and further the window evaluation is performed. The unit evaluates reliability and block shift over a plurality of blocks to detect the image shift amount of the main subject. In this range finder, the reliability and the block shift are evaluated over a plurality of blocks, the image shift is calculated, and it is also possible to weight the information by using the weighted average by the reliability.
It is possible to perform accurate distance measurement without the error component.

In the distance measuring apparatus according to the third aspect, since the continuity of the blocks is evaluated by the window evaluation unit, only the main subject can be captured, and the photographer does not need to purposely designate the main subject. You can easily focus on the main subject. In the distance measuring device according to the fourth aspect, the calculation is performed using a microcomputer or the like during the read cycle of the image information. This distance measuring device can be easily realized by using a microcomputer or the like, and the calculation can be performed during the reading cycle. Therefore, it is necessary to separately secure the calculation time for the distance measuring calculation. Without
Further, it is not necessary to store and hold all pixel data in a storage element or the like.

According to the fifth aspect of the present invention, the distance measuring device updates the relative shift amount of the pair of image information blocks by using the sequentially processed calculation result so that the relative shift amount is close to the image shift amount of the main subject. Since the values can be gradually approached, the relative shift amount becomes a value close to the image shift amount of the main subject, and the distance measurement calculation can be accurately performed.

According to a sixth aspect of the distance measuring apparatus, the relatively shifted blocks are compared and calculated for a plurality of different relative shift amounts to obtain a plurality of block shift amounts or a plurality of reliability, and the results are used. Since the relative shift amount serving as a reference is updated, the relative shift amount can be determined more accurately and easily. For example, if the block shift and the block reliability are calculated for S + 1 and S-1 with respect to the reference relative shift amount S, and if the reference relative shift amount S needs to be increased, it is more than S-1. The reliability of S + 1 is higher and the amount of block shift is also reduced.
In this way, the relative shift amount can be easily updated appropriately.

According to the seventh aspect of the present invention, the image shift amount or the subject distance is obtained from the block shifts of a plurality of windows, and the main subjects are not all present at a constant distance, but are distributed within a certain distance range. However, by setting the aperture value or the focal length of the photographing optical system, it is possible to bring all the main subjects into the depth of field. The photographing apparatus according to claim 8 can notify the photographer of the detected main subject by extracting the main subject with the main subject detection unit and displaying the range-finding visual field range corresponding to the main subject. The difference in recognition between the device side and the photographer becomes clear, and photographing due to erroneous distance measurement can be prevented.

The background processing device according to claim 9 extracts the main subject and measures the distance between the main subject and the two areas CC.
By combining with a photographing device such as a 3D camera using D or the like or a video camera using the contrast method, it is possible to obtain distance measurement information for each part in all parts in the photographing screen, and to obtain a subject image as a main subject ( It is possible to separate into a subject of the same distance) and other subjects, and by processing the video signal of the portion other than the same distance as the main subject,
A subject image other than the main subject can be made into a blurred image by electrical processing. With this background processing device, it is possible to obtain a portrait effect that emphasizes the main subject or to cut out only the main subject from the entire image, and it is possible to easily perform processing such as editing after shooting.

[Brief description of drawings]

FIG. 1 is a block diagram schematically showing a basic configuration of a distance measuring device according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram mainly showing a configuration of a distance measuring optical system in the distance measuring device of FIG.

FIG. 3 is a block diagram schematically showing a detailed configuration of a main subject detection unit in the distance measuring device of FIG.

FIG. 4 is a block diagram schematically showing a detailed configuration of a block forming unit in the distance measuring device of FIG.

5 is a diagram showing an example of image information output for explaining the operation of the distance measuring device in FIG.

6 is a diagram showing an example of a block shift detection calculation for explaining the operation of the range finder of FIG.

7 is a diagram showing an example of left and right tilt characteristics contributing to reliability for explaining the operation of the distance measuring device of FIG.

8 is a diagram showing an example of a contrast characteristic that contributes to reliability for explaining the operation of the distance measuring device in FIG.

9 is a diagram showing an example of a continuity characteristic of inclination contributing to reliability for explaining the operation of the distance measuring device of FIG.

10 is a diagram showing an example of a relationship between a viewfinder field of view and a range finding field of view for explaining the operation of the range finding device of FIG.

FIG. 11 is a diagram showing a relationship between an example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG. 1.

FIG. 12 is a diagram showing the relationship between another example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG.

13 is a diagram showing the relationship between another example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG.

FIG. 14 is a diagram showing the relationship between another example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG. 1.

15 is a diagram showing a relationship between a further example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG.

16 is a diagram showing the relationship between still another example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG. 1. FIG.

17 is a diagram showing the relationship between another example of a subject and a distance measuring field for explaining the operation of the distance measuring device in FIG. 1. FIG.

18 is a view for explaining the operation of the distance measuring device of FIG.
It is a figure which decomposes | disassembles and shows the to-be-photographed object.

19 is a view for explaining the operation of the distance measuring device of FIG.
It is a figure which shows the ranging result of each to-be-photographed object.

20 is a view for explaining the operation of the distance measuring device of FIG.
It is a figure which shows the ranging result which removed the unnecessary part of each to-be-photographed object.

FIG. 21 is a block diagram schematically showing a detailed configuration of a window evaluation unit in the distance measuring device of FIG.

22 is a diagram for explaining the relationship between the reliability, the block shift amount, and the reliability in the distance measuring device of FIG.

FIG. 23 is a block diagram schematically showing another detailed configuration of the window evaluation section in the distance measuring device of FIG.

24 is a diagram for explaining the relationship between the calculation time and the conventional calculation time in the distance measuring device of FIG.

FIG. 25 is a diagram for explaining block shift detection in the distance measuring device of FIG. 1.

FIG. 26 is a diagram for explaining an example of relative shift in the distance measuring device in FIG. 1.

27 is a diagram for explaining another example of relative shift in the distance measuring device of FIG. 1. FIG.

28 is a diagram for explaining another example of the relative shift in the distance measuring device in FIG.

FIG. 29 is a diagram for explaining details of a distance measuring principle in the distance measuring device of FIG. 1.

FIG. 30 is a diagram for explaining details of a distance measuring principle in the distance measuring device of FIG. 1.

FIG. 31 is a diagram for explaining details of a distance measuring principle in the distance measuring device of FIG. 1.

32 is a block diagram schematically showing another detailed configuration of the main subject detection unit in the range finder of FIG. 1. FIG.

FIG. 33 is a diagram for explaining details of a distance measuring principle in the distance measuring device of FIG. 1.

34 is a flow chart for explaining a distance measuring operation in the distance measuring device of FIG. 1 in detail.

FIG. 35 is a schematic diagram for explaining in detail the distance measuring operation in the distance measuring device of FIG. 1.

FIG. 36 is a schematic diagram for explaining in detail the distance measuring operation in the distance measuring device of FIG. 1.

37 is a flow chart for explaining in detail block shift detection in the distance measuring device of FIG. 1. FIG.

38 is a flowchart for explaining block reliability detection in the range finder of FIG. 1 in detail.

39 is a flow chart for explaining window evaluation detection in the range finder of FIG. 1 in detail.

FIG. 40 is a flowchart for explaining in detail the update of the relative shift amount in the distance measuring device of FIG. 1.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image detection section 1L, 1R Photoelectric conversion element array 2 Main subject detection section 3L, 3R Distance measurement optical system 21 Block formation section 22 Block deviation detection section 23 Reliability detection section 24 Window evaluation section 25, 51 A / D (analog- Digital) converter 26, 27 buffer 28 block selection unit 31 continuity check unit 32 block counter 33 window calculation unit 34 candidate window shift amount storage unit 35 adopted window shift amount storage unit 41 low-pass filter 42 analog gate 43 level determination unit 52 CPU (Central processing unit)

Claims (9)

[Claims] 1. A pair of left and right distance-measuring optical systems arranged with a base line length, and a pair of left and right photoelectric conversion element arrays for photoelectrically converting a subject optical image formed by the pair of distance-measuring optical systems, respectively. And a pair of image information photoelectrically converted by the pair of photoelectric conversion element rows, an image information detection unit that sequentially outputs from the end of each of the left and right photoelectric conversion element rows, and the left and right from the image information detection unit. A distance measuring device, comprising: a main subject detection unit that sequentially processes a pair of image information outputs to detect a main subject and detects an image shift amount of the main subject. 2. The main subject detecting section divides a pair of image information given thereto into a plurality of blocks, respectively, and a block forming section capable of relatively shifting the pair of image information block rows, and the block forming section. A block shift detection unit that detects a block shift of the pair of image information blocks relatively shifted by a unit, and a reliability that detects reliability evaluation information of the pair of image information blocks output from the block formation unit. A detection unit, by evaluating the relative block shift and reliability of the pair of image information over a plurality of image information blocks from the output of the block shift detection unit and the reliability detection unit,
The distance measuring apparatus according to claim 1, further comprising a window evaluation unit that detects at least one of an image shift of a main subject and a distance to the main subject in the pair of image information. 3. The window evaluation unit detects a plurality of blocks within an object distance within a predetermined range as continuous blocks based on a given relative shift amount and block reliability evaluation information, and continuously detects the blocks with a predetermined length. 3. The distance measuring apparatus according to claim 2, further comprising means for detecting the main subject by detecting a continuous block corresponding to the shortest distance among the continuous blocks. 4. The main subject detecting section includes a microcomputer for performing arithmetic processing for detecting a main subject and an image shift amount thereof during an image reading cycle from the image information detecting section. Item 7. The distance measuring device according to any one of items 1 to 3. 5. A pair of left and right distance measuring optical systems arranged with a base line length, and a pair of left and right photoelectric conversion element arrays for photoelectrically converting a subject optical image formed by the pair of distance measuring optical systems, respectively. A block forming unit that divides a pair of left and right image information output from the pair of photoelectric conversion element columns into a plurality of blocks, respectively, and a pair of image information blocks formed by the block forming unit, which are relatively predetermined amounts. The relative shift means for shifting and the pair of image information blocks shifted by the relative shift means are compared and calculated, and the relative shift means is controlled based on the calculation result to sequentially update the relative shift amount. A distance measuring device comprising: a calculation unit that detects an image shift amount of a subject. 6. The calculation unit includes means for updating a reference relative shift amount based on comparison of comparison calculation results of image information blocks shifted to a plurality of different relative shift amounts. The distance measuring device according to claim 5. 7. The distance measuring device according to claim 1 is provided, and a plurality of distances calculated from block shift amounts of a plurality of windows extracted by the main subject detection unit are within the depth of field. An image pickup apparatus comprising: a depth adjusting unit that sets at least one of an aperture value and a focal length. 8. The distance measuring device according to claim 1, further comprising distance measuring visual field display means for displaying a distance measuring visual field range corresponding to a window extracted by the main subject detecting section. Shooting device. 9. A means for electrically blurring image information of a subject corresponding to a distance area different from the main subject extracted by the main subject detecting unit, comprising the distance measuring device according to claim 1. A background processing device comprising: