JP3163668B2 - Subject recognition device and camera - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognizing device for detecting a position occupied by an object of interest to a photographer in an object scene captured by a photographing lens provided in a camera. In a normal camera, focus control and exposure control are performed based on information on the central portion of the field of view, on the assumption that the subject is located at the central portion of the field of view of the taking lens provided in the camera. .

On the other hand, photographing is often performed in a composition in which a subject is located at a position off center, and in such a case, focusing is performed based on information on a subject located at a position off center. There is a demand for performing control and exposure control. In response, the direction of the line of sight of the photographer is determined from the direction of the eyeball of the photographer using the fact that the photographer is gazing at the object. As a technique, focus control and exposure control have been proposed.

[0003]

2. Description of the Related Art As a technique for detecting a line of sight of a photographer as a subject position and performing focus control and exposure control, there is a technique disclosed in JP-A-63-94232. FIG. 19 shows the configuration of the camera control device described above.

In FIG. 19, an image pickup circuit 614 includes a two-dimensional CCD (hereinafter simply referred to as a CCD) 615, on which an image formed by an image pickup optical system including a photographing lens 610 and an aperture 612 is formed. Configuration. In the figure, the eye movement detector 6
Reference numeral 30 denotes a configuration in which the direction of the eyeball is determined from the position of the iris and transmitted to the gate control circuit 634 (see the Journal of the Institute of Television Engineers of Japan, Vol. No. 2, 1986, p. 41 et seq.).

The gate control circuit 634 generates a gate control signal for controlling the opening and closing of the gate 616 based on the result of the determination and the horizontal synchronization signal and the vertical synchronization signal from the clock circuit 628. CCD 61 corresponding to the gaze portion of the photographer indicated by the determined determination result
5, the gate 616 is instructed to open. As a result, a portion that the photographer is gazing at is extracted from the output of the imaging circuit 614, and the auto focus (A
F) Control circuit 620 and automatic exposure (AE) control circuit 6
The control signal is sent to the lens drive unit 22 and is used for controlling the lens drive unit 624 and the aperture drive unit 626, respectively.

As described above, it is possible to perform focus control and exposure control based on information on the line of sight of the photographer looking through the viewfinder, and to obtain information on the object portion regardless of the position of the object in the field. A camera control device that controls a camera based on a camera has been proposed.

[0007]

By the way, in the above-mentioned conventional system, the position of the line of sight of the photographer obtained from the direction of the eyeball is set as the subject position. However, the photographer
The user does not always look closely at the subject, but looks at something other than the subject to see the balance of the composition, or looks at information related to exposure control and the like displayed outside the screen. Therefore, if the line of sight is set as the subject position, there is a possibility that the subject intended by the photographer may not be correctly recognized.

[0008] Further, since the eyeball is constantly moving, and the line-of-sight position changes every moment with this movement, the exposure value and the focus position change according to the line-of-sight position that changes every moment, and the exposure control and the focus are changed. Control becomes unstable. In particular, the eyeball sometimes moves abruptly, and in this case, the control instability as described above becomes remarkable. SUMMARY OF THE INVENTION It is an object of the present invention to provide a subject recognizing apparatus that correctly recognizes a subject of interest to a photographer and obtains a stable subject position of the subject. It is another object of the present invention to provide a camera that captures an appropriate image of a subject using the obtained subject position.

[0009]

FIG. 1 is a diagram showing the configuration of the first and second aspects of the present invention. According to the first aspect of the present invention, a subject recognizing unit 111 determines, as a position of a subject, a position at which a photographer is gazing within a field of a camera, based on the movement of an eyeball of the photographer, and an intensity distribution of light in the field. The position corresponding to the subject is determined based on the information about the subject, the tracking means 112 for tracking this position is compared with the position obtained by the subject recognizing means 111 and the position obtained by the tracking means, A determination unit 113 that determines whether the subject has changed the subject, and one of the positions obtained by the subject recognition unit 111 and the tracking unit 112 is selected as the subject position according to the determination result by the determination unit 113. The position obtained by the object recognizing means 111 is selected as information about a new object by the selecting means 114 and the tracking means 112 in accordance with the determination result that the object has been changed.
And a control means 115 for controlling the tracking process by transmitting to the control unit.

According to a second aspect of the present invention, in the object recognition apparatus according to the first aspect, the tracking means sets the position obtained by the object recognition means as an initial position of the object, and based on the light information at the initial position. It is characterized in that the position corresponding to the subject is determined. FIG. 2 is a diagram showing a configuration of the subject recognition device according to claim 3.

According to a third aspect of the present invention, in the subject recognition apparatus according to the first aspect, the luminance at each position in the object scene is determined from the intensity distribution of light in the object scene, and is provided for tracking processing by tracking means. The tracking means 11
2 determines a position having a luminance close to the object,
It is characterized in that it is configured to track this position. FIG.
FIG. 7 is a diagram showing a configuration of a subject recognition device according to claim 4.

According to a fourth aspect of the present invention, in the subject recognition apparatus according to the first aspect, a color at each position in the object scene is determined from a light intensity distribution in the object scene, and is provided to tracking processing by tracking means. A color discriminating unit 117 is provided.
Is characterized in that a position having a color similar to a subject is determined and this position is tracked. FIG.
It is a figure which shows the structure of the camera of Claim 5 and Claim 6.

According to a fifth aspect of the present invention, the object recognizing device 110 according to the first aspect, information on the light intensity distribution in the object scene and the object position selected by the selection means are input, and the brightness at the object position is input. And an exposure calculating means 121 for obtaining an exposure value using the luminance information indicating the exposure value and for performing exposure control. According to a sixth aspect of the present invention, the subject recognition device 110 according to the first aspect, information on the light intensity distribution in the object scene, and the subject position selected by the selection unit are input, and the subject indicated by the subject position is input. A focus determining unit 131 that determines a focus state of a corresponding light image and provides the focus control.

[0014]

According to the first aspect of the present invention, the object recognizing means 111
Since the position where the photographer gazes is obtained as the position of the subject, when the photographer gazes at another subject, the new subject position is obtained by the subject recognizing unit 111.
Therefore, the selecting unit 114 selects the position obtained by the subject recognizing unit 111 as the subject position according to the determination result of the determining unit 113, so that the subject that the photographer is paying attention to can be correctly recognized. At this time, since the control unit 115 sends the information on the new subject to the tracking unit 112, the tracking unit 112 performs a tracking process on the new subject, and determines whether the photographer is gazing at the subject. Regardless, the latest subject position is always obtained. Therefore, the selection unit 114 selects the position obtained by the tracking unit 112 according to the determination result of the determination unit 113, so that the position of the subject can be stably obtained.

According to the second aspect of the present invention, the object recognition means
Is used as the initial position of the tracking processing by the tracking means 112, the tracking means 112 can track the correct position of the subject from the beginning. According to the third aspect of the present invention, since the luminance at each position is determined by the luminance determining means 116, the tracking means 112 obtains the position having the luminance closest to the luminance of the subject.
The position of the subject can be easily tracked.

According to a fourth aspect of the present invention, the color discriminating means 117 discriminates the color of each position in the object scene, and the tracking means 112 obtains a position having a color similar to the color of the object, thereby determining the object position. It is possible to more reliably seek and track. According to the fifth aspect of the present invention, the subject position can be stably obtained from the subject recognition device 110. Therefore, the exposure calculation unit 121 obtains an exposure value using the luminance information of the subject position, thereby adapting to the subject. Exposure can be controlled, and an appropriate image of the subject can be obtained.

According to a sixth aspect of the present invention, an object recognizing device 110 is provided.
Since the object position can be obtained stably from the above, it is possible to perform focus control suitable for the object by judging the in-focus state of the light image of the object indicated by the object position by the focus judging means 131. And an appropriate image of the subject can be obtained.

[0018]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 5 is a configuration diagram of an embodiment of a single-lens reflex camera to which the subject recognition device of the present invention is applied. In FIG. 5, a part of the light beam incident on the camera via the photographing lens 210 is jumped up by the main mirror 201 and forms an image on a focus plate 203 provided on the incident surface of the pentaprism 202 except at the time of exposure. When the photographer looks into the viewfinder, the image on the focus plate 203 can be observed via the eyepiece 204.

A beam splitter (BS) 205 is provided inside the above-mentioned eyepiece 204, and the subject recognizing means 111 observes the movement of the photographer's eyeball through this eyepiece 204. It has a configuration. FIG. 6 shows a detailed configuration of the subject recognition means 111.
In FIG. 6, the light emitted from the light emitting element 221 is incident on the eyeball of the photographer looking through the finder by the half mirror 222 and the beam splitter 205 of the eyepiece 204. Also, the lens 223
Is configured to form an image of light reflected by the retina of the eyeball on a two-dimensional CCD image sensor (hereinafter simply referred to as CCD) 224. The CCD 224 is formed of nxm elements, and is configured to measure the intensity distribution of an image by the lens 223. The detection processing circuit 225 is configured to detect the position of the photographer's line of sight based on the output of the CCD 224.

Here, the reflected light from the retina has the highest intensity in the direction in which the human eye is gazing. That is, it is possible to determine the gaze direction of the photographer based on the position of the element corresponding to the maximum value of the intensity distribution obtained by the above-described CCD 224. For example, the field of the photographing lens 210 is divided into n × m element regions, and the magnification of the lens 223 and the like are determined so that each element region corresponds to each element of the CCD 224 described above. If so, the line-of-sight position can be detected as the coordinates of the element corresponding to the local maximum value.

In this case, the detection processing circuit 225 is a CCD
224 are sequentially compared to find the maximum value, and the coordinates of the corresponding element may be output as the line-of-sight position. Here, since the eyeball of the photographer is always moving, the gaze position obtained as described above is always moving with the movement of the eyeball. However, when deciding a subject or the like, the photographer gazes at the subject to be photographed.
As shown in Fig. 7, during this time, the range of movement of the line-of-sight position is concentrated in a relatively small area. In other words, if the state where the gaze position is concentrated within the predetermined range continues for a predetermined time or more, it is considered that the photographer is gazing at the subject,
If the range in which the line-of-sight position has moved during the predetermined time T is smaller than a predetermined size, it is determined that the photographer is gazing at the subject, and the position of the subject may be obtained from this range.

For example, the coordinates of the line-of-sight position obtained by the detection processing circuit 225 are input to the line-of-sight tracking unit 226, and the line-of-sight tracking unit 2
26 is the maximum value Xmax, Ymax and minimum value X of each coordinate component
min, Ymin, and based on these values, a decision circuit 2
27 may be configured to determine whether the photographer is gazing at the subject. The determination circuit 227 determines, for example, a difference between the maximum value and the minimum value of each coordinate component, and determines that the subject is gazing when both of the obtained differences are smaller than a predetermined threshold. If at least one of them is equal to or more than the threshold, it may be determined that the user is not watching. Also,
In accordance with this determination result, the position calculation unit 228 calculates an intermediate value between the maximum value and the minimum value of each coordinate component and outputs it as coordinates (Xa, Ya) indicating the position of the subject.

In this way, the position where the photographer gazes can be determined as the position of the subject from the line-of-sight position that changes in accordance with the movement of the photographer's eyeball. Alternatively, the gaze tracking unit 226 may hold the coordinates of the gaze position, and the position calculation unit 228 may calculate an average value of these coordinates as the position of the subject. Also, in FIG.
The lens 206 is disposed on another optical axis slightly deviated from the optical axis of the eyepiece 204, and a two-dimensional CCD image sensor (hereinafter, referred to as a “CCD”) disposed at a position crossing the optical axis.
An image on the focus plate 203 is formed on a CCD 231. This CCD 231 is formed of n × m elements similarly to the above-described CCD 224, and each element is further divided into three, and each of them is red (R) component, green (G) component, and blue (B). A filter for selectively transmitting the components is provided. FIG. 8 shows the spectral characteristics of the filters for these color components. That is,
Each element of the CCD 231 is connected to the focus plate 203 described above.
The configuration is such that the amount of light in the minute area corresponding to each of the upper images is measured for each color component, and the intensity of each color component is output.

Here, since the image on the focus plate 203 corresponds to the field of the photographing lens 210, the CCD 231
Represents the color in each element region of the object field of the photographing lens 210. That is, the above-described lens 206
And the CCD 231, the function of the color determining means 117 can be realized, and the outputs r (X, Y), g of the CCD 231 can be realized.
What is necessary is just to send (X, Y) and b (X, Y) to the tracking means 112 as a determination result.

Next, a method for tracking the position of a subject based on the above-described spectral information will be described. Here, when the subject position at a certain time is given as the coordinates (Xb, Yb) in the object scene, the subject position when the time Δt has elapsed from the certain time is described above when the time Δt is short. It is considered that the position is near the coordinates (Xb, Yb).
Further, it is considered that the spectral characteristic indicating the color of the subject is substantially constant.

Therefore, the tracking means 112 may determine a location close to the previously detected subject position and having a similar spectral characteristic as a position having characteristics corresponding to the subject, and track this position. . FIG. 9 shows the tracking means 11.
2 shows a detailed configuration. FIG. 10 is a flowchart showing the tracking processing.

In FIG. 9, a tracking unit 112 includes a coordinate storage unit 241, a spectral information storage unit 242, and a distance calculation unit 2.
43, a difference calculation section 244, and a discrimination processing section 245. The coordinate holding unit 241 holds the coordinates (Xb, Yb) of the element of the CCD 231 corresponding to the subject position obtained so far, and the spectral information holding unit 242
As spectral information indicating the color of the subject, values R (Xb, Yb) obtained from the outputs of the elements of the CCD 231 indicated by the coordinates,
G (Xb, Yb) and B (Xb, Yb) are held.

The spectral information R (Xb, Yb), G (Xb, Yb), B
(Xb, Yb) represents R,
Outputs r (Xb, Yb), g (Xb, Yb), b (X
b (Yb) is normalized by using the sum of them, and R (Xb, Yb) = r (Xb, Yb) / (r (Xb, Yb) + g (Xb, Yb) + b (Xb, Yb) G (Xb, Yb) = g (Xb, Yb) / (r (Xb, Yb) + g (Xb, Yb) + b (Xb, Yb)) B (Xb, Yb) = b (Xb, Yb) / ( r (Xb, Yb) + g (Xb, Yb) + b (Xb, Yb)). Therefore, the above-mentioned spectral information R (Xb, Y
b), G (Xb, Yb) and B (Xb, Yb) each have a value range of 0 to 1.

These coordinates and spectral information are sent to a distance calculator 243 and a difference calculator 244, respectively. First, the coordinate holding unit 2 described above
It is determined whether the coordinates of the subject and the spectral information are stored in the spectral information storage unit 41 and the spectral information storage unit 242 (step 301). When the tracking processing by the tracking means 112 is started, since information on the subject has not been set, a negative determination is always made in step 301 and step 3 is performed.
In 02, the coordinates indicating a specific position in the field (for example, the center of the field) and the output of the corresponding element of the CCD 231 are set as initial values in the coordinate storage unit 241 and the spectral information storage unit 242. . In this case, the following tracking processing is performed assuming that the subject is located at the above-described position.

When the tracking process is started, it is considered that the photographer is gazing at the subject, and the coordinates obtained by the subject recognizing unit 111 are stored in the coordinate holding unit 241.
The output of the CCD 231 corresponding to the coordinates may be stored in the spectral information storage unit 242. In this case, the position where the photographer gazes is set as the initial position, so that the subject can be accurately captured and tracked from the beginning.

A warning to reset the position of the object is displayed outside the screen corresponding to the object field in the viewfinder, and the photographer is prompted to set the reset switch (shown in the figure) with the object in the center of the screen. You may be prompted to press zu). In this case, the coordinates corresponding to the center of the screen and the output of the corresponding element of the CCD 231 may be set according to the depression of the reset switch.

On the other hand, in the case of an affirmative determination in step 301, the distance calculation unit 243 calculates the distance between the position of the subject and each element region of the object scene, and the difference calculation unit 244 calculates The difference between the spectral information indicating the color of each element region of the object scene and the spectral information indicating the color of the subject position is obtained (step 303). Here, as described above, the CCD
Each element 231 corresponds to an element region of the field. Therefore, the distance calculation unit 243 replaces the distance in the object scene with the CC indicated by the coordinates (Xb, Yb) described above.
The distance between the element of D231 and each of the other elements is the maximum distance I.
The distance dI standardized by max (= {(X _n −X ₁ ) ² + (Y _m −Y ₁ ) ² }) may be obtained. That is, the distance calculation unit 2
43 uses the maximum distance Imax and coordinates of each element (X, Y), is represented by dI (X, Y) = { (Xb-X) 2 + (Yb-Y) 2} 1/2 / Imax What is necessary is just to find the distance dI (X, Y) and send it to the determination processing unit 245. However, the maximum distance Imax described above is C
Coordinates of two elements at opposite corners on diagonal line of CD231
_{(X 1, Y 1),} (Xn, Ym) using, represented by _{Imax = {(Xn-X 1} ) 2 + (Ym-Y 1) 2} 1/2.

The difference calculator 244 is provided with a CCD 231.
From the output of each element, spectral information R indicating the color of each element area
(X, Y), G (X, Y), and B (X, Y) are obtained, and the spectral information of the subject is subtracted from the spectral information to obtain dR (X, Y) = R (Xa, Ya) −R (X, Y) dG (X, Y) = G (Xa, Ya) −G (X, Y) dB (X, Y) = B (Xa, Ya) −B (X, Y) dR (X, Y), dG (X, Y), and dB (X, Y) are obtained, and the obtained differences dR (X, Y), d
G (X, Y) and dB (X, Y) may be sent to the determination processing unit 245.

The discrimination processing unit 245 calculates the difference d between the distance dI obtained in step 303 and the spectral information.
By using R, dG, and dB as parameters, a fuzzy calculation process described later is performed to determine a position corresponding to the subject,
The corresponding position is set as the position of the new subject. The fuzzy arithmetic processing described below is performed if dR (X, Y) = small and dG (X, Y) = small and dB (X, Y) = sma
Using the fuzzy rule represented by ll and dI (X, Y) = nearthen S (X, Y) = big, the position of each element region of the scene represented by coordinates (X, Y) corresponds to the subject. This is a process of calculating the degree of conformity S (X, Y) for the proposition that the position is to be performed. In the fuzzy rules described above, each expression indicates a proposition about the corresponding parameter, and as a whole, the differences dR (X, Y) and dG (X,
When the proposition that each of Y) and dB (X, Y) is small (small) and the proposition that the distance dI (X, Y) is short (near) are all “true”, the above-described fitness S It means that the proposition that (X, Y) is big is “true”.

First, from the membership function for each parameter shown in FIGS. 11 (a), (b), (c) and (d), the distance dI
Is near and the difference dR, d between the spectral information
The suitability for the proposition that G and dB are small is calculated (step 304). In step 304, the discrimination processing unit 245 obtains the value of the intersection between the hedge of the membership function corresponding to each proposition and the value of the corresponding parameter, and sets this value as the degree of conformity of each proposition.

Next, the degree of conformity corresponding to each parameter is m
Using the ax-min synthesis method, the degrees of fitness for the above four propositions are synthesized, and the position of each element region of the scene indicated by coordinates (X, Y) is a position corresponding to the subject. Find the degree of conformity S (X, Y) for the proposition (step 30)
5). Here, in the max-min combining method, the minimum value of the fitness of each parameter is taken as the fitness of the whole fuzzy rule. Therefore, step 3
In step 05, the minimum value is selected from the fitness of each parameter, the intersection of the membership function with the corresponding hedge of the fitness S shown in FIG. 11 (e) is obtained, and the value of this intersection is determined by the above-mentioned proposition. What is necessary is just to make it final S (X, Y).

The degree of conformity S (X, Y) corresponding to each element of the CCD 231 thus obtained indicates the degree to which the position of each element region of the object scene corresponds to the object. Then, the maximum value of the conformity S (X, Y) is detected (step 306), and the coordinates of the corresponding element of the CCD 231 may be set as the coordinates (Xb, Yb) indicating the position of the subject. As described above, the degree of the element area having a color similar to the object from each element area of the object field and close to the position of the object obtained previously is evaluated, and the position corresponding to the object is evaluated. Can be determined. Thus, even when the brightness of the subject changes greatly, such as when the subject moves from the shade to the sun, the change in the spectral information indicating the color of the subject is relatively small, and the The corresponding position can be determined.

The discrimination processing unit 245 sends the coordinates (Xb, Yb) obtained as described above and the output of the corresponding element of the CCD 231 to the coordinate holding unit 241 and the spectral information holding unit 242, respectively ( Step 307), the coordinates (Xb, Yb) indicating the position of the subject and the spectral information R (Xb, Yb), G (Xb, Yb), B (Xb, Yb) indicating the color of the subject are updated. ing.

Therefore, by repeating steps 301 to 307 described above, the position of the subject can be reliably tracked following the movement of the subject in the scene. Note that the discrimination processing unit 245 obtains the sum of the parameters obtained in step 303 for each element region of the object scene, and sets the element region of the object scene that gives the minimum value of the sum as the subject position. Is also good.

In this case, since it is not necessary to perform the above-described complicated arithmetic processing, the configuration of the discrimination processing unit 245 can be simplified, and the tracking processing can be speeded up. However, even when only one parameter is significantly different from the parameter corresponding to the previous subject position, if the values of the other parameters are close to each other, the position of the subject may be determined. The error is larger than when the processing is used.

Also, in FIG.
1, the coordinates (Xa, Ya) obtained by the tracking means 112 and the coordinates (Xb, Yb) obtained by the tracking means 112 are input to the recognition processing unit 250, and the recognition processing unit 250 (Xo, Yo) are determined and provided for exposure control processing and focus control processing to be described later. FIG.
2 shows a detailed configuration diagram of the recognition processing unit 250. FIG.
FIG. 3 shows a flowchart of the processing for determining the subject position.

In FIG. 12, a distance calculation circuit 251 and a comparison circuit 252 form a determination means 113, and the distance calculation circuit 251 generates two coordinates (the two coordinates obtained by the object recognition means 111 and the tracking means 112). The configuration is such that a distance D between two points indicated by Xa, Ya) and (Xb, Yb) is obtained, and a comparison circuit 252 compares this distance D with a predetermined threshold. The comparison circuit 252 includes the object recognition unit 111 described above.
The determination result of the determination circuit 227 is input, and the comparison operation is performed in response to the input of the determination result indicating that the photographer is gazing at the subject.

That is, the above-described subject recognition means 111
When the determination circuit 227 determines that there is a position where the photographer is gazing, an affirmative determination is made in step 401, and the comparison circuit 252 compares the distance D obtained by the distance calculation circuit 251 with the threshold. (Step 402).
Also, in FIG. 12, the selector 253 is
14 and the comparison result of the comparison circuit 252 and the judgment circuit 2 of the object recognizing means 111.
27, the two coordinates (Xa,
One of (Ya) and (Xb, Yb) is selected as the subject position (Xo, Yo).

Here, if a value corresponding to the size occupied by a standard subject (for example, a person or the like) in the object scene is set as a threshold value in the comparison circuit 252, the comparison result of step 402 indicates that the object recognition means 111 Tracking means 112
Then, it can be determined whether or not the obtained positions of the objects correspond to the same object. That is, when the comparison circuit 252 determines that the above-described distance D exceeds the threshold value (a positive determination in step 402), it determines that the subject that the photographer is paying attention to has changed, and the selector 253 according to the determination result. Is the subject recognition means 11
Select the coordinates (Xa, Ya) obtained in step 1 (step 40).
3) It is sufficient to output as the subject position (Xo, Yo).

In FIG. 12, the read circuit 254 and the transfer circuit 255 form the control means 115.
In response to the determination result that the subject has been changed, the readout circuit 2
Numeral 54 reads out the output of the element indicated by the coordinates (Xa, Ya) from the CCD 231. Also, the transfer circuit 2
55 represents the coordinates (Xa, Ya) described above and the output of the element obtained by the readout circuit 254 as the coordinates (X
b, Yb) and spectral information R (Xb, Y
b), G (Xb, Yb), and B (Xb, Yb) are sent to the tracking means 112 (step 404) and stored in the coordinate storage unit 241 and the spectral information storage unit 242, respectively.

Therefore, thereafter, the tracking means 112
Based on the new subject position and spectral information described above,
Tracking processing of the position of the subject is performed. As a result, it is possible to detect that the photographer has changed the subject without fail, and when the subject changes, quickly obtain a new subject position and track the position of the new subject.

In step 302 described above,
Even when the center of the screen is set as the initial position of the subject irrespective of the position where the photographer is gazing, coordinates indicating the position of the correctly recognized subject are set by the processing of step 404 described above. The tracking processing of the position of the subject is performed. Step 401 or 40
In the case of a negative determination of 2, it is considered that the photographer does not intend to change the subject. Therefore, the selector 253 determines the coordinates (Xb, Yb) obtained by the tracking means 112 in response to the input of the determination result that the photographer is not gazing at the subject or the input of the determination result that the subject has not changed. Is selected (step 405) and output as the subject position (Xo, Yo).

If it is determined that the subject has not changed,
By setting the coordinates (Xb, Yb) obtained by the tracking means 112 as the subject position, it is possible to prevent the subject position from fluctuating more than necessary due to random movement of the photographer's eyeball, and to obtain a stable subject position. Becomes possible. In this way, it is possible to correctly recognize the position of the subject intended by the photographer and obtain a stable position of the subject, so that the obtained subject position is subjected to the exposure control process and the focus control process. Thereby, a good image of the subject can be taken.

In FIG. 5, the light transmitted through the main mirror 201 is bent by the half mirror 207,
It is configured to be incident on the focus determination unit 131. As shown in FIG. 14, the focus determining means 131 converts an image corresponding to the object field into two two-dimensional CCD image sensors (hereinafter simply referred to as CCDs) 264a and 264b by two separator lenses 263a and 263b. An image is formed on each of the CCDs 264a and 264b, and a correlation operation circuit 265 obtains an output correlation for an arbitrary portion of the CCDs 264a and 264b. The two CCDs 264 described above
As the a and 264b, similarly to the above-described CCD 224, a device having n × m elements may be used.

Further, the correlation operation circuit 265 responds to the input of the subject position obtained by the recognition processing section 250,
For a predetermined range including the subject position, the CCD 26
The correlation between the outputs 4a and 264b may be obtained, and the result of the correlation operation may be output as the result of determining the in-focus state of the subject.
For example, the correlation operation circuit 265 includes a CCD 26 corresponding to a region including p × q elements including the subject position (Xo, Yo).
4a and 264b are obtained, and the obtained correlation is used as a determination result of the in-focus state as the lens driving mechanism 2 shown in FIG.
62. Also, the lens driving mechanism 262
Is configured to drive the taking lens 210 to an in-focus position in accordance with the result of this determination.

Thus, the focus of the photographing lens 210 can be correctly adjusted in accordance with the movement of the subject intended by the photographer. In addition, the above-described recognition processing unit 250 correctly follows the movement of the subject and obtains a stable subject position. Therefore, the focus position of the photographing lens 210 does not change drastically, and focus control that follows the movement of the subject is not performed. And the focus control can be stabilized.

The exposure calculating means 121 is composed of a luminance calculator 271, a weight calculator 272, and an exposure value calculator 273. The brightness calculation unit 271 is provided with the CCD 23
1, the output R (X, Y), G (X, Y), B (X, Y) of each element is multiplied by predetermined constants K _1, K _2, K ₃ respectively, and BV = K ₁ × B ( X, Y) + K ₂ × G (X, Y) + K ₂ × R (X, Y) The luminance value BV of the element region of the object scene corresponding to each element is obtained.

The weight calculating section 272 performs a process described later to obtain a weight corresponding to each element region of the object scene. The exposure calculating section 273 uses the weight to perform the above-described processing. The luminance value of each element region is weighted and averaged, and the obtained result is sent to the mechanical control unit 274 as an exposure value. FIG. 15 shows a detailed configuration of the weight calculator 272.

In FIG. 15, a weight calculator 272 includes a distance calculator 275, a difference calculator 276, and a fitness calculator 27.
7 are formed. The distance calculation unit 275 receives the subject position (Xo, Yo), finds the distance dI between each element of the CCD 231 and the subject position (Xo, Yo), and sends the distance dI to the suitability calculation unit 275. ing. The output of the CCD 231 and the output of the above-described brightness calculation unit 271 are input to the difference calculation unit 276, and the difference information and the brightness value of the subject are subtracted from the spectrum information and the brightness value of the element region of the object scene. DR (X, Y) = R (Xo, Yo) -R (X, Y) dG (X, Y) = G (Xo, Yo) -G (X, Y) dB (X, Y) = B ( Xo, Yo) −B (X, Y) dBV (X, Y) = BV (Xa, Ya) −BV (X, Y) Differences dR, dG, dB, and dBV are obtained. .

The fitness calculating section 277 uses the distance dI and the differences dR, dG, dB, and dBV as parameters, and if dR (X, Y) = small and dG (X, Y) = small and dB (X , Y) = sma
ll and dI (X, Y) = nearand dBV (X, Y) = small then Apply the fuzzy rule expressed by S (X, Y) = big, and the object field indicated by coordinates (X, Y) Of each element area is within the area occupied by the subject in the object scene, and the fitness S (X, Y) is obtained, and the value of the fitness S (X, Y) is output as a weight. ing. As a membership function corresponding to the above parameter dBV, a membership function similar to each of the parameters dI, dR, dG, and dB shown in FIGS. 11 (a) to 11 (d) is used. What is necessary is just to find the conformity of the proposition.

In this case, the exposure value BVa obtained by the exposure calculator 273 is

[0057]

(Equation 1)

Exposure value of luminance information of each element area
The contribution to BVa is determined according to the magnitude of the value of the fitness S (X, Y). In other words, the luminance information of the element area which is considered to be almost certainly included in the area occupied by the subject is surely reflected in the exposure value, and conversely, the luminance information of the element area which is doubtful whether or not it is included in the area occupied by the subject Has a small contribution to the exposure value. As a result, it is possible to accurately evaluate the luminance information of the subject, so that an exposure value suitable for the subject can be obtained in accordance with a change in the luminance of the subject.

In this manner, the exposure value can be obtained based on the luminance information of the subject intended by the photographer, and the exposure can be controlled according to the exposure value. Note that the exposure calculation unit 121 may be configured to obtain an exposure value corresponding to the luminance of a predetermined area including the subject position (Xo, Yo). For example, the outputs of the element corresponding to the object position (Xo, Yo) and r × s elements around the element are read out from the CCD 231 and the average value of the luminance values obtained from these outputs is used as the luminance value of the object. What is necessary is just to obtain a value.

In this case, it is not necessary to perform a complicated fuzzy operation as described above, so that the exposure operation process can be sped up. On the other hand, there is a possibility that luminance information of an element region not included in the subject is reflected on the exposure value. In addition, a configuration may be adopted in which an element region in which the value of the above-described degree of conformity S (X, Y) is equal to or larger than a predetermined threshold is extracted, and an exposure value is obtained only from luminance information of the corresponding element region.

For example, when an object (cat) as shown in FIG. 16A is captured and coordinates (Xo, Yo) indicating the cat's forehead portion as the object position are obtained, The weight calculator 272 obtains a distribution of the values of the fitness S (X, Y) as shown by changing the type of hatching in FIG. In FIG. 16B, the unhatched portion indicates that the value of the fitness S (X, Y) is 0.3 or less.

For example, assuming that the above-described threshold value is 0.3, C in FIG. 16A corresponds to a portion surrounded by a solid line.
If the output of each element of the CD 231 is extracted, it is possible to extract the photometric result corresponding to the subject portion almost completely, and it is possible to obtain an exposure value reflecting the luminance information of the subject portion without fail. Based on the exposure value thus obtained, the mechanical control unit 274 shown in FIG. 5 controls the operation of the aperture 211 provided on the taking lens 210 and the operation of a shutter (not shown) in the same manner as in a conventional camera. It has become. The mechanical control unit 274 may be configured to control the opening of the diaphragm 211 according to the above-described exposure value, and to control the operation of the shutter during exposure to control the exposure time.

Here, since the stable subject position (Xo, Yo) is obtained by the recognition processing section 250, the stable exposure value BVa is obtained by the above-described exposure calculation section 273. Therefore, the size of the aperture of the stop 211 is not changed rapidly by the mechanical control unit 274, and the exposure process can be performed in accordance with the luminance change of the subject, and the exposure control can be stabilized.

For example, when the release button (not shown) is half-pressed by the photographer, the above-described subject recognition processing is performed, and according to the obtained subject position (Xo, Yo),
If the focus control and the exposure value calculation process described above are performed, the photographing lens 210 is always in focus on the subject, and an exposure value suitable for the subject is obtained. Is controlled. Therefore,
In response to the full press of the release button, the above-described mechanical control unit 273 controls the opening and closing operation of the shutter,
A good image of the subject can be obtained on an imaging medium such as a film.

Note that the tracking means 112 may track a position having a luminance similar to that of the subject. FIG.
FIG. 7 shows a main configuration of an embodiment of the subject recognition apparatus according to claim 3. In FIG. 17, a two-dimensional CCD image sensor (CCD) 233 and a brightness calculation unit 234 correspond to the brightness determination unit 116,
The image on the focus plate 203 is formed on the light receiving surface of the CCD 233. The CCD 233 has a configuration in which a filter for selectively transmitting the R, G, and B components is removed from the above-described CCD 231. The CCD 233 is formed of n × m elements, and reduces the light intensity in each element region of the object scene. It is configured to measure. The luminance calculator 234 is configured to calculate the luminance value of each element region of the object scene based on the output of the CCD 233.
The brightness value BV (X, Y) of each element region obtained in step 4
12 is sent.

In response, the tracking means 112 calculates the difference dBV between the luminance value of each element region of the object scene and the luminance value of the subject by the difference calculation section 244, and the discrimination processing section 245
Based on the distance dI and the difference dBV, an element region having a luminance value close to the luminance value of the subject and close to the position of the subject may be determined. Therefore, in this case, the discrimination processing unit 24
5, the number of parameters required for the fuzzy calculation processing is reduced, so that the time required for the subject tracking processing can be reduced. In addition, since the configuration of the CCD 233, which is a photometric element, is simpler than that of the color determination means 117, The object recognition device can be reduced in size and cost.

A luminance value is calculated from the spectral information obtained by the above-described color discriminating means 117, and the tracking means 112 applies the above-mentioned fuzzy rule by adding the difference of the luminance value to the parameter together with the difference of the spectral information. Alternatively, a configuration may be employed in which a position having a color similar to a subject and having similar luminance is determined. Also, the focus determination means 131
The position of the subject may be determined based on the intensity distribution of the light in the object scene captured by the two CCDs 264a and 264b provided in the.

In this case, as shown in FIG. 18, the above-described CCDs 264a,
The tracking means 112 is provided with a distance discriminating unit 246 for discriminating the distance L (X, Y) between the object captured in each element region of the object field and the photographing lens 210 from the correlation of the output of the 64b and calculating the difference. The unit 244 obtains a difference dL (X, Y) between the distance L (Xb, Yb) to the subject and the distance L (X, Y) corresponding to each element region. Based on dI (X, Y) obtained in and the difference dL (X, Y), an object at a distance close to the distance L (Xb, Yb) to the subject is captured, and the position of the subject ( What is necessary is just to determine the position of the element area indicated by the coordinates near Xb, Yb).

In the above-described embodiment, the case where the present invention is applied to a single-lens reflex camera has been described.
The present invention is not limited to a single-lens reflex camera, but may be applied to a compact camera, a movie camera, or the like. In short, the tracking unit performs the above-described tracking processing based on information about the intensity distribution of light in an object scene, and Any configuration can be applied as long as tracking is performed.

The present invention is not limited to a configuration in which information relating to the intensity distribution of light in an object field is obtained from photometric means provided in a finder system. For example, in a movie camera or an electronic still camera, a CCD image sensor as an image pickup medium is used. It may be configured to be used as photometric means.

[0071]

As described above, in the subject recognition apparatus of the present invention, the subject recognition means determines the position where the photographer is gazing as the position of the subject, and the determination means and the selection means perform the subject recognition. By selecting one of the two positions according to whether the position of the subject obtained by the means and the position of the subject obtained by the tracking means coincide with each other, the photographer can correctly recognize the position of the subject intended by the photographer and obtain a stable image. The subject position can be obtained and provided to the exposure control process and the focus control process.

In particular, by performing the tracking process using the position obtained by the subject recognition means as the initial position of the subject, the correct subject can be tracked from the beginning. Further, the tracking means can reliably track the subject by determining a position having a color similar to the subject, and can easily perform the tracking process by determining a position having a brightness similar to the subject. Thus, the size and cost of the object recognition device can be reduced.

Since a stable subject position can be obtained by the above-described subject recognizing device, an exposure value is calculated based on luminance information at the subject position, and exposure control is performed. The control can be performed, and the exposure control can be stabilized. Similarly, by providing the focus control with the determination result of the in-focus state of the optical image corresponding to the subject indicated by the above-described subject position, focus control following the movement of the subject is enabled, and the focus control is stabilized. Can be achieved.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a subject recognition device according to claims 1 and 2;

FIG. 2 is a diagram showing a configuration of a subject recognition device according to a third embodiment.

FIG. 3 is a diagram showing a configuration of a subject recognition device according to claim 4;

FIG. 4 is a diagram showing a configuration of a camera according to claims 5 and 6;

FIG. 5 is a configuration diagram of an embodiment of a single-lens reflex camera to which the invention of claim 1 is applied.

FIG. 6 is a diagram showing a detailed configuration of a subject recognizing unit.

FIG. 7 is a diagram illustrating movement of a line-of-sight position.

FIG. 8 is a diagram illustrating spectral characteristics of a filter.

FIG. 9 is a diagram showing a detailed configuration of a tracking unit.

FIG. 10 is a flowchart illustrating a tracking process.

FIG. 11 is a diagram showing a membership function.

FIG. 12 is a diagram illustrating a detailed configuration of a recognition processing unit.

FIG. 13 is a flowchart illustrating a process of determining a subject position.

FIG. 14 is a diagram illustrating a detailed configuration of a focus determination unit.

FIG. 15 is a detailed configuration diagram of a weight calculation unit.

FIG. 16 is an explanatory diagram of a subject portion extraction process.

FIG. 17 is a diagram showing a configuration of a main part of an embodiment of a subject recognition apparatus according to claim 3;

FIG. 18 is a diagram showing a main part configuration of another embodiment of the subject recognition device of claim 1;

FIG. 19 is a diagram illustrating a configuration of a camera having a conventional line-of-sight detection unit.

[Explanation of symbols]

111 Object Recognition Unit 112 Tracking Unit 113 Judgment Unit 114 Selection Unit 115 Control Unit 116 Luminance Judgment Unit 117 Color Judgment Unit 121 Exposure Calculation Unit 131 Focus Judgment Unit 201 Main Mirror 202 Penta Prism 203 Focusing Plate 204 Eyepiece 205 Beam Splitter 206, 223 lens 207, 222 half mirror 210, 610 photographing lens 211, 612 aperture 221 light emitting element 224, 231, 233, 264, 615 two-dimensional CC
D image sensor (CCD) 225 Detection processing circuit 226 Eye tracking unit 227 Judgment circuit 228 Position calculation unit 234,271 Brightness calculation unit 241 Coordinate storage unit 242 Spectral information storage unit 243,275 Distance calculation unit 244,276 Difference calculation unit 245 Processing unit 246 Distance discrimination unit 250 Recognition processing unit 251 Distance calculation circuit 252 Comparison circuit 253 Selector 254 Read circuit 255 Transfer circuit 262 Lens drive mechanism 263 Separator lens 265 Correlation calculation circuit 272 Weight calculation unit 273 Exposure value calculation unit 274 Mechanical control unit 277 Fitness calculation unit 614 Image pickup circuit 620 Automatic focus (AF) control circuit 622 Automatic exposure (AE) control circuit 624 Lens drive unit 626 Aperture drive unit 628 Clock circuit 630 Eye movement detector 634 Gate control circuit

Claims (6)

(57) [Claims] 1. A subject recognizing means for obtaining, as a position of a subject, a position at which a photographer is gazing within a field of a camera from movement of an eyeball of the photographer, and information on an intensity distribution of light in the field. A position corresponding to the subject is determined based on the following, a tracking unit that tracks this position, and a position obtained by the subject recognition unit and a position obtained by the tracking unit are compared. Determining means for determining whether or not the subject has been changed; selecting means for selecting any of the positions obtained by the subject recognizing means and the tracking means as a subject position in accordance with the determination result by the determining means In accordance with the determination result that the subject has been changed, the position obtained by the subject recognizing unit is sent to the tracking unit as information on a new subject to control the tracking process. Object recognition apparatus characterized by comprising a control means. 2. The object recognition device according to claim 1, wherein the tracking unit sets a position obtained by the object recognition unit as an initial position of the object, and determines the position of the object based on light information at the initial position. An object recognizing device having a configuration for determining a corresponding position. 3. The luminance discriminating device according to claim 1, wherein the luminance at each position in the object field is determined from the intensity distribution of light in the object field, and is used for a tracking process by the tracking unit. An object recognizing device, characterized in that the tracking means determines a position having a luminance similar to that of the object and tracks this position. 4. The object recognizing device according to claim 1, wherein a color at each position of the object field is determined from a light intensity distribution in the object field, and is provided to a tracking process by the tracking unit. An object recognizing device, characterized in that the tracking means determines a position having a color similar to the object and tracks this position. 5. An object recognition apparatus according to claim 1, wherein information relating to an intensity distribution of light in an object scene and an object position selected by said selection means are inputted, and luminance indicating brightness at said object position is input. A camera comprising: an exposure calculating unit that obtains an exposure value using information and performs exposure control. 6. An object recognition apparatus according to claim 1, wherein information on an intensity distribution of light in an object scene and an object position selected by said selection means are input, and correspond to an object indicated by said object position. And a focus determining means for determining a focus state of the light image to be subjected to focus control.