JPS614011A - Automatic tracking device of camera - Google Patents

Abstract

PURPOSE:To perform stable tracking operation regardless of variation in the brightness of illumination light by extracting the color difference signals and brightness signal of a subject as to a tracking visual field, and detecting the movement of the subject to be tracked on the basis of the signal obtained by normalizing the color difference signals with the brightness signal. CONSTITUTION:A tracked subject image is photodetected by an image pickup element (CCD)5 through a photographic optical system. The signal outputted from the element 5 through an image pickup element driving circuit 8 is inputted to a signal processing circuit 9. The circuit 9 outputs the color difference signals (R-Y) and (B-Y) and brightness signal Y to a color detecting circuit 12 through a tracking gate setting circuit (corresponding to the tracking visual field) 11 to extract normalized color signals (R-Y)/Y and (B-Y)/Y by the circuit 12, and they are stored in a memory 13. A signal Y, on the other hand, is inputted to a range finding gate setting circuit 16. The data stored in the memory 13 is compared with a newly extracted signal by a movement decision circuit 14 to detect whether a subject moves or nor, its direction, etc., from variation in the color of the background of the subject, thereby moving the tracking visual field through a tracking gate moving circuit 15 and the circuit 11.

Description

Detailed Description of the Invention (Technical Field) The present invention relates to an automatic tracking device for a moving subject in an automatic focus detection or automatic focus adjustment device for a camera, especially a video camera, and particularly relates to an automatic tracking device for a moving subject, regardless of changes in the brightness of illumination light. The present invention relates to means that can perform stable tracking operations.

(Background Art) Regarding an automatic focus detection device that uses a video signal from a video camera, for example, US Pat.
Many proposals have been made, such as Publication No. 172.

Regarding the so-called mountain-climbing control method, which is one of the methods mentioned in "2," refer to "Mountain-climbing reservo method" by Ishida et al. ``Automatic focus adjustment of television cameras'' by 1981, and a method that combines this mountain climbing control with a rear focus drive lens.
In the Technical Report of the Television Society of Japan on January 29th, Haka Hanban announced the method in detail as a ``contour detection autofocus method''.

By the way, in this type of device, the distance measurement field of view is fixed at the center of the photographing screen as shown in Figure 1 (A), so as shown in Figure 1 (B), the object to be focused on (hereinafter referred to as the target) is If the subject (in this example, a person) moves, an object at a different distance from the target subject (in this example, a house) will be in focus, and the target subject, the person, will become blurred. be. In addition, Figure 1 and Part 2 below
The figure shows a screen when distance measurement is performed using a camera equipped with an automatic focusing device with negligible difference.

In order to eliminate the above drawbacks, the applicant first sets a movable tracking field of view, extracts the features of the object to be ranged with respect to this tracking field of view, stores the extracted features,
The presence or absence of movement of the object is detected based on the memorized characteristics and the newly extracted characteristics of the object to be ranged, and the distance measurement field of view is tracked to the movement of the object according to the relative movement of the object. We have proposed an automatic tracking focus detection device that can be moved (patent application dated May 25, 1981, title of invention [Automatic tracking focus detection device]). It is desirable to be able to perform stable tracking operations regardless of changes in brightness.

(the purpose) ? =ty>:5M1N-t:, No. , - YayotfJ*H (1) @ll5 (1) Eliminate the drawbacks, automatically detect the movement of a moving subject, and move the distance measurement field of view to track the movement of the subject to detect focus or An object of the present invention is to provide an automatic tracking device that can perform stable tracking operations when performing focus adjustment regardless of changes in the brightness of illumination light.

(Explanation based on Examples) Hereinafter, the means taken in this invention to achieve the above object will be exemplified and explained with reference to FIGS. 2 to 9 and the like. The following explanation will be given in the following order: an outline of the automatic tracking focus detection function to which the present invention is applied, a signal normalization means in an embodiment of the present invention, and an overall configuration of the embodiment of the present invention.

(Outline of automatic tracking focus detection function to which this invention is applied) (
(Figs. 2 to 5) First, an overview of an example of the automatic tracking focus detection function to which the present invention is applied will be explained. When the subject moves toward the upper right of the screen while maintaining the same distance as shown in A), the tracking means described later automatically detects the subject's movement, and the distance measurement field of view is adjusted as shown in Fig. 2 (A). The object is tracked and moved, and focus detection or focus adjustment is performed at this movement position. That is, some parameter representing the feature of the subject, such as the color of the subject and the background, is extracted with respect to the tracking field of view set by the tracking means, this extracted feature is stored, and the newly extracted feature is combined with the stored feature. The tracking field of view is moved to follow the movement of the subject by detecting whether the subject is moving or not, and if the subject moves, the moving direction or moving position based on the subject's characteristics. As the field of view moves, the distance measurement field of view is moved in the same positional relationship. Therefore, FIG. 2 can also be regarded as showing the relationship between the movement of the subject and the movement of the tracking field of view. Note that the tracking field of view is one of the means for determining the movement of a subject, and is normally displayed on a finder screen or the like like a distance measurement field of view, and the subject is not observed through this. Furthermore, if the tracking field of view were to be displayed on the screen, the tracking field of view and the distance measurement field of view would be displayed in the same positional relationship on the screen as described above, but the sizes of these fields may vary depending on the tracking field of view or distance measurement field of view, as described above. Either field of view can be enlarged.

In Figure 2 (A), since the distances are the same, the focusing lens of the photographic lens is not adjusted;
), the subject moves to the upper right of the screen and the distance also changes, so the focusing lens moves according to the distance measurement results. Therefore, the size of the tracking field of view is changed by the tracking gate size determining means described later, and is always maintained at a size suitable for the subject, and focus detection or focus adjustment is performed in this state. Here, since the movement between the subject and the camera is relative, the above-mentioned tracking effect works not only when the camera is fixed and the subject moves, but also when the subject stops and the camera moves. Alternatively, it functions effectively when both move together, and the size of the tracking field of view can be adjusted not only when the subject distance changes, but also when the focal length of the lens changes.

In principle, the tracking field of view has a two-dimensional extent, but to simplify the explanation, it is shown in Fig. 3 (A).
Assume that the tracking field of view has a one-dimensional extension extending in the horizontal direction as shown in FIG. Also, the tracking field of view is A, B,
Suppose that it is divided into three parts (hereinafter each part is referred to as a pixel) of C. In order to construct a two-dimensional tracking field of view, for example, pixels positioned above and below pixel B or A, B, and C in the figure may be provided as the center.

The color difference signals (R-Y) and (B-Y) and the luminance signal Y are extracted from each pixel mentioned above, and these are processed by the means described later with reference to FIG. Y) and (B-Y) are each divided by the luminance signal Y and the normalized signals are stored in a memory to be described later. This stored value is used for each pixel A, B and C as (R, -Y/Y) and (B
-Y/Y) When plotted on orthogonal coordinates, it is displayed as shown in FIG. 4, for example. Note that FIG. 4 shows the case where the luminance of the illumination light is in a standard state, and the numerical examples described later are also the numerical values obtained in this state. 41st
In the figure, points A, B, and Co are respectively shown in Figure 3 (
It represents the signals obtained from each pixel of A, B, and C in A). Here, it is assumed that from one pixel B, a signal representing only the clothing of the subject, for example, is extracted, and from pixels A and C, a signal in which signals representing the clothing of the subject and the background are added together is extracted. Furthermore, assume that the background colors on the left and right sides of the subject in the figure are different. Therefore, points A and Co are at different positions on the color difference signal coordinates.

Next, when the subject shown in Fig. 3 (A) moves to the right within the screen as shown in Fig. 3 (B), the ratio of the subject to the background in pixels A and C changes, and as a result, the pixels The signals obtained from A and C change as shown in FIG. 5 A1 and C1, respectively. On the other hand, since pixel B remains within the object as shown in FIG. 3(B), the signal obtained from pixel B hardly changes if the clothing is almost monochromatic. Therefore, here, for simplicity, it is assumed that B1=B. In this case, as shown in FIG. 5, point C1 approaches point Bo (=Bt), and point A1 approaches point Bo (=Bt).
Bt), the line segment BIC1 becomes the line segment BoC,
The line segment AIB becomes larger than the line segment AOBO. Conversely, if the line segment BICI becomes larger than the line segment noco and the line segment AlB1 becomes smaller than the line segment AOB, it means that the subject is moving to the left in FIG. 3(B).

In this invention, the signals obtained from pixels A, B, and C are changed to (RY/Y) and (B-
Y/Y) AO, BO on the plane.

Automatically detects the relative movement of the subject using the phenomenon that changes from Co to A, B, and CI, respectively, and automatically moves the tracking field of view to follow the movement of the subject according to the detection result. It is something that makes you Note that specific means for that purpose will be described later with reference to FIGS. 5 to 7.

Next, the color difference signal (R-Y) and (B-Y
) is normalized by the luminance signal Y to detect the movement of the subject. In other words, it is possible to detect the movement of the subject using only color difference signals and achieve the purpose of automatic tracking, but in this case, when the brightness of the illumination light changes over time, even though the subject is not moving, Points on the color difference signal coordinates may change. For example, when the illumination light becomes dark, each point on the coordinates approaches the 0 point, and when the illumination light becomes bright, it moves away from the 0 point. On the other hand, in addition to the (R-Y) and (B-Y) signals from each pixel, the Y signal is also extracted, and (R-Y/Y) and (B-Y) are extracted.
Y/Y) (normalized signal), and detect the movement of the subject from the change in position on the (R-Y/Y) and (B-Y/Y) orthogonal coordinates of the point representing each pixel. If you do this,
It is possible to automatically compensate for changes in the brightness of illumination light and perform stable tracking operations.

Assuming that the background color is the same on both the left and right sides of the subject, when the subject moves to the right in Figure 3 (B) within the screen, the above point A1 is an extension of the line segment AoB. The point C1 will occupy a position on the line segment B o Co U2. This invention can be applied to either of the above cases.

(Signal normalization means in the embodiment of the present invention) (Fig. 5) Fig. 5 shows the creation of the normalized signals (R-Y/Y) and (B-Y/Y) in the embodiment of the invention. This figure shows an example of a circuit in which color difference signals (RY) and (B-Y
) are integrated by integration circuits 100a and 1oob, respectively, and are integrated by sample and hold (S/H) circuits 10ta and 101b.
are sampled and held by the divider 102a, respectively.
'.

102b. On the other hand, the input luminance signal Y is integrated by the integrating circuit 100C, and the input luminance signal Y is integrated by the integrating circuit 100C.
is sampled and held and input to the dividers 102a and 102b, and the normalized signals (R
-Y/Y) and (B-Y/Y) are output. These normalized signals are sent to A/D conversion circuits 103a and 1, respectively.
At step 03b, the signals are A/D converted and stored in memories 104a and 104b. As a variation of the circuit described above, divider 102a
, 102b is an A/D conversion circuit 103a.

” It is provided after the divider 103b, and after A/D converting the sampled and held Y signal, the divider 102a
.

Normalization processing may be performed in step 102b. Or divider 1
02a and 102b are an integrating circuit 100a.

1oob, and each of these dividers has (R
-Y) signal and Y signal and (B-Y) signal and Y signal may be input and normalized.

(Overall configuration of an embodiment of the present invention) (Figs. 6 to 9) Fig. 6 shows an embodiment of an automatic tracking device of the invention,
This example is an example in which the present invention is applied to an automatic tracking focus detection device for a video camera.

In FIG. 6, the photographing optical system consists of a focusing lens 1, a zoom lens 2, an aperture 3, and a relay lens 4, and a subject image is captured by an image sensor 5 (eg C, C).

The light is received on D, ). Reference numeral 6 denotes a clock signal generation circuit, whose output is frequency-divided by a frequency divider 7 to a required ratio, and this frequency-divided output is used by an image sensor drive circuit 8, a tracking gate setting circuit 11, and a ranging gate setting circuit 16, which will be described later. granted to. The image sensor 5 is driven by an image sensor drive circuit 8 to output a time-series signal, and this output is subjected to necessary synchronization signal synthesis, modulation, and correction processing in a signal processing circuit 9 to form an output video signal, for example, an NTSC signal. be done. These processes are
Since it is well known to those skilled in the art, detailed explanation thereof will be omitted.

Note that in the following description, it is assumed that the output video signal is an NTSC signal.

The signal processing circuit 9 simultaneously processes color difference signals (RY) and (
B-Y) and luminance signal Y to the tracking gate setting circuit 11 (
(corresponding to the tracking field of view), and also outputs the luminance signal Y to the ranging gate setting circuit 16. The output of the tracking gate setting circuit 11 is supplied to the color detection circuit 12 to extract the normalized color difference signal for the subject, which is input to the memory 13 via manual mechanical input means such as a switch (not shown). is memorized. Note that color detection circuit 1
2 includes the normalization means shown in FIG. 5 or described above as a modification thereof. The above processing is performed according to the average value during 1/60 second, which is the period of one field of the television signal, or during a period of several fields thereof. In the following, both will be explained as being processed in one field period.

In the next field, the newly extracted signal and the signal stored in the memory 13 are compared in the movement determination circuit 14, and the presence or absence of movement of the subject and the direction of movement of the subject are detected. If there is movement, the tracking gate setting circuit 11 is controlled by the gate movement circuit 15 to move the tracking field of view, perform the same calculation in the next field, and repeat the above process until tracking is completed. return.

When tracking is completed, the gate moving circuit 15 sets the ranging field of view set by the ranging gate setting circuit 16 to the same relative position as the tracking field of view, and the video signal within this ranging field of view (signal processing circuit 9) automatic focus adjustment (A
F) The circuit 17 performs focus detection by known means such as hill climbing control, and its output drives the motor M to control the position of the focusing lens l.

In FIG. 6, Pl is a position sensor that detects the absolute position of the focusing lens 1 (corresponding to the subject distance), and P2 is a position sensor that detects the absolute position of the zoom lens 2 (corresponding to the focal length). Based on these signals, the tracking gate size determining circuit 10 controls the tracking gate setting circuit 11 and the ranging gate setting circuit 16, and determines the sizes of the tracking field of view and the ranging field of view, respectively. Now, the focal length of the photographic lens is f, and the subject distance is R1.
The longitudinal dimension of the imaging surface is y, and the tracking field of view length (Fig. 3 (A
) is a, the length of the tracking field of view on the subject is W, and υ/y=, then k=fW
/Ry is given as an adult, and if W = 500mm, then = 0.3
It becomes 4. Here, y is an image sensor such as C, C, or D.

Since W is determined by the object to be tracked, if k is calculated by the tracking gate size determination circuit 10 from the output values of the position sensors P1 and P2 using the above formula, it will always be optimal for the object. A tracking field of view of the size can be obtained.

FIG. 7 shows details of the color detection circuit 12, memory 13, and movement determination circuit 14 described above, and shows the color difference signals (R -Y) and (B -Y) signals and the luminance signal Y, the distance calculation circuit 21 calculates (R-Y/Y) in FIG.
) and the length DAOoBO of the line segment AOB on the (B-Y/Y) orthogonal coordinates are determined and stored in the memory 22. DAl, B1 or DAl, BO are found in the same way from the signal of the next field. Here, for simplicity, B
, =B, then DAI·B1=DA1−BO, and the divider 23 calculates DAl, Bl/DAO,BO. This value is compared with the first threshold value set by the threshold setting device 24 in the comparison circuit 25, and if there is a change exceeding the threshold value, l'' is output to the movement determination circuit 14.Similarly, , DCl, B1/DCO, BO are calculated by the circuits from the distance calculation circuit 31 to the comparison circuit 35, and if there is a change in this that exceeds the second threshold, the comparison circuit 35 sends a signal to the movement determination circuit 14.
Output. To explain a specific numerical example, the fifth
In the example shown in the figure, both the first and second thresholds are set to 2.
As, DAl, Bl/DAO, BO=2.2°D c 1.
Since B 1 /D c O, B O = 0, 36, only the comparison circuit 25 outputs "1". In this case, the movement determination circuit 14 adjusts the gate setting timing for a predetermined period of time (for example, in the case of NTSC, l/l/horizontal scanning period).
125). Conversely, when only the comparison circuit 35 outputs 111 II, the movement determination circuit 14 generates a signal that advances the gate setting timing by the predetermined time. The latter case is when the subject moves to the left in FIG.

Therefore, in response to the output "I11" of the comparator circuit 25 or 35, the movement determination circuit 14 generates a signal that changes the gate setting timing, for example, by the above-mentioned predetermined time, and in response to this signal, the gate movement circuit 15 By controlling the gate setting circuits 11 and 16 as shown in FIG.
/Y) and (B-Y/Y) on the orthogonal coordinates DAl, B1/
By calculating DAO, BO and DCl, 81/DCO0BO, stable tracking operation can be performed regardless of changes in the brightness of the illumination light, as described above.

In the above embodiment, the normalization process is performed using the circuit shown in FIG.
The output (R-Y/Y) and (B-Y/Y) signals are processed by the distance calculation circuit 21.31 shown in FIG. Integration, sample hold and A/D conversion were carried out as follows (R-
Y).

(B - Y) and Y signals to the distance calculation circuit 21 of FIG.
31, here normalization processing and (RY/Y),
(BY/Y) Distance calculation may be performed on orthogonal coordinates. In addition, in FIGS. 8 and 9, circuits that have basically the same configuration and function as each circuit in FIGS. 4 and 7, respectively, are given the same symbols as in FIGS. 4 and 7. , and 103c and 104c in FIG. 8 indicate an A/D conversion circuit and memory for the Y signal, respectively.

In addition, in the above embodiment, two color difference signals (R-Y/Y) are used.
The movement of the subject is detected by (B-Y/Y), but the change in color signal from each pixel representing the characteristics of the subject and background due to the movement of the subject is mainly caused by one of the above two signals. If the two signals depend on each other, it is also possible to detect movement toward the subject using only one of the signals.

(Effect) As described above, according to the present invention, the tracked subject 1
,,,. -1ゎoh. Since it is equipped with a means for extracting the *4h-aqh-*n field and a means for detecting the movement of the tracked subject based on the signal obtained by normalizing the color difference signal with the brightness signal, changes in the brightness of the illumination light can be detected. Stable tracking operation can be performed regardless of the

[Brief explanation of drawings]

Figure 1 shows the relationship between the distance measurement field of view and the subject image in a conventional camera. Figure 1 (A) shows when the distance measurement field and the subject image match, and Figure 1 (B) shows the relationship between the distance measurement field of view and the subject image. Figure 2 shows the relationship between the distance measurement field of view and the subject image in a camera implementing this invention.
Figure 3 (A) is an explanatory diagram showing the case where the subject moves within the screen at the same distance, Figure 3 (B) is an explanatory diagram showing the case where the subject moves within the screen and the distance becomes farther away, and Figure 3 (A) )
is an explanatory diagram showing the relationship between the tracking field of view and the subject image when the tracking field of view is divided in an automatic tracking device embodying the present invention, and Figure CB) is an explanatory diagram showing the relationship between the tracking field of view and the subject image when the tracking field of view is divided in the automatic tracking device implementing the present invention, and Figure CB) is an explanatory diagram showing the relationship between the tracking field of view and the subject image in the case where the subject moves within the screen in Figure (A). The explanatory diagram, Figure 4, is similar to Figures 3 (A) and (B).
) is the signal obtained from each pixel of the tracking visual field shown in (R-Y
/Y) and (B-Y/Y) plane, and FIG.
FIG. 6 is a block diagram showing an example of a device for obtaining Y/Y) and (B-Y/Y) signals, FIG. 6 is a block diagram showing the overall configuration of an embodiment of the automatic tracking device of the present invention, and FIG. is the 6th
FIG. 8 is a block diagram showing the details of the main parts of the device shown in FIG. 8, and FIG. 8 is a block diagram of a device for processing the signal obtained from each pixel in FIG. 9 are block diagrams of main parts of another embodiment of the automatic tracking device of the present invention. Explanation of symbols 1: Focusing lens, 2: Zoom lens, 5: Image sensor,
8: Image sensor drive circuit, 9: Signal processing circuit, ll: Tracking gate setting circuit, 12: Color detection circuit, 13: Memory, 1
4: movement determination circuit, 15: gate movement circuit, 16: ranging gate setting circuit, 17: automatic focus adjustment circuit. Figure 7 Figure 8 Figure 9

Claims (1)

[Scope of Claims] Means for extracting a color difference signal and a luminance signal representing characteristics of a tracked subject with respect to the tracking field of view, and detecting movement of the tracked subject based on a signal obtained by normalizing the color difference signal with the luminance signal. an automatic tracking device in a camera, comprising: a means for doing so;