JP3075937B2 - Motion vector detection circuit and subject tracking camera device using the same - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion vector detecting circuit and a subject tracking camera apparatus using the same, and more particularly, to a motion vector detecting circuit used for a video camera for automatically tracking a subject and a subject using the same. The present invention relates to a tracking camera device.

[0002]

2. Description of the Related Art When photographing a moving subject on a screen, a technique for automatically tracking the subject by a camera includes:
There is a technique using color information in a video signal. For example, color information is detected from a video signal in a designated detection block, and the detected color information is stored as a specific color to be used for determination of a target subject. For example, in the related art disclosed in Japanese Patent Application Laid-Open No. 61-9084, this memory color is updated according to the environment of the subject, thereby preventing a malfunction of the subject tracking operation.

[0003]

However, in the above-mentioned prior art, the memory color is updated each time the environment of the subject changes, so that if the memory color is updated many times, the memory color is changed to the target object. There is a risk that it will be different from the color,
There is a problem that it becomes difficult to track a target subject.

[0004] Therefore, a main object of the present invention is to provide a novel motion vector detecting circuit. Another object of the present invention is to provide a subject tracking camera device that can favorably track a subject.

[0005]

According to a first aspect of the present invention, a plurality of detection blocks set in a motion vector detection area are obtained based on color information of a specific detection block in a field immediately after a tracking operation of a subject is started. First storage means for storing first color information data, second storage means for storing second color information data obtained based on color information of a specific detection block in an arbitrary field, and based on color information for each detection block Calculating means for obtaining third color information data for each detection block, detecting means for detecting a motion vector based on the first color information data or the second color information data and the third color information data, and first color information data Alternatively, by determining whether or not a motion vector can be detected based on the second color information data and the third color information data, the first color information data and the second color information data are determined. Comprising setting means for setting whether to use the detection of the motion vector one of the information data, a motion vector detecting circuit.

A second invention is a subject tracking camera device using the above-described motion vector detection circuit.

[0007]

First, the first color information data of the specific detection block in the field immediately after the start of the tracking operation of the subject is the first color information data.
It is stored in the storage means. Further, the third color information data for each detection block is obtained by the calculating means. Then, a correlation value included in the setting means is obtained based on the first color information data and the third color information data, and the comparison value is compared with a predetermined threshold value by the comparison means. When the correlation value is greater than the threshold value in all the detection blocks, the determination unit determines that the motion vector cannot be detected even by using the first color information data.

Then, the data used for detecting the motion vector is switched from the first color information data to the second color information data obtained in the field at that time. The second color information data is stored in the second storage means. Thereafter, the detecting means detects a motion vector using the second color information data and the third color information data, and at the same time, as described above,
Using the second color information data, the determination means determines whether or not a motion vector can be detected. If it is determined that the motion vector cannot be detected in the second color information data,
The second color information data is updated. That is, the second color information data obtained in the field at that time is stored in the second storage means.

When the determination means determines that the motion vector can be detected using the first color information data while the motion vector is detected using the second color information data, the detection means returns A motion vector is detected using the first color information data. The color information forming the first to third color information data includes three different types of color elements such as Y, RY, BY signals and Y, Cr, Cb signals. .

As described above, appropriate color information data is used for detecting a motion vector in consideration of a result of determination as to whether or not a motion vector can be detected. Such a motion vector detection circuit is used in a subject tracking camera device.

[0011]

According to the present invention, since a motion vector can be detected using appropriate color information data, a target subject can be detected and tracked satisfactorily regardless of changes in the environment. The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0012]

FIG. 1 shows an object tracking camera device 10 constructed using a motion vector detecting circuit 14. As shown in FIG. The subject tracking camera device 10 includes a camera 12 that images a subject and converts the image into a video signal. The video signal from the camera 12 is provided to the motion vector detection circuit 14. The motion vector detection circuit 14 is configured, for example, as shown in FIG.

The motion vector detecting circuit 14 shown in FIG.
An A / D converter 16 is included. The A / D converter 16 converts the given video signal into digital data for each pixel. This digital data is, for example, Y, RY, B
-Data obtained by digitizing the Y signal. The video signal is supplied to the H-SYNC separation circuit 18 and the horizontal address counter 20. Horizontal address counter 20
In, the number of pixels in the horizontal direction is counted according to the video signal, and is reset for each line based on the horizontal synchronization signal from the H-SYNC separation circuit. The output from the horizontal address counter 20 is applied to a horizontal decoder 22, which outputs six types of horizontal signals H0 to H.
3, H2 'and H3' are output.

The video signal is applied to a V-SYNC separation circuit 24 and a vertical address counter 26. In the vertical address counter 26, the number of pixels in the vertical direction is counted according to the video signal, and V-SYN
It is reset based on the vertical synchronization signal from the C separation circuit 24. The output from the vertical address counter 26 is applied to a vertical decoder 28, which outputs three types of vertical signals V0 to V2.

Horizontal signals H0 to H3, H2 ', H3'
And vertical signals V0 to V2 are, for example, as shown in FIG.
Or (C). FIG. 3 shows a screen 30 of the viewfinder 12a of the camera 12, and the motion vector detection area 32 displayed on the screen 30 is divided into, for example, 25 detection blocks 34 in this embodiment.

Further, the motion vector detecting circuit 14 uses the CP
U36. A tracking mode switch 38 for setting the subject tracking mode is connected to the CPU 36. When the tracking mode switch 38 is turned on, the CPU 36 outputs the initial field signal a in the field immediately after the start of the tracking operation of the subject, that is, in the initial field.

Further, a ROM 40 and a RAM 42 are connected to the CPU 36. The ROM 40 stores a program for controlling the operation of the motion vector detection circuit 14. In the RAM 42, a flag indicating whether or not a correlation value (described later) of each detection block 34 is greater than a threshold value, and a flag indicating whether to use the first representative color data or the second representative color data for detecting a motion vector. Is stored.

The digital data of the video signal is
Based on the horizontal signals H0 and H1, the vertical signals V0 and V1, and the initial field signal a, the latch circuit 4
4 latched. That is, the tracking mode switch 38
Is turned on, an initial field signal a is output to the AND gate 46 during the initial field period. When the horizontal signal H0 and the vertical signal V0 are output in this initial field, an enable signal is output to the latch circuit 44 via the OR gate 48. Then, the latch circuit 44
Then, the data of each pixel of the specific detection block 34a shown in FIG. 4A is latched based on the enable signal. In this embodiment, the detection block 34 at the center of the motion vector detection area 32 is used as the detection block 34a. As described above, the reason why the detection block 34a at the center of the motion vector detection area 32 is used as the detection block 34a is that the distance between the detection block 34 and the end of the motion vector detection area 32 is such that the distance is left and right, and upward and downward. For each is equal.

Since the initial field signal a is not output in the fields subsequent to the initial field, when the horizontal signal H1 and the vertical signal V1 are output, the latch circuit 44 is output via the AND gate 50 and the OR gate 48.
Output an enable signal. Then, the latch circuit 4
At 4, the data of each pixel of each of the 25 detection blocks 34 shown in FIG. 4B is latched based on the enable signal.

The data from the detection block (either the detection block 34a in FIG. 4A or the entire detection block 34 in FIG. 4B) corresponding to the enable signal is output from the latch circuit 44 to the detection block integration circuit 52. Given to. The operation of the detection block integration circuit 52 will be described with reference to FIG. The detection block 34 shown in FIG. 5 includes, for example, 8 × 8 = 64 pixels. As shown in FIG. 5, let the respective values of the three color elements constituting the color information of each pixel be y _ij , (ry) _ij , and (by) _ij .

Then, the integrated color data (color information data) of the color information of the pixels included in the specific detection block 34a, for example, the first representative color data and the second representative color data are obtained by Expressions 1 and 2, respectively. .

[0022]

(Equation 1)

[0023]

(Equation 2) Here, the first representative color data is integrated color data obtained in a field (initial field) immediately after the start of the tracking operation of the subject to which the initial field signal a is output.
The second representative color data is integrated color data obtained after the initial field and in any field where the signal b or c is output. And selector 5
4, when the initial field signal a is output,
The first representative color data is stored in the representative color memory 56 via the selector 54. If the signal b or c is output to the selector 54 in the fields subsequent to the initial field, the second representative color data is stored in the representative color memory 58 via the selector 54.

Further, in the fields subsequent to the initial field, the latch circuit 44 outputs the signal shown in FIG.
The pixel data of all 25 detection blocks 34 shown in FIG. In the detection block integrating circuit 52, 25 detection blocks 34
The color information is integrated every time, and integrated color data is obtained. 25
The integrated color data obtained by integrating the color information of the pixels for each of the detection blocks 34 is obtained by Expression 3.

[0025]

(Equation 3) At this time, since the initial field signal a is not output to the selector 54, the integrated color data of the number of detection blocks 34 (25 in this embodiment) obtained for each field after the initial field is The data is stored in the integrated color memory 60 via the selector 54 for each field.

The detection block integration circuit 52 includes, for example, counters A and B (not shown) for counting the number of pixels supplied from the latch circuit 44. Then, in the initial field, the counter A is 8 × 8 = 64.
Is counted, the detection block integrating circuit 52 outputs the first
Representative color data is output. Similarly, in each field after the initial field, the counter B is set to 8 × 8 × 25.
= 1600, the detection block integrating circuit 5
Two to twenty-five integrated color data are output. The second representative color data is selected from the integrated color data.

The selector 62 is controlled by the signal b. When the signal b is not output, only the first representative color data is selected. When the signal b is output, the first representative color data and the second representative color data are selected. Both data are selected. And AND connected to the latch circuit 64
Gates 66 are connected to horizontal signals H2 and H2 'and a vertical signal V2 as shown in FIG.
When 2 is given, the representative color data is latched. That is, when the signal b is not output to the selector 62, the first representative color data is latched when the horizontal signal H2 and the vertical signal V2 are output. When the signal b is being output to the selector 62, the first representative color data is latched when the horizontal signal H2 and the vertical signal V2 are output, and the second representative color data is output when the horizontal signal H2 'and the vertical signal V2 are output. The representative color data is latched. The latched representative color data is provided to the correlator 70.

The AND connected to the latch circuit 72
Gates 74 are connected to horizontal signals H3 and H3 'and a vertical signal V3 as shown in FIG.
When 2 is given, the integrated color data stored in the integrated color memory 60 is latched. In the latch circuit 72, FIG.
As shown in (C), at the timing when the horizontal signals H3 and H3 'are output, the integrated value data in the integrated color memory 60 is latched twice and applied to the correlator 70, respectively. Note that the horizontal signals H3 and H3 'are output for the number of detected blocks, and therefore, 25 signals are output in this embodiment.

In the correlator 70, the representative color data from the latch circuit 64 and the integrated color data from the latch circuit 72 are operated for each detection block 34 to obtain a correlation value. With respect to the first representative color data and the integrated color data from the latch circuit 72, the absolute difference between the integrated values of the respective color elements is calculated by _{Expression 4} , and the sum D _xy, that is, the first correlation value is calculated. You.

[0030]

D _xy = | Y _xy −Y 1 | + | (RY) _xy − (R
−Y) 1 | + | (B−Y) _xy − (B−Y) 1 | Similarly, with respect to the second representative color data and the integrated color data from the latch circuit 72, the respective color elements are calculated by the following equation (5). , And the sum D _xy, that is, the second correlation value is calculated.

[0031]

D _xy = | Y _xy −Y 2 | + | (R−Y) _xy − (R
−Y) 2 | + | (B−Y) _xy − (B−Y) 2 | Here, when the signal b is not output to the correlator 70, the first correlation value is obtained. When b is output, both the first and second correlation values are obtained. The correlation value is provided to the CPU 36.

The operation of the CPU 36 will be described later in detail, but the CPU 36 compares a correlation value obtained for each detection block with a predetermined threshold value. First, in the fields subsequent to the initial field, if the correlation value> threshold is satisfied in all the detection blocks 34, it is determined that the correlation between the image in the motion vector detection area 32 between the initial field and the current field is low. At this time, it is determined that the motion vector cannot be detected even if the first representative color data is used, the signal b is output, and the representative color data is switched. Then, a motion vector is detected using the second representative color data instead of the first representative color data.

In the field where the signal b is output, the second representative color data obtained by using the second representative color data is used.
The CPU 36 determines whether or not the correlation value satisfies the correlation value> threshold value for all the detection blocks 34. If this condition is satisfied, the motion vector detection area 32 between the field from which the second representative color data is obtained and the current field is obtained.
It is determined that the correlation between the images in the image is low. At this time, it is determined that the motion vector cannot be detected using the second representative color data, the signal c is output, and the representative color data is updated. Then, a motion vector is detected using the updated second representative color data.

In the field where the signal "b" is output, not only the second correlation value but also the first correlation value obtained by using the first representative color data in all the detection blocks. It is determined whether the threshold is satisfied. If the first correlation value does not satisfy this condition, it is determined that the correlation between the image in the motion vector detection region 32 between the initial field and the current field has increased. At this time, it is determined that the motion vector can be detected by using the first representative color data, and the signal b is not output. Then, the motion vector is detected again using the first representative color data.

As described above, the representative color data used for detecting the motion vector is set in accordance with the judgment by the CPU 36. When the signal b is not output, the first correlation value is supplied from the correlator 70, and when the signal b is output, the second correlation value is supplied to the motion vector generation circuit 78. In the motion vector generation circuit 78, the specific detection block 3 is located at the position of the detection block 34 where D _xy is minimum.
It is determined that 4a has moved, and a motion vector is obtained (see FIGS. 6A and 6B). Then, in the next field, as shown in FIG. 6C, the motion vector detection area 32 is moved by the motion vector. Also, the motion vector is output from the motion vector generation circuit 78 as shown in FIG.
Is output to the pan head control circuit 80 shown in FIG.

The head control circuit 80 drives the driving device 82 based on the given motion vector. The movement of the camera platform 84 is controlled in the horizontal direction, the vertical direction, and the like by the driving device 82, and the tracking operation of the camera 12 is controlled accordingly. That is, the motion vector becomes zero (see FIG. 6).
The broken line portion 32a shown in FIG.
2), the motion vector detection area 32 is always displayed on the screen 3
The attitude of the camera 12 is controlled so that the image is shot at the center of 0. In this way, the camera 12 automatically tracks the subject.

Such an operation is repeated for each field. Next, main operations of the subject tracking camera device 10 will be described with reference to FIGS. First, in step S1 shown in FIG. 7, a video signal is input, and the process proceeds to step S3. In step S3, it is determined whether or not the field is an initial field. If the tracking mode switch 38 is turned on and the field is immediately after the start of the tracking operation of the subject, the field is determined to be the initial field, and if it is a field after that, it is determined that the field is not the initial field.

If "YES" is determined in the step S3, the vertical signal V0 is output in the step S5, and the horizontal signal H0 is output in the step S7.
Go to 9. In step S9, the detection block integration circuit 52 calculates integrated color data of the color information of the detection block 34a. In step S11, it is determined whether the counter A in the detection block integration circuit 52 has reached a predetermined count value. You. In this embodiment, the count value is 64
Is determined. If "YES", the flow proceeds to step S13, where the integrated color data is stored in the representative color memory 56 as the first representative color data, and the flow proceeds to step S15. When steps S5, S7 and S11 are "NO", the process proceeds to step S15.

When the vertical signal V2 is output in step S15 and the horizontal signal H2 is output in step S17, the process proceeds to step S19. In step S19, the first representative color data stored in the representative color memory 56 is supplied to the correlator 7 via the selector 62 and the latch circuit 64.
0, and the process returns to step S1. If steps S15 and S17 are "NO", then step S1
Return to

Then, a video signal is input in step S1, and if step S3 is "NO", the process proceeds to step S21. When the vertical signal V1 is output in step S21 and the horizontal signal H1 is output in step S23, the process proceeds to step S25. In step S25, the detection block integration circuit 52 calculates the integrated color data of each of the 25 detection blocks 34, and then proceeds to step S27.
Proceed to. In step S27, it is determined whether or not the counter B in the detection block integrating circuit 52 has reached a predetermined count value. In this embodiment, 64 × 25 = 160
It is determined whether the value has become 0. If "YES", in step S29, the integrated color data is stored in the integrated color memory 60, and the process proceeds to step S31. Step S2
When each of steps S1, S23 and S27 is "NO", the operation proceeds to step S31.

When the vertical signal V2 is output in step S31 and the horizontal signals H3 and H3 'are output in step S33, the process proceeds to step S35. When steps S31 and S33 are "NO", the process returns to step S1. In step S35, the integrated color data stored in the integrated color memory 60 is provided to the correlator 70 via the latch circuit 72, and the process proceeds to step S37.

In step S37, the correlator 70 performs a correlation operation. In this embodiment, a correlation value is obtained for each of the 25 detection blocks 34. And step S3
Go to 9. In step S39, it is determined whether a motion vector is detected using the first representative color data, that is, the first correlation value. If “YES”, step S
At 41, it is determined whether or not the first correlation value> the threshold is satisfied for all the detection blocks 34. If “NO”, the motion vector is detected using the first representative color data as it is, and the processing ends. On the other hand, step S41 is "Y
If "ES", it is determined that the correlation between the image in the motion vector detection area 32 between the initial field and the current field is low. For example, the color designated in advance disappears from the video signal due to rotation of the subject or the like. At this time, the process proceeds to step S43, where the representative color data is switched, that is, the signal b is output, and the second representative color data of the current field is stored in the representative color memory 58 via the selector 54. After the next field, a motion vector is detected using the second representative color data, and the process ends.

On the other hand, when step S39 is "NO", the flow proceeds to step S45 shown in FIG. If "NO" in the step S39, the motion vector is detected by using the second representative color data, that is, the second correlation value, and the correlator 70 uses the first representative color data to detect the first correlation value. Is obtained, and the second correlation value is obtained using the second representative color data.

In step S45, it is determined whether or not the first correlation value> the threshold is satisfied in all the detection blocks 34. If "NO", that is, if there is the detection block 34 whose first correlation value is equal to or less than the threshold value, it is determined that the correlation between the image of the initial field and the image of the current field is high. For example, this corresponds to the case where the subject rotates and the color designated in advance returns to the video signal again. At this time, the process proceeds to step S47. In step S47, the signal b is no longer output, and after the next field, a motion vector is detected again using the first representative color data stored in the representative color memory 56, and the process ends.

On the other hand, if "YES" in the step S45, the process proceeds to a step S49. In step S49,
It is determined whether the second correlation value> the threshold is satisfied for all the detection blocks 34. If "YES", it is determined that the correlation between the image of the field from which the second representative color data is obtained and the current field is low, and the flow advances to step S51 to update the second representative color data. That is, the signal c is output, and the second representative color data of the current field is output to the selector 54.
Is written to the representative color memory 58 via the. Then, after the next field, a motion vector is detected using the updated second representative color data, and the process ends. On the other hand, if step S49 is “NO”, the second
The motion vector is detected using the representative color data as it is without updating, and the process ends.

As described above, according to this embodiment, even if the representative color designated in advance is lost, the representative color is switched to perform the object detection operation, and when the original representative color is detected again, the original representative color is restored. By returning to the color detection operation,
The camera 12 can satisfactorily track the target subject. In the above-described embodiment, Y,
Three color elements of RY and BY signals were used. This is effective when processing a video signal or the like. In addition, when processing signals of a primary color CCD or the like, R,
When processing a CCD or the like having a mosaic filter in a complementary color system using three color elements of G and B signals, Y, Cr
It is preferable to use three color components of (= R-2G) and Cb (= B-2G) signals.

Further, three color elements of R / Y, G / Y, and B / Y signals obtained by dividing the R, G, and B signals by the Y signal may be used. In this manner, by normalizing by dividing by the Y signal, it is possible to remove the influence of lightness due to a difference in illumination or the like, and if the hue and the saturation are the same, they can be regarded as the same color. The threshold value is set arbitrarily according to the required degree of correlation and the type of the color element constituting the color information.

In the above-described embodiment, the case where the motion vector detection circuit 14 is applied to the subject tracking camera device 10 has been described.
Needless to say, 4 can also be applied to a video camera having a camera shake correction device.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a subject tracking camera device according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of a motion vector detection circuit.

3A is an illustrative view showing a horizontal signal and a vertical signal for generating an enable signal of a latch circuit 44; FIG. 3B is a schematic diagram showing a horizontal signal and a vertical signal for generating an enable signal of a latch circuit 64; FIG. 4 is an illustrative view showing a signal,
FIG. 3C is an illustrative view showing a horizontal signal and a vertical signal for generating an enable signal of the latch circuit 72;

FIG. 4A is an illustrative view showing a specific detection block, and FIG. 4B is an illustrative view showing all detection blocks.

FIG. 5 is an illustrative view showing a detection block and pixels;

FIGS. 6A to 6C are illustrative views for explaining generation of a motion vector and an object tracking operation.

FIG. 7 is a flowchart showing main operations of this embodiment.

FIG. 8 is a flowchart showing a continuation of the operation in FIG. 7;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Camera tracking camera apparatus 12 ... Camera 14 ... Motion vector detection circuit 22 ... Horizontal decoder 28 ... Vertical decoder 32 ... Motion vector detection area 34 ... Detection block 34a ... Specific detection block 36 ... CPU 38 ... Tracking mode switch 44,64 , 72 ... Latch circuit 52 ... Detection block integration circuit 54, 62 ... Selector 56, 58 ... Representative color memory 60 ... Integration color memory 70 ... Correlator 78 ... Motion vector generation circuit 80 ... Pan head control circuit 82 ... Driving device 84 ... Head

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/232 G06T 7/20 H04N 7/18

Claims (10)

(57) [Claims] A first memory for storing a plurality of detection blocks set in a motion vector detection area, and first color information data obtained based on color information of a specific detection block in a field immediately after a tracking operation of a subject is started. Storage means; second storage means for storing second color information data obtained based on color information of the specific detection block in an arbitrary field; third storage means for each detection block based on color information for each detection block Calculating means for obtaining color information data; detecting means for detecting a motion vector based on the first color information data or the second color information data and the third color information data; and the first color information data or the first color information data. By determining whether or not a motion vector can be detected based on two-color information data and the third color information data, the first color information data is determined. A motion vector detecting circuit comprising: a setting unit that sets which of the data and the second color information data is used for detecting a motion vector. 2. The method according to claim 1, wherein when the motion vector cannot be detected using the first color information data, the setting means uses the second color information data instead of the first color information data to detect a motion vector. The motion vector detection circuit according to claim 1, which is used. 3. The motion vector detecting circuit according to claim 1, wherein said setting means updates said second color information data when it is impossible to detect a motion vector using said second color information data. 4. The method according to claim 1, wherein the setting unit detects a motion vector using the first color information data when the motion vector is detected using the second color information data.
4. The motion vector detecting circuit according to claim 1, wherein a motion vector is detected using the first color information data instead of the second color information data. 5. The correlation means for determining a correlation value based on the first color information data and the third color information data, or the second color information data and the third color information data, wherein the correlation means 5. The motion vector detection circuit according to claim 1, further comprising: a comparison unit configured to compare a value with a predetermined threshold; and a determination unit configured to determine whether a motion vector can be detected based on the comparison result. 6. . 6. The motion vector detection circuit according to claim 5, wherein said determination means determines that the detection of the motion vector is impossible when the correlation value is larger than the threshold value in all the detection blocks. 7. The motion vector detecting circuit according to claim 1, wherein said color information includes three different color elements. 8. The three color elements are Y, RY, B-
The motion vector detection circuit according to claim 7, wherein the motion vector detection circuit is a Y signal. 9. The motion vector detecting circuit according to claim 7, wherein said three types of color elements are Y, Cr, and Cb signals. 10. A subject tracking camera device using the motion vector detection circuit according to claim 1.