JPH04282989A - Automatic tracking device - Google Patents

Abstract

PURPOSE:To track an object whose shape changes such as a person with respect to the automatic tracking device using a digital moving picture signal so as to track the object. CONSTITUTION:A person 19 is surrounded by an object frame 17, a color in the object frame 17 is detected and a color whose picture element number is largest is used for a representative color 18 of the object. The tracing of the object is obtained based on the distribution state (picture element number) of the representative color 18 for each frame. The distribution state of the representative color of a search range 15 is calculated for each frame and the object frame 17 is tracked to a position of the person.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic tracking device for tracking an object using a digital moving image signal.

0002

2. Description of the Related Art A method called template matching is generally used to find out which part of another image a certain part of an image corresponds to.

In template matching, as shown in FIG.
i, j), measure the degree of similarity between the template 101 and a partial pattern of the image frame 102 that overlaps with it, and use that value as the probability that the object 103 exists at the point (i, j). It is. In order to find the position of the target object 103, it is sufficient to perform this operation on all points in the image and find the position where the degree of dissimilarity is minimum. As a matching measure, there is something like (Equation 1).

0004

[Math 1]

[0005] Here, S indicates the domain of the template. R is the degree of dissimilarity between the template and the image frame, and the smaller this degree is, the better the degree of matching is.

[0006] Therefore, if template matching is performed for each frame, it is possible to track an object using a template in a moving image.

Next, apart from the template matching method,
A method of dividing an image signal into multiple density levels will be explained. The process of determining which degree of density level each pixel of an image frame is included in will be referred to as density conversion. For example, the input image signal as shown in Fig. 12 is RG.
In the case of three-dimensional data of B (Red, Green, Blue), if each signal is divided into 8 from 0 to the maximum value, the density level will be divided into 512 (8×8×8). Also, when expressing one pixel, for example, R: frequency 3, G: frequency 0, B: frequency 5, and the color of the pixel is RG
It is a composite of B at each frequency. Furthermore, a moving image is divided into regions using up to 512 colors.

[0008]

However, with the above configuration, it is extremely difficult to track objects such as people. In the case of the template matching method, as shown in FIG. 13(a), it is impossible for the person 107 to move to 108 without changing its shape;
), the shape will always change if it moves from 112 to 113. Since the template matching method calculates the degree of similarity on a pixel-by-pixel basis, matching accuracy deteriorates if the shape changes. If the dissimilarity R takes the minimum value at a position where the target object does not exist, the template 110 will move to an abnormal position such as 111, and tracking will fail.

Furthermore, although there is a conventional method of dividing the video content of an image frame into regions based on a plurality of density levels, there is no method of describing and tracking the characteristics of an object based on the density levels.

0010

[Means for Solving the Problems] Target frame setting means for surrounding a tracked object in a first frame of a digital moving image; target frame storage means for storing the position of the target frame after the first frame; and the digital moving image From the density conversion means for classifying the signal into density levels and the output result of the density conversion means,
Representative color determination storage means for determining a plurality of density levels within the target frame stored in the target frame storage means, and determining and storing a density level to be used for tracking as a representative color from among the density levels, and storing the density level in the target frame storage means. From the position of the target frame,
A search range determining means for determining the range in which the tracking target of the next frame can be tracked; and target frame searching means for searching for the position to which the tracking target object has moved and outputting the searched position to the target frame storage means.

[0011]

[Operation] According to the above configuration, within the search range, the area where the most pixels of the representative color are distributed is determined by calculating the number of pixels of the representative color existing within the target frame, and the target object is tracked. I do. Calculating the distribution of the number of pixels is
Since pixel position information is ignored, there is no information regarding shape. Therefore, it is possible to continue tracking an object such as a person whose shape changes but whose color information does not change.

0012

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a block diagram of an automatic tracking device according to a first embodiment of the present invention, and 1 is an image signal input terminal. Reference numeral 2 denotes a density conversion means for classifying the image signal obtained from the image signal input terminal 1 into density levels, 3 a target frame setting means for arbitrarily setting a target frame by the operator, and 4 a target for storing the position of the target frame. Frame storage means 5 determines and stores a color for tracking (representative color) from the position of the target frame and the density level determined by the density conversion means 2; 6 stores the color of the target frame for each frame; Search range determining means 7 determines a search range, and target frame searching means 7 searches for the moving position of the target frame within the search range based on the representative color.

In the automatic tracking device of the first embodiment configured as described above, first, the image signal of the first frame is input to the input terminal 1 pixel by pixel. The density conversion means 2 performs density conversion based on the image signal. Determining which degree of density level each pixel of an image frame is included in is called density conversion. As described in the conventional example, the input image signal as shown in FIG.
In the case of three-dimensional data (n, Blue), if each signal is divided into 8 from 0 to the maximum value, the density level will be divided into 512 (8×8×8). Also, when expressing one pixel, for example, R: frequency 3, G: frequency 0, B:
The frequency is 5, and the color of the pixel is a combination of RGB at each frequency.

In the first frame, the operator arbitrarily determines the target object 10 to be tracked as shown in FIG. 2 using the target frame setting means 3, and sets a rectangular target frame 9 around the target object. The target frame storage means 4 stores the position of the target frame. The representative color determination storage means 5 obtains the position of the target frame from the target frame storage means 4, and selects one density level as a representative color from among the plurality of density levels existing within the target frame 13 as shown in FIG. It is determined and stored as 12. For example, the one with the largest number of pixels is selected. The density level of the representative color is fixed after the first frame. The density level of the representative color is output to the target frame search means 7. Further, the search range determining means 6 obtains the position of the target frame 13 from the target frame storage means 4, and determines the search range 14 of the target frame of the next frame. For example, a certain range around the target frame is determined.

Next, the second frame will be explained. The image signal of the second frame is input to the input terminal 1 . The density conversion means 2 performs density conversion based on the image signal. In the previous frame, the search range determining means 6 has already determined the range in which the target frame is to be searched. Further, the density level of the representative color used for the search has already been determined by the representative color determination storage means 5.

The target frame search means 6 searches for the moving position of the target frame within the search range by determining the distribution state of the representative colors. As the target frame 16 is sequentially moved within the search range 15 as shown in FIG. 4, the number of pixels of the representative color is calculated for each position. Then, the position 17 of the target frame for which the maximum number of pixels, such as the representative color 18, is determined is set as the movement position of the target frame (position of the target object). The moved position is stored in the target frame storage means 4, and the previous position is updated. The search range determining means 6 again determines the range in which the target frame is searched from the movement position. The representative color determination storage means 5 does not newly determine a representative color, but stores the density level of the representative color obtained in the first frame, and continues to output it to the target frame search means 7. After the third frame, the process for the second frame is repeated.

As described above, according to this embodiment, the area in which the most pixels of the representative color are distributed within the search range is found by calculating the number of pixels of the representative color existing within the target frame. , to track the object. Calculating the distribution of the number of pixels means ignoring pixel position information, so information regarding the shape is lost. Therefore, for objects such as people whose shape changes but whose color information does not change, performance can be improved over conventional methods.

Furthermore, even if there is a difference in the size of the target frame and the size of the distribution of representative colors, the target frame will be slightly deviated by that amount, but tracking will not fail.

Furthermore, even if the target frame search means 7 performs template matching based only on the positions of pixels of representative colors,
Higher correlation than conventional template matching is obtained and tracking performance is good.

FIG. 5 shows a block diagram of an automatic tracking device according to a second embodiment of the present invention, and 20 is an image signal input terminal. 21 is a density conversion means for classifying the image signal obtained from the image signal input terminal 1 into color density levels; 22 is a target frame setting means for arbitrarily setting a target frame by the operator; and 23 is for storing the position of the target frame. Target frame storage means, 2
Candidate color determination storage means 4 determines a plurality of candidate colors for tracking from the position of the target frame and the density level determined by the density conversion means 21; and 25 determines one representative color to be used for tracking from among the candidate colors. Reference numeral 26 denotes a representative color determination means, search range determination means for determining the search range of the target frame for each frame, and target frame search means 27 determines the movement position of the target frame within the search range.

In the automatic tracking device of the second embodiment configured as described above, first, the image signal of the first frame is input to the input terminal 20. The density conversion means 21 performs density conversion within the target frame based on the image signal, as in the first embodiment.

In the first frame, the operator uses the object frame setting means 22 to select the object 10 to be tracked as shown in FIG.
is arbitrarily determined, and a rectangular target frame 9 is set around the target object. The target frame storage means 23 stores the position of the target frame. In the candidate color determination storage means 24, the target frame storage means 23
The position of the target frame is obtained from , and a plurality of density levels are extracted as candidate colors 31 from among the plurality of density levels existing within the target frame 29 in FIG. For example, a plurality of density levels in which the number of pixels existing within the target frame is a predetermined number or more are extracted. Further, candidate colors are stored in a tabular format as a set of density level and number of pixels, as shown in FIG. The density level of the candidate color is fixed after the first frame. The representative color determining means 25 selects one candidate color having the largest number of pixels as a representative color from among the candidate colors. The density level of the representative color is output to the target frame search means 27. Further, the search range determining means 26 obtains the position of the target frame 29 from the target frame storage means 23 and determines the search range 28 of the target frame of the next frame. For example, the search range determining means 26
sets a certain range around the target frame.

Next, the second frame will be explained. The image signal of the next frame is input to the input terminal 20 . The density conversion means 21 calculates the color density within the target frame based on the image signal. In the previous frame, the range in which the target frame is searched has already been determined by the search range determining means 26. Further, the density level of the representative color used for the search is also determined by the representative color determining means 25.

The target frame search means 27 searches for the movement position of the target frame within the search range by determining the distribution state of the representative colors. As shown in FIG. 4, the target frame 16 within the search range 15
While moving sequentially, the number of pixels of the representative color is calculated for each position. Then, the position 17 where the maximum number of pixels of the representative color 18 is determined is set as the movement position of the target frame (position of the target object). The position of the target frame is stored in the target frame storage means 23, and the previous position is updated. The search range determining means 26 again determines the range in which the target frame is searched from the target position. The candidate color determination storage means 24 does not newly update the density level of the candidate color.
The density level of the candidate color obtained in the first frame is stored, and the number of pixels of that density level is calculated and updated within the target frame. The representative color determining means 25 selects the density level with the largest number of pixels among the candidate colors, and outputs this to the target frame searching means 27 as the representative color. After the third frame, the process for the second frame is repeated.

As described above, according to this embodiment, within the search range, the area where the most pixels of the representative color are distributed is found by calculating the number of pixels of the representative color that exist within the target frame. , to track the object. Calculating the distribution of the number of pixels means ignoring pixel position information, so information regarding the shape is lost. Therefore, for objects such as people whose shape changes but whose color information does not change, performance can be improved over conventional methods. Furthermore, by having multiple representative colors as candidate colors, high-performance tracking that is more responsive to color changes is realized.

FIG. 8 shows a block diagram of an automatic tracking device according to a third embodiment of the present invention, and 32 is an image signal input terminal. 33 is an attribute conversion means for converting the attribute of the color signal from the image signal input terminal 32; 34 is a density conversion means for classifying the attribute-converted color into density levels; and 35 is an object for which the operator arbitrarily sets a target frame. frame setting means, 36
37 is a target frame storage means for storing the position of the target frame; 37 is a candidate color determination storage means for determining a plurality of candidate colors for tracking from the position of the target frame and the density level obtained by the density conversion means 34; and 38 is a candidate color determination storage means for determining a plurality of candidate colors for tracking. Representative color determining means for determining one representative color to be used for tracking from candidate colors; 39 is a search range determining means for determining the search range of the target frame for each frame; 40 is an object for determining the moving position of the target frame within the search range It is a frame search means.

In the automatic tracking device of the third embodiment configured as described above, first, a digital color image signal is input to the input terminal 32. For example, RGB (Red,
Three-dimensional data (green, blue) is input. Next, the attribute conversion means 33 converts the RGB data into HCY data and outputs it to the density conversion means 34. Here, H means hue, Y means brightness, and C means saturation. The hue is red, green,
Luminance is a measure of the brightness of a color such as yellow, and saturation is a measure of the vividness of a color. For example, the conversion formula is (Math. 2)
etc. are possible.

[0028]

[Math 2]

This is shown in FIG. The vertical direction represents brightness, the rotation angle represents hue, and the rotation radius represents saturation.

Next, the effectiveness of the invention will be explained using the density conversion means. In general natural scenery, when tracking, the R, G, and
All signals of B change. If the RGB signal changes,
Tracking using the memorized representative color will no longer be possible. On the other hand, among the HCY signals, the brightness (luminance Y) and vividness (chroma C) of the object itself are also affected by sunlight. However, the attributes (hue) of colors such as red and green are not easily affected by changes in the brightness of the object. Therefore, the concentration conversion means 34
Then, as shown in FIG. 10, for example, the density level is divided into 10 hues, 1 luminance, and 2 chroma, so that the hue is finely divided, and the brightness and chroma are coarsely divided. In this way, even if the brightness changes due to a change in light, the density level of the color classified by density conversion will not change. Therefore, if there is no change in the representative color, higher performance tracking is possible.

Other means realize tracking by the same processing as in the second embodiment. As described above, according to this embodiment, RGB digital color image signals are converted into HCY signals, and division is performed based on color density with emphasis on H (hue). Since the H signal is a stable signal that is not affected by light,
Tracking can be achieved even if the brightness of the target changes.

[0032]

[Effect of the invention] According to the present invention, within the search range,
Find the area where the most pixels of the representative color are distributed by calculating the number of pixels of the representative color that exist within the target frame,
Tracks the object. Calculating the distribution of the number of pixels almost completely ignores pixel position information, so almost no information regarding the shape is lost. Therefore, for objects such as people whose shape changes but whose color information does not change, performance can be improved over conventional methods.

Furthermore, by having a plurality of representative colors as candidate colors, more robust tracking can be realized in response to changes in color information.

Furthermore, the RGB digital color image signals are converted into HCY signals, and division is performed based on color density with emphasis on H (hue). Since the H signal is a stable signal that is not affected by light, tracking can be achieved even if the brightness of the object changes.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the operation of an automatic tracking device according to a first embodiment of the present invention.

FIG. 2 is a diagram in which an operator sets a target frame for a target object.

FIG. 3 is a diagram illustrating representative colors and search ranges.

FIG. 4 is a diagram illustrating a search procedure.

FIG. 5 is a block diagram showing the operation of an automatic tracking device according to a second embodiment of the present invention.

FIG. 6 is a diagram illustrating candidate colors.

FIG. 7 is a diagram illustrating candidate colors.

FIG. 8 is a block diagram showing the operation of an automatic tracking device according to a third embodiment of the present invention.

FIG. 9 is a diagram illustrating color attributes.

FIG. 10 is a diagram illustrating density conversion.

FIG. 11 is an explanatory diagram of a template.

FIG. 12 is a diagram showing color density conversion.

FIG. 13 is an explanatory diagram of a problem.

[Explanation of symbols]

1 Image signal input terminal 2 Concentration conversion means 3 Target frame setting means 4 Target frame storage means 5 Representative color determination storage means 6 Search range determining means 7 Target frame search means 13 16, 17 Target frame 11, 19 Object 12, 18 Representative colors 14, 15 Search range

Claims (4)

[Claims] 1. A target frame setting means for surrounding a tracked object in a first frame of a digital moving image;
target frame storage means for storing the position of the target frame; density conversion means for classifying the digital moving image signal into density levels; Find multiple concentration levels within the frame,
A representative color determination storage unit that determines and stores a density level to be used for tracking as a representative color from among the density levels and a target frame position stored in the target frame storage unit, determines the range in which the target to be tracked in the next frame can be tracked. A search range determining means is used to move the target frame within the search range, and the position to which the tracking target has moved is searched based on the number of pixels of the representative color existing within the target frame or the correlation between the pixel distribution state. and target frame searching means for outputting the searched position to the target frame storage means. 2. Target frame setting means for surrounding a tracking target object in a first frame of a digital moving image;
target frame storage means for storing the position of the target frame; density conversion means for classifying the digital moving image signal into density levels; candidate color storage means for determining a plurality of density levels within the frame as candidate colors and storing a combination of the density level of the candidate color and the number of pixels in the target frame; and the density of the candidate color stored in the candidate color storage means. From the level and number of pixels,
representative color determining means for determining a density level used for tracking as a representative color; and search range determining means for determining a range in which the target to be tracked in the next frame can be tracked from the position of the target frame stored in the target frame storage means. Then, the target frame is moved within the search range, and the position to which the tracking target has moved is searched based on the number of pixels of the representative color existing in the target frame or the correlation of the pixel distribution state, and the searched position is and target frame searching means for outputting the target frame to the target frame storage means. 3. Attribute converting means for converting a digital color moving image signal into three attributes of hue, saturation, and brightness on a pixel basis; and density for classifying the three attributes obtained by the attribute converting means by density level. The automatic tracking device according to claim 1 or 2, further comprising a converting means. 4. Moving the target frame within the search range determined by the search range determining means, searching for a position where the number of pixels of the representative color existing within the target frame is maximum, and using the searched position as the target frame. 3. The automatic tracking device according to claim 1, further comprising target frame searching means for outputting to frame storage means.